Video models
Kandinsky S2V 1.0: open speech-to-video avatar models
Sber's Kandinsky Lab has released Kandinsky S2V 1.0, a family of open models that turn one reference image and a speech track into a talking, gesturing video. The code and weights are MIT-licensed, with a 2B and a 21B version.
Kandinsky S2V 1.0 is an open family of speech-to-video models from Sber’s Kandinsky Lab. Give it one reference image, a speech recording and a text prompt, and it generates long-form 25 fps video in which lip movement, expression, pose and gesture follow the speech. The technical report was posted to arXiv on 13 October 2026.
Two sizes, plus fast versions
The family has Kandinsky S2V 1.0 Lite (2B parameters) and Kandinsky S2V 1.0 Pro (21B). Both start from the Kandinsky 5.0 Video foundation models. The repository says the whole figure is animated, not only the head, and that output comes in SD or HD at the reference image’s aspect ratio. Long speech is generated in five-second chunks.
There are also distilled variants using a method called π-Flow. The distilled Pro model generates video in 16 sampling steps without classifier-free guidance, a 5.4 times end-to-end speedup, while matching or exceeding the base model on nearly all reported benchmarks.
How it works
Three choices carry the results, according to the project page:
- One shared front end. Text and audio conditioning both come from a single Qwen2.5-Omni model, replacing the separate Wav2Vec or Whisper audio path most speech-to-video systems use.
- Three-scale visual memory. Past frames are packed at progressively coarser sizes, and the identity reference is kept far from the target in time, which the authors say suppresses pose copying over long videos.
- Face-weighted training. An area-adaptive face loss keeps lip-sync learning effective when the speaker fills only a small part of the frame.
The training set is 1.17 million single-speaker clips selected by active-speaker detection, captioned in English and Russian.
Results, access and limits
In the authors’ side-by-side human evaluation against the strongest open-source baseline and two proprietary commercial models, Pro performs “comparably to or better” on average and is preferred for lip synchronisation. These are the authors’ own tests.
Code, checkpoints and Diffusers integration are released under the MIT licence. The repository asks for one NVIDIA GPU with 48 GB or more (80 GB for HD), or about 28 GB with VAE tiling and faint tile seams. The repository’s demo space and Diffusers instructions were updated on 3 November, including short five-second clips for free ZeroGPU accounts.
Our take
Speech-driven faces are the missing last step for many conversational characters: the dialogue is written and voiced, but the face does not move. An open, permissively licensed model lets studios test that step locally, though 48 GB of GPU memory is far from real time on a phone. For intelligent characters it is a useful piece of the stack, not a finished one. See intelligent NPCs for the wider picture.