Lune

CVPR2025Top-tier venue

VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis

Enric Corona, Andrei Zanfir, Eduard Gabriel Bazavan, Nikos Kolotouros, Thiemo Alldieck, Cristian Sminchisescu

Abstract

Input Image Generated Video E. Corona et al. our method does not require training for each person, does not rely on face detection and cropping, generates the complete image (not just the face or the lips), and considers a broad spectrum of scenarios (e.g. visible torso or diverse subject identities) that are critical to correctly synthesize humans who communicate. We also curate MENTOR, a new and diverse dataset with 3d pose and expression annotations, one order of magnitude larger than previous ones (800,000 identities) and with dynamic gestures, on which we train and ablate our main technical contributions. VLOGGER outperforms state-of-the-art methods in three public benchmarks, considering image quality, identity preservation and temporal consistency while also generating upper-body gestures. We analyze the performance of VLOGGER with respect to multiple diversity metrics, showing that our architectural choices and the use of MENTOR beneőt training a fair and unbiased model at scale. Finally we show applications in video editing and personalization.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

13 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios cites Video Diffusion Models
  • SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios cites Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
  • Video Motion Graphs cites Adding Conditional Control to Text-to-Image Diffusion Models
  • Video Motion Graphs cites Video Diffusion Models
  • Video Motion Graphs cites A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
  • EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing cites Adding Conditional Control to Text-to-Image Diffusion Models
  • Adding Conditional Control to Text-to-Image Diffusion Models cites Diffusion Models Beat GANs on Image Synthesis
  • Adding Conditional Control to Text-to-Image Diffusion Models cites Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
  • Video Diffusion Models cites Diffusion Models Beat GANs on Image Synthesis
  • GRPose: Learning Graph Relations for Human Image Generation with Pose Priors cites Adding Conditional Control to Text-to-Image Diffusion Models
  • OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models cites Video Diffusion Models
  • Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding cites Diffusion Models Beat GANs on Image Synthesis
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers13

  1. GRPose: Learning Graph Relations for Human Image Generation with Pose PriorsXiangchen Yin, Donglin Di, Lei Fan, Hao Li et al.AAAI 2025 · 17 citations
  2. SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction ScenariosLingwei Dang, Ruizhi Shao, Hongwen Zhang, Wei Min et al.NeurIPS 2025 · 12 citations
  3. OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation ModelsGaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng et al.ICCV 2025 · 11 citations
  4. Video Motion GraphsHaiyang Liu, Zhan Xu, Fa-Ting Hong, Hsin-Ping Huang et al.ICCV 2025 · 6 citations
  5. EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video EditingYehonathan Litman, Shikun Liu, Dario Seyb, Nicholas Milef et al.CVPR 2026 · 5 citations

Builds on36

  1. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
  2. Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
  3. Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 6,759 citations
  4. Video Diffusion ModelsJonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan et al.NeurIPS 2022 · 2,948 citations
  5. A Lip Sync Expert Is All You Need for Speech to Lip Generation In the WildK. R. Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, C. V. JawaharACM MM 2020 · 869 citations

Related papers

  1. Monocular and Generalizable Gaussian Talking Head AnimationShengjie Gong, Haojie Li, Jiapeng Tang, Dongming Hu et al.CVPR 2025
  2. TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head SynthesisShunian Chen, Hejin Huang, Yexin Liu, Zihan Ye, Pengcheng Chen, Chenghao Zhu, Michael Guan, Rongsheng Wang, Junying Chen, Jianye Hou, Bo Li, Guanbin Li, Ser-Nam Lim, Harry Yang, Benyou WangCVPR Findings 2026
  3. WildActor: Unconstrained Identity-Preserving Video GenerationQin Guo, Tianyu Yang, Xuanhua He, Fei Shen et al.ICML 2026 · 4 citations
  4. Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal AnimationJiangning Zhang, junwei zhu, Zhenye Gan, Donghao Luo et al.CVPR 2026 · 4 citations
  5. ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video GenerationMingyang Wu, Ashirbad Mishra, Soumik Dey, Shuo Xing et al.CVPR 2026 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines