Bridging Facial Understanding and Animation via Language Models
Luchuan Song, Pinxin Liu, Haiyang Liu, Zhenchao Jin, Yolo Y. Tang, Zichong Xu, Susan Liang, Jing Bi, Jason J. Corso, Chenliang Xu
Abstract
tions capturing content, style, and dynamics; and (2) Lan-guage2Motion: given a prompt, the model synthesizes the corresponding sequence of 3D facial parameters via quantized motion tokens for downstream animation. Extensive experiments show that in this setting language models can both interpret and synthesize facial motion with strong generalization. To the best of our knowledge, this is the first work to cast facial-parameter modeling as a language problem, establishing a unified path for text-conditioned facial animation and motion understanding.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext fd8ee25a-b809-4ef8-92fe-cf0deed032eaBuilds on22
- SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 4,104 citations
- TSM: Temporal Shift Module for Efficient Video UnderstandingJi Lin, Chuang Gan, Song HanICCV 2019 · 2,049 citations
- DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object DetectionHao Zhang, Feng Li, Shilong Liu, Lei Zhang et al.ICLR 2023 · 753 citations
- 3D-LLM: Injecting the 3D World into Large Language ModelsYining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng et al.NeurIPS 2023 · 662 citations
- Generating Diverse and Natural 3D Human Motions from TextChuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang et al.CVPR 2022 · 462 citations
Related papers
- Can Language Models Learn to Listen?Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa et al.ICCV 2023 · 44 citations
- Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and ApplicationsKarren D. Yang, Anurag Ranjan, Jen-Hao Rick Chang, Raviteja Vemulapalli et al.CVPR 2024 · 13 citations
- SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face AnimationWenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang et al.CVPR 2023
- AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and FusionBeibei Jing, Youjia Zhang, Zikai Song, Junqing Yu et al.AAAI 2024 · 6 citations
- Model See Model Do: Speech-Driven Facial Animation with Style ControlYifang Pan, Karan Singh, Luiz Gustavo HafemannSIGGRAPH 2025 · 2 citations
