Generating Human Motion in 3D Scenes from Text Descriptions
Zhi Cen, Huaijin Pi, Sida Peng, Zehong Shen, Minghui Yang, Shuai Zhu, Hujun Bao, Xiaowei Zhou
Abstract
2 Ant Group "Walk to the door." "Sit on the table that is near the sofa." "Walk to the table that is near the sofa." "Stand up from the table." Figure 1. Generating human motions in 3D scenes from text descriptions. Our method can generate human motions containing accurate human-object interactions in 3D scenes based on textural descriptions. Although our method is trained and tested on the HUMANISE dataset, it can generalize to other scenes, e.g., the scenes in the PROX dataset. Left: test results on the HUMANISE dataset. Right: generalization results on the PROX scenes.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext f9170c1d-cec4-46b1-bf80-87165c6b18c2Cited by top-tier papers36
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object InteractionSirui Xu, Ziyin Wang, Yu-Xiong Wang, Liangyan GuiNeurIPS 2024 · 78 citations
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated ObjectsHuaijin Pi, Zhi Cen, Zhiyang Dou, Taku KomuraNeurIPS 2025 · 14 citations
- EgoTwin: Dreaming Body and View in First PersonJingqiao Xiu, Fangzhou Hong, Yicong Li, Mengze Li et al.ICLR 2026 · 14 citations
- 🎧MOSPA: Human Motion Generation Driven by Spatial AudioShuyang Xu, Zhiyang Dou, Mingyi Shi, Liang Pan et al.NeurIPS 2025 · 13 citations
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion ModelYonghao Zhang, Qiang He, Yanguang Wan, Yinda Zhang et al.AAAI 2025 · 10 citations
Builds on54
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
Related papers
- HUMANISE: Language-conditioned Human Motion Generation in 3D ScenesZan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu et al.NeurIPS 2022 · 207 citations
- GenZI: Zero-Shot 3D Human-Scene Interaction GenerationLei Li, Angela DaiCVPR 2024
- Move as you Say, Interact as you can: Language-Guided Human Motion Generation with Scene AffordanceZan Wang, Yixin Chen, Baoxiong Jia, Puhao Li et al.CVPR 2024 · 38 citations
- MIME: Human-Aware 3D Scene GenerationHongwei Yi, Chun-Hao P. Huang, Shashank Tripathi, Lea Hering et al.CVPR 2023
- CG-HOI: Contact-Guided 3D Human-Object Interaction GenerationChristian Diller, Angela DaiCVPR 2024
