Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control
Zhe Li, Cheng Chi, Yangyang Wei, Boan Zhu, Tao Huang, Zhenguo Sun, Yibo Peng, Pengwei Wang, Zhongyuan Wang, Fangzhou Liu, Chang Xu, Shanghang Zhang
摘要
Humans intuitively move to sound, but current humanoid robots lack expressive improvisational capabilities, confined to predefined motions or sparse commands. Generating motion from audio and then retargeting it to robots relies on explicit motion reconstruction, leading to cascaded errors, high latency, and disjointed acoustic-actuation mapping. We propose RoboPerform, the first unified audio-to-locomotion framework that can directly generate music-driven dance and speech-driven co-speech gestures from audio. Guided by the core principle of "motion = content + style", the framework treats audio as implicit style signals and eliminates the need for explicit motion reconstruction. RoboPerform integrates a ResMoE teacher policy for adapting to diverse motion patterns and a diffusion-based student policy for audio style injection. This retargeting-free design ensures low latency and high fidelity. Experimental validation shows that RoboPerform achieves promising results in physical plausibility and audio alignment, successfully transforming robots into responsive freestyle performers capable of reacting to audio.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper3
- VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic ModelWenhao Li, Xiu Su, Yichao Cao, Hongyan Xu 等ICML 2026 · 被引用 13 次
- Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error RecoveryWenhao Li, Xiu Su, Dan Niu, Yichao Cao 等ICML 2026 · 被引用 8 次
- Action-Sketcher: From Reasoning to Action via Visual Sketches for Robotic ManipulationHuajie Tan, Peterson Co, Yijie Xu, Shanyu Rong 等CVPR 2026
它引用的顶会 Paper10
- Directly Denoising Diffusion ModelsDan Zhang, Jingjing Wang, Feng LuoICML 2024 · 被引用 11,724 次
- AMP: adversarial motion priors for stylized physics-based character controlXue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine 等SIGGRAPH 2021 · 被引用 392 次
- RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for RoboticsEnshen Zhou, Jingkun An, Cheng Chi, Yi Han 等NeurIPS 2025 · 被引用 159 次
- KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic SkillsWeiji Xie, Jinrui Han, Jiakun Zheng, Huanyu Li 等NeurIPS 2025 · 被引用 120 次
- FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance GenerationRonghui Li, Junfan Zhao, Yachao Zhang, Mingyang Su 等ICCV 2023 · 被引用 110 次
相关 Paper
- From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent GuidanceZhe Li, Yangyang Wei, Boan Zhu, Yibo Peng 等ICLR 2026 · 被引用 29 次
- Emotional Speech-Driven 3D Body Animation via Disentangled Latent DiffusionKiran Chhatre, Radek Danecek, Nikos Athanasiou, Giorgio Becherini 等CVPR 2024
- Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion ModelXu He, Qiaochu Huang, Zhensong Zhang, Zhiwei Lin 等CVPR 2024
- Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion ModelsSimon Alexanderson, Rajmund Nagy, Jonas Beskow, Gustav Eje HenterSIGGRAPH 2023 · 被引用 191 次
- VidSTR: Automatic Spatiotemporal Retargeting of Speech-Driven Video CompositionsJoshua Kong Yang, Mackenzie Leake, Jeff Huang, Stephen DiVerdiCHI 2025
