CVPR Findings2026
Teleoperation, Simulation, or Human Video? Data Utilization Law for Robot Manipulation
Chenhao Shi, Yichen Zhu, Junjie Wen, Yefei Chen, Ziang Liu, Faming Fang
Abstract
Teleoperation, simulation, and human video represent the three primary data sources for robotic manipulation. Teleoperation data offers high quality at a high collection cost, whereas simulation and human video data are cheaper to acquire but introduce significant embodiment gaps. This trade-off has sparked a debate in the robotics community about the most effective data types for training robot policies. To address this, we introduce a data utilization law for robotic manipulation, drawing an analogy from economics to establish a formal "exchange rate" across data types. We quantify data utilization by using a single real-world teleoperated demonstration as a base unit and then measuring the volume of other data (i.e., simulation or human video) required to achieve equivalent performance. Through a comprehensive investigation across three manipulation tasks-training Diffusion Policy and π 0 on over 10000 trajectories-we systematically analyze the interplay between teleoperation, simulated, and human video data. Our analysis reveals several key findings: 1) Simulation data generally improves model generalization, with an approximate exchange rate of 8 simulation samples providing the equivalent benefit of 1 teleoperated sample. 2) Human video tends to degrade in-domain model performance, where adding approximately 10 human video samples can negate the benefit of a single teleoperated data point. 3
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Ego4D: Around the World in 3, 000 Hours of Egocentric Video cites Language Models are Few-Shot Learners
- Learning Transferable Visual Models From Natural Language Supervision cites Language Models are Few-Shot Learners
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video cites Learning Transferable Visual Models From Natural Language Supervision
Builds on8
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Ego4D: Around the World in 3, 000 Hours of Egocentric VideoKristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis et al.CVPR 2022 · 525 citations
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic ManipulationTianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai et al.ICML 2026 · 394 citations
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric VideoRyan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu et al.ICLR 2026 · 248 citations
Related papers
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-TrainingShuo Cheng, Liqian Ma, Zhenyang Chen, Ajay Mandlekar et al.NeurIPS 2025 · 16 citations
- Dexterous Manipulation Transfer via Progressive Kinematic-Dynamic AlignmentWenbin Bai, Qiyu Chen, Xiangbo Lin, Jianwen Li et al.AAAI 2026
- RoboWheel: A Data Engine from Real-World Human Demonstrations for Cross-Embodiment Robotic LearningYuhong Zhang, Zihan Gao, Shengpeng Li, Ling-Hao Chen et al.CVPR 2026 · 11 citations
- GraphMimic: Graph-to-Graphs Generative Modeling from Videos for Policy LearningGuangyan Chen, Te Cui, Meiling Wang, Chengcai Yang et al.CVPR 2025
- H-RDT: Human Manipulation Enhanced Bimanual Robotic ManipulationHongzhe Bi, Lingxuan Wu, Tianwei Lin, Hengkai Tan et al.AAAI 2026 · 25 citations
