EgoRoC: Towards Egocentric Robotic Control via Task-Agnostic Visual Alignment
Wei Feng, Chi Zhang, Nan Li, Qian Zhang, Qi Zhang, Mingyan Li
摘要
Recent Vision-Language-Action (VLA) models map visualtextual inputs to robotic actions via end-to-end architectures, yet this approach entangles visual understanding with task-specific actions. This leads to an exhaustive collection of full operational sequences and parameter redundancy across tasks, while generic third-person camera setups require fine-tuning for different hardware due to implicit hand-eye assumptions. We argue that decoupling how robots see from how robots act is a missing primitive in VLA systems. We present EgoRoC, a plug-and-play egocentric alignment head that precedes any task policy and exposes only a thin 6-DoF pose interface. EgoRoC establishes task-agnostic viewpoint consistency from a wristmounted (first-person) camera and then alternates alignment with manipulation, while a diffusion-based online hand-eye module corrects the action in the end-effector frame for hardware-agnostic deployment. Trained once from static wrist-target image pairs with relative poses, rather than full manipulation trajectories, EgoRoC leaves downstream VLAs unchanged. By turning egocentric alignment into a reusable capability, EgoRoC reduces training redundancy, strengthens zero-shot cross-scene transfer, and scales across VLA backbones without manual calibration. Across simulation and real settings, attaching EgoRoC consistently boosts success rates, especially on long-horizon and out-of-distribution tasks, and improves data efficiency during fine-tuning.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper22
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu 等ICLR 2022 · 被引用 18,833 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 被引用 7,873 次
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech 等NeurIPS 2022 · 被引用 6,707 次
相关 Paper
- Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action PolicyTianyi Zhang, Haonan Duan, Haoran Hao, Yu Qiao 等AAAI 2026 · 被引用 5 次
- Cross-Hand Latent Representation for Vision-Language-Action ModelsGuangqi Jiang, Yutong Liang, Jianglong Ye, Jia-Yang Huang 等CVPR 2026 · 被引用 14 次
- Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent GuidanceYang Zhang, Chenwei Wang, Ouyang Lu, Yuan Zhao 等ICLR 2026 · 被引用 21 次
- Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAsJunhao Shi, Siyin Wang, Xiaopeng Yu, Li Ji 等ICML 2026
- Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic ForgettingAsher J. Hancock, Xindi Wu, Lihan Zha, Olga Russakovsky 等ICLR 2026 · 被引用 58 次
