Lune

ICML2026顶会

ManiSoft: Towards Vision-Language Manipulation for Soft Continuum Robotics

Ziyu Wei, Luting Wang, Chen GAO, Li Wen, Si Liu

摘要

Most existing vision-language manipulation research targets rigid robotic arms, whose fixed morphology limits adaptability in cluttered or confined spaces. Soft robotic arms offer an appealing alternative due to their deformability, but confront challenges such as unreliable proprioception and distributed low-level actuation. To investigate these challenges, we introduce ManiSoft, a benchmark for vision-language manipulation with soft arms. ManiSoft features a tailored simulator that couples realistic soft-body dynamics with contact-rich interactions via an elastic force constraint. On this basis, ManiSoft defines four tasks, each highlighting distinct aspects of deformable control, from basic end-effector coordination to obstacle avoidance. To support policy training and evaluation, ManiSoft includes an automated pipeline that generates 6,3006{,}300 diverse scenes and corresponding expert trajectories. To produce high-quality trajectories at scale, we first employ a high-level planner to decompose each task into a sequence of waypoints, followed by a low-level reinforcement learning policy that generates torque commands to track waypoints. Benchmarking three representative policy models shows relatively promising results in clean scenes but substantial performance drop under randomization. Visualization analysis indicates that failures stem primarily from inaccurate visual estimation of proprioceptive state and limited exploitation of deformability for adaptive obstacle avoiding. We anticipate ManiSoft to serve as a valuable testbed, bridging the gap between rigid and soft arms in the context of vision-language manipulation.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

可以从这些问题问起

Lune 可以查阅

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • Habitat 2.0: Training Home Assistants to Rearrange their Habitat 引用了 ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
  • RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation 引用了 ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
  • ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills 引用了 Habitat 2.0: Training Home Assistants to Rearrange their Habitat
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

它引用的 Paper6

  1. Habitat 2.0: Training Home Assistants to Rearrange their HabitatAndrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans 等NeurIPS 2021 · 826 次引用
  2. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic ManipulationTianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai 等ICML 2026 · 394 次引用
  3. ManiSkill2: A Unified Benchmark for Generalizable Manipulation SkillsJiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling 等ICLR 2023 · 21 次引用
  4. REVERIE: Remote Embodied Visual Referring Expression in Real Indoor EnvironmentsYuankai Qi, Qi Wu, Peter Anderson, Xin Wang 等CVPR 2020
  5. ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday TasksMohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk 等CVPR 2020

相关 Paper

  1. Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and LanguageQiwei Wu, Rui Zhang, Xin Xiang, Tao Li 等ICML 2026 · 2 次引用
  2. RobotArena ∞: Scalable Robot Benchmarking via Real-to-Sim TranslationYash Jangir, Yidi Zhang, Kashu Yamazaki, Chenyu Zhang 等ICLR 2026 · 22 次引用
  3. DLO-Lab: Benchmarking Deformable Linear Object Manipulations with Differentiable PhysicsJunyi Cao, Yian Wang, Ziyan Xiong, Chunru Lin 等ICML 2026
  4. SIMPACT: Simulation-Enabled Action Planning using Vision-Language ModelsHaowen Liu, Shaoxiong Yao, Haonan Chen, Jiawei Gao 等CVPR 2026 · 8 次引用
  5. DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable PhysicsZhiao Huang, Feng Chen, Yewen Pu, Chunru Lin 等NeurIPS 2023 · 8 次引用
黄昏的海面,两侧是细线勾勒的悬崖