Lune

ICCV2025Top-tier venue

2HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videos

Marvin Heidinger, Snehal Jauhri, Vignesh Prasad, Georgia Chalvatzaki

Abstract

When interacting with objects, humans effectively reason about which regions of objects are viable for an intended action, i.e., the affordance regions of the object. They can also account for subtle differences in object regions based on the task to be performed and whether one or two hands need to be used. However, current visionbased affordance prediction methods often reduce the problem to naive object part segmentation. In this work, we propose a framework for extracting affordance data from human activity video datasets. Our extracted 2HANDS dataset contains precise object affordance region segmentations and affordance class-labels as narrations of the activity performed. The data also accounts for bimanual actions, i.e., two hands co-ordinating and interacting with one or more objects. We present a VLM-based affordance prediction model, 2HandedAfforder, trained on the dataset and demonstrate superior performance over baselines in affordance region segmentation for various activities. Finally, we show that our predicted affordance regions are actionable, i.e., can be used by an agent performing a task, through demonstration in robotic manipulation scenarios. Project-website: sites.google.com/view/2handedafforder Dataset Image type & source # Images Affordance data Annotation source Annotation type # Aff. classes # Obj. classes Class-labels Bimanual IIT-AFF [34] Exocentric [44] 8.8K Manually-labeled Masks 9 10 Explicit No AGD20K [31] Exo+Egocentric [3, 27] 23.8K Manually-labeled Heatmaps 36 50 Explicit No 3DOI [38] Exo+Egocentric [5, 39] 10K Manually-labeled Points 3 n.a. Explicit No ACP [12] Egocentric [5] 15K Auto-labeled Heatmaps n.a. n.a. none No VRB [1] Egocentric [5] 54K Auto-labeled Heatmaps n.a. n.a. none No 2HANDS Egocentric [5] 278K Auto-labeled Precise Masks 73 163 Narrations Yes

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • 3DAffordSplat: Efficient Affordance Reasoning with 3D Gaussians cites LoRA: Low-Rank Adaptation of Large Language Models
  • 3DAffordSplat: Efficient Affordance Reasoning with 3D Gaussians cites Segment Anything
  • 3DAffordSplat: Efficient Affordance Reasoning with 3D Gaussians cites Learning Transferable Visual Models From Natural Language Supervision
  • Segment Anything cites Learning Transferable Visual Models From Natural Language Supervision
  • Visual Instruction Tuning cites Learning Transferable Visual Models From Natural Language Supervision
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. 3DAffordSplat: Efficient Affordance Reasoning with 3D GaussiansZeming Wei, Junyi Lin, Yang Liu, Weixing Chen et al.ACM MM 2025 · 4 citations

Builds on23

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. Segment AnythingAlexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao et al.ICCV 2023 · 13,211 citations
  4. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  5. Ego4D: Around the World in 3, 000 Hours of Egocentric VideoKristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis et al.CVPR 2022 · 525 citations

Related papers

  1. DualAfford: Learning Collaborative Visual Affordance for Dual-gripper ManipulationYan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang et al.ICLR 2023 · 2 citations
  2. Understanding Human Hands in Contact at Internet ScaleDandan Shan, Jiaqi Geng, Michelle Shu, David F. FouheyCVPR 2020
  3. Multi-label affordance mapping from egocentric visionLorenzo Mur-Labadia, Josechu J. Guerrero, Ruben Martinez-CantinICCV 2023 · 26 citations
  4. Grounded Human-Object Interaction Hotspots From VideoTushar Nagarajan, Christoph Feichtenhofer, Kristen GraumanICCV 2019 · 194 citations
  5. Human Hands as Probes for Interactive Object UnderstandingMohit Goyal, Sahil Modi, Rishabh Goyal, Saurabh GuptaCVPR 2022 · 26 citations
Dusk over the sea between two cliffs drawn in fine vertical lines