LINGO-Space: Language-Conditioned Incremental Grounding for Space
Dohyun Kim, Nayoung Oh, Deokmin Hwang, Daehyung Park
Abstract
We aim to solve the problem of spatially localizing composite instructions referring to space: space grounding. Compared to current instance grounding, space grounding is challenging due to the ill-posedness of identifying locations referred to by discrete expressions and the compositional ambiguity of referring expressions. Therefore, we propose a novel probabilistic space-grounding methodology (LINGO-Space) that accurately identifies a probabilistic distribution of space being referred to and incrementally updates it, given subsequent referring expressions leveraging configurable polar distributions. Our evaluations show that the estimation using polar distributions enables a robot to ground locations successfully through 20 table-top manipulation benchmark tests. We also show that updating the distribution helps the grounding method accurately narrow the referring space. We finally demonstrate the robustness of the space grounding with simulated manipulation and real quadruped robot navigation tasks. Code and videos are available at https://lingo-space.github.io .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on5
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- PaLM-E: An Embodied Multimodal Language ModelDanny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch et al.ICML 2023 · 2,601 citations
- Strategies for Pre-training Graph Neural NetworksWeihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik et al.ICLR 2020 · 1,744 citations
- Recipe for a General, Powerful, Scalable Graph TransformerLadislav Rampásek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu et al.NeurIPS 2022 · 1,216 citations
- LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language ModelsChan Hee Song, Brian M. Sadler, Jiaman Wu, Wei-Lun Chao et al.ICCV 2023 · 685 citations
Related papers
- Frame of Reference: Addressing the Challenges of Common Ground Representation in Situational DialogsBiswesh Mohapatra, Théo Charlot, Giovanni Duca, Mayank Palan et al.ACL Findings 2026
- Refer360: A Referring Expression Recognition Dataset in 360 ImagesVolkan Cirik, Taylor Berg-Kirkpatrick, Louis-Philippe MorencyACL 2020 · 11 citations
- Behavioral Analysis of Vision-and-Language Navigation AgentsZijiao Yang, Arjun Majumdar, Stefan LeeCVPR 2023
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and InteractionsHe Zhu, Quyu Kong, Kechun Xu, Xunlong Xia et al.CVPR 2025
- Naver: a Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic ReasoningZhixi Cai, Fucai Ke, Simindokht Jahangard, Maria Garcia de la Banda et al.ICCV 2025 · 1 citation
