Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigation
Badi Li, Renjie Lu, Yu Zhou, Jingke Meng, Wei-Shi Zheng
Abstract
The Object Goal Navigation (ObjectNav) task challenges agents to locate a specified object in an unseen environment by imagining unobserved regions of the scene. Prior approaches rely on deterministic and discriminative models to complete semantic maps, overlooking the inherent uncertainty in indoor layouts and limiting their ability to generalize to unseen environments. In this work, we propose GOAL, a generative flow-based framework that models the semantic distribution of indoor environments by bridging observed regions with LLM-enriched full-scene semantic maps. During training, spatial priors inferred from large language models (LLMs) are encoded as two-dimensional Gaussian fields and injected into target maps, distilling rich contextual knowledge into the flow model and enabling more generalizable completions. Extensive experiments demonstrate that GOAL achieves state-of-the-art performance on MP3D and Gibson, and shows strong generalization in transfer settings to HM3D. Codes and pretrained models are available at https://github.com/Badi-Li/GOAL.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext c1b219f3-ef0f-43be-9888-d47943528c40Cited by top-tier papers4
- TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object NavigationYiyao Wang, Sixian Zhang, Keming Zhang, Xinhang Song et al.CVPR 2026 · 4 citations
- Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and ReasoningSixian Zhang, Yiyao Wang, Xinhang Song, Keming Zhang et al.CVPR 2026 · 3 citations
- Plug-and-Play Label Map Diffusion for Universal Goal-Oriented NavigationZhixuan Shen, Yijie Zeng, Shengxiang Luo, Tianrui Li et al.ICML 2026
- Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied NavigationZhixuan Shen, Jiawei Du, Ziyu Guo, Han Luo et al.ICML 2026
Builds on42
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
Related papers
- Imagine Before Go: Self-Supervised Generative Map for Object Goal NavigationSixian Zhang, Xinyao Yu, Xinhang Song, Xiaohan Wang et al.CVPR 2024 · 14 citations
- Learning to Map for Active Semantic Goal NavigationGeorgios Georgakis, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh et al.ICLR 2022 · 105 citations
- PEANUT: Predicting and Navigating to Unseen TargetsAlbert J. Zhai, Shenlong WangICCV 2023 · 52 citations
- ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal EmbeddingsArjun Majumdar, Gunjan Aggarwal, Bhavika Devnani, Judy Hoffman et al.NeurIPS 2022 · 344 citations
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object NavigationZibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li et al.NeurIPS 2025 · 31 citations
