Lune

ACL Findings2023

Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers

Philipp Sadler, Sherzod Hakimov, David Schlangen

Abstract

The ability to pick up on language signals in an ongoing interaction is crucial for future machine learning models to collaborate and interact with humans naturally. In this paper, we present an initial study that evaluates intraepisodic feedback given in a collaborative setting. We use a referential language game as a controllable example of a task-oriented collaborative joint activity. A teacher utters a referring expression generated by a well-known symbolic algorithm (the "Incremental Algorithm") as an initial instruction and then monitors the follower's actions to possibly intervene with intra-episodic feedback (which does not explicitly have to be requested). We frame this task as a reinforcement learning problem with sparse rewards and learn a follower policy for a heuristic teacher. Our results show that intra-episodic feedback allows the follower to generalize on aspects of scene complexity and performs better than providing only the initial statement.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TEACh: Task-Driven Embodied Agents That Chat cites Episodic Transformer for Vision-and-Language Navigation
  • Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions cites Episodic Transformer for Vision-and-Language Navigation
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. TEACh: Task-Driven Embodied Agents That ChatAishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange et al.AAAI 2022 · 251 citations
  2. Episodic Transformer for Vision-and-Language NavigationAlexander Pashevich, Cordelia Schmid, Chen SunICCV 2021 · 228 citations
  3. Improving Intrinsic Exploration with Language AbstractionsJesse Mu, Victor Zhong, Roberta Raileanu, Minqi Jiang et al.NeurIPS 2022 · 81 citations
  4. Continual Learning for Instruction Following from Realtime FeedbackAlane Suhr, Yoav ArtziNeurIPS 2023 · 27 citations
  5. Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future DirectionsJing Gu, Eliana Stefani, Qi Wu, Jesse Thomason et al.ACL 2022

Related papers

  1. Learning Rewards From Linguistic FeedbackTheodore R. Sumers, Mark K. Ho, Robert X. D. Hawkins, Karthik Narasimhan et al.AAAI 2021 · 67 citations
  2. Learning Multi-Object Positional Relationships via Emergent CommunicationYicheng Feng, Boshi An, Zongqing LuAAAI 2024 · 4 citations
  3. Emergent Communication of GeneralizationsJesse Mu, Noah D. GoodmanNeurIPS 2021 · 60 citations
  4. Intra-agent speech permits zero-shot task acquisitionChen Yan, Federico Carnevale, Petko Georgiev, Adam Santoro et al.NeurIPS 2022 · 10 citations
  5. On the interaction between supervision and self-play in emergent communicationRyan Lowe, Abhinav Gupta, Jakob N. Foerster, Douwe Kiela et al.ICLR 2020 · 30 citations
Dusk over the sea between two cliffs drawn in fine vertical lines