Lune

ACL Findings2023

Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind

Ece Takmaz, Nicolo' Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernández

Abstract

Dialogue participants may have varying levels of knowledge about the topic under discussion. In such cases, it is essential for speakers to adapt their utterances by taking their audience into account. Yet, it is an open question how such adaptation can be modelled in computational agents. In this paper, we model a visually grounded referential game between a knowledgeable speaker and a listener with more limited visual and linguistic experience. Inspired by psycholinguistic theories, we endow our speaker with the ability to adapt its referring expressions via a simulation module that monitors the effectiveness of planned utterances from the listener's perspective. We propose an adaptation mechanism building on plug-andplay approaches to controlled language generation, where utterance generation is steered on the fly by the simulator without finetuning the speaker's underlying language model. Our results and analyses show that our approach is effective: the speaker's utterances become closer to the listener's domain of expertise, which leads to higher communicative success.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Plug and Play Language Models: A Simple Approach to Controlled Text Generation cites The Curious Case of Neural Text Degeneration
  • Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue cites Language Models are Few-Shot Learners
  • Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue cites The Curious Case of Neural Text Degeneration
  • Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference Tuning cites Language Models are Few-Shot Learners
  • Language Models are Few-Shot Learners cites The Curious Case of Neural Text Degeneration
  • Flamingo: a Visual Language Model for Few-Shot Learning cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and ChallengesBolei Ma, Yuting Li, Wei Zhou, Ziwei Gong et al.ACL 2025 · 28 citations
  2. Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for DialogueJian Wang, Chak Tou Leong, Jiashuo Wang, Dongding Lin et al.ACL 2024 · 4 citations
  3. Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference TuningShengguang Wu, Shusheng Yang, Zhenglun Chen, Qi SuEMNLP 2024 · 2 citations
  4. Evaluating Theory of (an uncertain) Mind: Predicting the Uncertain Beliefs of Others from Conversational CuesAnthony B. Sicilia, Malihe AlikhaniACL 2025 · 2 citations

Builds on14

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  3. Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
  4. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
  5. Plug and Play Language Models: A Simple Approach to Controlled Text GenerationSumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung et al.ICLR 2020 · 1,166 citations

Related papers

  1. Learning to Mediate Disparities Towards Pragmatic CommunicationYuwei Bao, Sayan Ghosh, Joyce ChaiACL 2022
  2. Grounding Language in Multi-Perspective Referential CommunicationZineng Tang, Lingjun Mao, Alane SuhrEMNLP 2024 · 1 citation
  3. Learning Multi-Object Positional Relationships via Emergent CommunicationYicheng Feng, Boshi An, Zongqing LuAAAI 2024 · 4 citations
  4. Reference-Centric Models for Grounded Collaborative DialogueDaniel Fried, Justin T. Chiu, Dan KleinEMNLP 2021 · 12 citations
  5. Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language LearningAngeliki Lazaridou, Anna Potapenko, Olivier TielemanACL 2020 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines