ACL Findings2024
Concept-Best-Matching: Evaluating Compositionality In Emergent Communication
Abstract
Artificial agents that learn to communicate in order to accomplish a given task acquire communication protocols that are typically opaque to a human. A large body of work has attempted to evaluate the emergent communication via various evaluation measures, with compositionality featuring as a prominent desired trait. However, current evaluation procedures do not directly expose the compositionality of the emergent communication. We propose a procedure to assess the compositionality of emergent communication by finding the best-match between emerged words and natural language concepts. The best-match algorithm provides both a global score and a translationmap from emergent words to natural language concepts. To the best of our knowledge, it is the first time that such direct and interpretable mapping between emergent words and human concepts is provided.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Communicating in Emergent Language with an Induced Morphological Phrasebook cites Unsupervised Translation of Emergent Communication
- Communicating in Emergent Language with an Induced Morphological Phrasebook cites Compositionality and Generalization In Emergent Languages
- Unsupervised Translation of Emergent Communication cites Emergent Quantized Communication
- Unsupervised Translation of Emergent Communication cites Compositionality and Generalization In Emergent Languages
- Unsupervised Translation of Emergent Communication cites Linking Emergent and Natural Languages via Corpus Transfer
- Unsupervised Translation of Emergent Communication cites Iterated learning for emergent systematicity in VQA
- CtD: Composition through Decomposition in Emergent Communication cites Emergent Quantized Communication
- CtD: Composition through Decomposition in Emergent Communication cites Compositionality and Generalization In Emergent Languages
- CtD: Composition through Decomposition in Emergent Communication cites Iterated learning for emergent systematicity in VQA
- CtD: Composition through Decomposition in Emergent Communication cites Emergent Communication of Generalizations
- Emergent Quantized Communication cites Linking Emergent and Natural Languages via Corpus Transfer
- Emergent Quantized Communication cites Emergent Communication of Generalizations
- Linking Emergent and Natural Languages via Corpus Transfer cites Compositionality and Generalization In Emergent Languages
- Linking Emergent and Natural Languages via Corpus Transfer cites Emergent Communication of Generalizations
- Semantics and Spatiality of Emergent Communication cites Compositionality and Generalization In Emergent Languages
- Semantics and Spatiality of Emergent Communication cites Linking Emergent and Natural Languages via Corpus Transfer
- Semantics and Spatiality of Emergent Communication cites Emergent Communication of Generalizations
- One-to-Many Communication and Compositionality in Emergent Communication cites Compositionality and Generalization In Emergent Languages
- One-to-Many Communication and Compositionality in Emergent Communication cites Linking Emergent and Natural Languages via Corpus Transfer
- One-to-Many Communication and Compositionality in Emergent Communication cites Emergent Communication of Generalizations
- Iterated learning for emergent systematicity in VQA cites Compositionality and Generalization In Emergent Languages
- Emergent Communication of Generalizations cites Compositionality and Generalization In Emergent Languages
Cited by top-tier papers5
- Semantics and Spatiality of Emergent CommunicationRotem Ben Zion, Boaz Carmeli, Orr Paradise, Yonatan BelinkovNeurIPS 2024 · 5 citations
- Unsupervised Translation of Emergent CommunicationIdo Levy, Orr Paradise, Boaz Carmeli, Ron Meir et al.AAAI 2025 · 3 citations
- One-to-Many Communication and Compositionality in Emergent CommunicationHeeyoung LeeEMNLP 2024 · 3 citations
- Communicating in Emergent Language with an Induced Morphological PhrasebookBrendon Boldt, David R. MortensenACL 2026
- CtD: Composition through Decomposition in Emergent CommunicationBoaz Carmeli, Ron Meir, Yonatan BelinkovICLR 2025
Builds on5
- Emergent Communication of GeneralizationsJesse Mu, Noah D. GoodmanNeurIPS 2021 · 60 citations
- Compositionality and Generalization In Emergent LanguagesRahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux et al.ACL 2020 · 40 citations
- Iterated learning for emergent systematicity in VQAAnkit Vani, Max Schwarzer, Yuchen Lu, Eeshan Dhekane et al.ICLR 2021 · 27 citations
- Linking Emergent and Natural Languages via Corpus TransferShunyu Yao, Mo Yu, Yang Zhang, Karthik R. Narasimhan et al.ICLR 2022 · 20 citations
- Emergent Quantized CommunicationBoaz Carmeli, Ron Meir, Yonatan BelinkovAAAI 2023 · 10 citations
Related papers
- Speaking Your Language: Spatial Relationships in Interpretable Emergent CommunicationOlaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. NormanNeurIPS 2024 · 5 citations
- Multi-Agent Language Learning: Symbolic MappingYicheng Feng, Zongqing LuACL Findings 2023 · 1 citation
- Compositionality with Variation Reliably Emerges in Neural NetworksHenry Conklin, Kenny SmithICLR 2023
- Agents generalize to novel levels of abstraction by using adaptive linguistic strategiesKristina Kobrock, Xenia Ohmer, Elia Bruni, Nicole GotznerACL Findings 2025
- Compositional languages emerge in a neural iterated learning modelYi Ren, Shangmin Guo, Matthieu Labeau, Shay B. Cohen et al.ICLR 2020 · 111 citations
