Lune

EMNLP Findings2022

Revisiting the Roles of "Text" in Text Games

Yi Gu, Shunyu Yao, Chuang Gan, Josh Tenenbaum, Mo Yu

Abstract

Text games present opportunities for natural language understanding (NLU) methods to tackle reinforcement learning (RL) challenges. However, recent work has questioned the necessity of NLU by showing random text hashes could perform decently. In this paper, we pursue a fine-grained investigation into the roles of text in the face of different RL challenges, and reconcile that semantic and non-semantic language representations could be complementary rather than contrasting. Concretely, we propose a simple scheme to extract relevant contextual information into an approximate state hash as extra input for an RNN-based text agent. Such a lightweight plug-in achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval, suggesting non-NLU methods might suffice to tackle the challenge of partial observability. However, if we remove RNN encoders and use approximate or even ground-truth state hash alone, the model performs miserably, which confirms the importance of semantic function approximation to tackle the challenge of combinatorially large observation and action spaces. Our findings and analysis provide new insights for designing better text game task setups and agents.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning cites Interactive Fiction Games: A Colossal Adventure
  • Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning cites Graph Constrained Reinforcement Learning for Natural Language Action Spaces
  • Graph Constrained Reinforcement Learning for Natural Language Action Spaces cites Interactive Fiction Games: A Colossal Adventure
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. When Reasoning Meets Information Aggregation: A Case Study with Sports NarrativesYebowen Hu, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang et al.EMNLP 2024 · 10 citations
  2. SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMsYebowen Hu, Kaiqiang Song, Sangwoo Cho, Xiaoyang Wang et al.ACL 2024 · 6 citations

Builds on3

  1. Interactive Fiction Games: A Colossal AdventureMatthew J. Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, Xingdi YuanAAAI 2020 · 242 citations
  2. Graph Constrained Reinforcement Learning for Natural Language Action SpacesPrithviraj Ammanabrolu, Matthew J. HausknechtICLR 2020 · 138 citations
  3. Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement LearningXiaoxiao Guo, Mo Yu, Yupeng Gao, Chuang Gan et al.EMNLP 2020 · 21 citations

Related papers

  1. Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement LearningSubhajit Chaudhury, Sarathkrishna Swaminathan, Daiki Kimura, Prithviraj Sen et al.ACL 2023 · 4 citations
  2. Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based GamesYunqiu Xu, Meng Fang, Ling Chen, Yali Du et al.NeurIPS 2020 · 48 citations
  3. Learning Knowledge Graph-based World Models of Textual EnvironmentsPrithviraj Ammanabrolu, Mark O. RiedlNeurIPS 2021 · 43 citations
  4. Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive FictionVishal Jain, William Fedus, Hugo Larochelle, Doina Precup et al.AAAI 2020 · 32 citations
  5. Eye of the Beholder: Improved Relation Generalization for Text-Based Reinforcement Learning AgentsKeerthiram Murugesan, Subhajit Chaudhury, Kartik TalamadupulaAAAI 2022 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines