ACL Findings2023
Abstract then Play: A Skill-centric Reinforcement Learning Framework for Text-based Games
Anjie Zhu, Peng-Fei Zhang, Yi Zhang, Zi Huang, Jie Shao
Abstract
Text-based games present an exciting test-bed for reinforcement learning algorithms in the natural language environment. In these adventure games, an agent must learn to interact with the environment through text in order to accomplish tasks, facing large and combinational action space as well as partial observability issues. However, existing solutions fail to decompose the task and abstract the action autonomously, which either pre-specify the subtasks or pretrain on the human gameplay dataset. In this work, we introduce a novel skill-centric reinforcement learning framework, which is capable of abstracting the action in an end-to-end manner. To learn a more disentangled skill, we focus on the informativeness and distinguishability of the skill in accordance with the information bottleneck principle. Specifically, we introduce a discriminator to enable the skill to reflect the trajectory and push their representations onto the unit hypersphere to distribute uniformly. Moreover, a self-predictive mechanism is employed to learn inverse and forward dynamics, and a self-recovery mechanism is leveraged to refine the action representation, thus resulting in a more comprehensive perception of dynamics and more effective representations of textual state and action. Empirical experiments are carried out on the Jericho environment and the results validate the superiority against state-of-the-art baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Language Model Adaption for Reinforcement Learning with Natural Language Action Space cites Graph Constrained Reinforcement Learning for Natural Language Action Spaces
- Language Model Adaption for Reinforcement Learning with Natural Language Action Space cites Learning Dynamic Belief Graphs to Generalize on Text-Based Games
- Language Model Adaption for Reinforcement Learning with Natural Language Action Space cites Keep CALM and Explore: Language Models for Action Generation in Text-based Games
- Graph Constrained Reinforcement Learning for Natural Language Action Spaces cites Interactive Fiction Games: A Colossal Adventure
- Learning Dynamic Belief Graphs to Generalize on Text-Based Games cites Interactive Fiction Games: A Colossal Adventure
- Learning Dynamic Belief Graphs to Generalize on Text-Based Games cites Graph Constrained Reinforcement Learning for Natural Language Action Spaces
- Keep CALM and Explore: Language Models for Action Generation in Text-based Games cites Interactive Fiction Games: A Colossal Adventure
- Keep CALM and Explore: Language Models for Action Generation in Text-based Games cites Graph Constrained Reinforcement Learning for Natural Language Action Spaces
Cited by top-tier papers1
Builds on11
- Interactive Fiction Games: A Colossal AdventureMatthew J. Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, Xingdi YuanAAAI 2020 · 242 citations
- Graph Constrained Reinforcement Learning for Natural Language Action SpacesPrithviraj Ammanabrolu, Matthew J. HausknechtICLR 2020 · 138 citations
- Hierarchical Reinforcement Learning by Discovering Intrinsic OptionsJesse Zhang, Haonan Yu, Wei XuICLR 2021 · 97 citations
- Learning Dynamic Belief Graphs to Generalize on Text-Based GamesAshutosh Adhikari, Xingdi Yuan, Marc-Alexandre Côté, Mikulas Zelinka et al.NeurIPS 2020 · 91 citations
- Keep CALM and Explore: Language Models for Action Generation in Text-based GamesShunyu Yao, Rohan Rao, Matthew J. Hausknecht, Karthik NarasimhanEMNLP 2020 · 67 citations
Related papers
- Multi-Stage Episodic Control for Strategic Exploration in Text GamesJens Tuyls, Shunyu Yao, Sham M. Kakade, Karthik NarasimhanICLR 2022 · 30 citations
- Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive FictionVishal Jain, William Fedus, Hugo Larochelle, Doina Precup et al.AAAI 2020 · 32 citations
- Dual-Scale World Memory for LLM Agents towards Hard-Exploration ProblemsMinsoo Kim, Seung-won HwangICLR 2026
- Unsupervised Skill Discovery with Bottleneck Option LearningJaekyeom Kim, Seohong Park, Gunhee KimICML 2021 · 39 citations
- Perceiving the World: Question-guided Reinforcement Learning for Text-based GamesYunqiu Xu, Meng Fang, Ling Chen, Yali Du et al.ACL 2022 · 22 citations
