Lune

EMNLP Findings2020

Learning to Generalize for Sequential Decision Making

Xusen Yin, Ralph M. Weischedel, Jonathan May

Abstract

We consider problems of making sequences of decisions to accomplish tasks, interacting via the medium of language. These problems are often tackled with reinforcement learning approaches. We find that these models do not generalize well when applied to novel task domains. However, the large amount of computation necessary to adequately train and explore the search space of sequential decision making, under a reinforcement learning paradigm, precludes the inclusion of large contextualized language models, which might otherwise enable the desired generalization ability. We introduce a teacher-student imitation learning methodology and a means of converting a reinforcement learning model into a natural language understanding model. Together, these methodologies enable the introduction of contextualized language models into the sequential decision making problem space. We show that models can learn faster and generalize more, leveraging both the imitation learning and the reformulation. Our models exceed teacher performance on various held-out decision problems, by up to 7% on in-domain problems and 24% on out-of-domain problems.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on2

  1. LeDeepChef Deep Reinforcement Learning Agent for Families of Text-Based GamesLeonard Adolphs, Thomas HofmannAAAI 2020 · 48 citations
  2. Algorithmic Improvements for Deep Reinforcement Learning Applied to Interactive FictionVishal Jain, William Fedus, Hugo Larochelle, Doina Precup et al.AAAI 2020 · 32 citations

Related papers

  1. Active Example Selection for In-Context LearningYiming Zhang, Shi Feng, Chenhao TanEMNLP 2022 · 84 citations
  2. On the Modeling Capabilities of Large Language Models for Sequential Decision MakingMartin Klissarov, R. Devon Hjelm, Alexander T. Toshev, Bogdan MazoureICLR 2025
  3. Uni[MASK]: Unified Inference in Sequential Decision ProblemsMicah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu et al.NeurIPS 2022 · 29 citations
  4. DLM: Unified Decision Language Models for Offline Multi-Agent Sequential Decision MakingZhuohui Zhang, Bin Cheng, Bin HeICML 2026
  5. Efficient Reinforcement Learning with Large Language Model PriorsXue Yan, Yan Song, Xidong Feng, Mengyue Yang et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines