EMNLP Findings2022
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training
Qi Shi, Qian Liu, Bei Chen, Yu Zhang, Ting Liu, Jian-Guang Lou
Abstract
Language-based environment manipulation requires agents to manipulate the environment following natural language instructions, which is challenging due to the huge space of the environments. To address this challenge, various approaches have been proposed in recent work. Although these approaches work well for their intended environments, they are difficult to generalize across environments. In this work, we propose LEMON, a general framework for language-based environment manipulation tasks. Specifically, we first specify a task-agnostic approach for language-based environment manipulation tasks, which can deal with various environments using the same generative language model. Then we propose an execution-guided pre-training strategy to inject prior knowledge of environments to the language model with a pure synthetic pre-training corpus. Experimental results on tasks including ALCHEMY, SCENE, TANGRAMS, PROPARA and RECIPES demonstrate the effectiveness of LEMON: it achieves new state-of-the-art results on four of the tasks, and the executionguided pre-training strategy brings remarkable improvements on all experimental tasks 1 .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on8
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- TAPEX: Table Pre-training via Learning a Neural SQL ExecutorQian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi et al.ICLR 2022 · 347 citations
- A Dataset for Tracking Entities in Open Domain Procedural TextNiket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal et al.EMNLP 2020 · 38 citations
- ProTo: Program-Guided Transformer for Program-Guided TasksZelin Zhao, Karan Samel, Binghong Chen, Le SongNeurIPS 2021 · 37 citations
- SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual ReasoningTsu-Jui Fu, Xin Wang, Scott T. Grafton, Miguel P. Eckstein et al.EMNLP 2020 · 26 citations
Related papers
- Generalizable Coarse-to-Fine Robot Manipulation via Language-Aligned 3D KeypointsJianshu Hu, Lidi Wang, Shujia Li, Yunpeng Jiang et al.ICLR 2026 · 6 citations
- Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World EnvironmentsYu Gu, Xiang Deng, Yu SuACL 2023 · 36 citations
- Language-based General Action Template for Reinforcement Learning AgentsRyosuke Kohita, Akifumi Wachi, Daiki Kimura, Subhajit Chaudhury et al.ACL Findings 2021 · 4 citations
- villa-X: Enhancing Latent Action Modeling in Vision-Language-Action ModelsXiaoyu Chen, Hangxing Wei, Pushi Zhang, Chuheng Zhang et al.ICLR 2026 · 59 citations
- Ask-before-Plan: Proactive Language Agents for Real-World PlanningXuan Zhang, Yang Deng, Zifeng Ren, See-Kiong Ng et al.EMNLP Findings 2024 · 14 citations
