EMNLP Findings2024
Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft
Abstract
In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Builder. Leveraging LLMs' in-context learning abilities, we use few-shot prompting techniques, that significantly improve performance over baseline methods. Additionally, we present a detailed analysis of the gaps in performance for future work.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- QLoRA: Efficient Finetuning of Quantized LLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- PaLM-E: An Embodied Multimodal Language Model cites Language Models are Few-Shot Learners
- PaLM-E: An Embodied Multimodal Language Model cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds on12
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
- PaLM-E: An Embodied Multimodal Language ModelDanny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch et al.ICML 2023 · 2,601 citations
Related papers
- Learning to execute instructions in a Minecraft dialoguePrashant Jayannavar, Anjali Narayan-Chen, Julia HockenmaierACL 2020 · 22 citations
- Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks WorldGuande Wu, Chen Zhao, Cláudio T. Silva, He HeACL Findings 2024 · 3 citations
- CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language ProgrammingLi Feng, Ryan Yen, Yuzhe You, Mingming Fan et al.CHI 2024 · 28 citations
- Aligning Predictive Uncertainty with Clarification Questions in Grounded DialogKata Naszádi, Putra Manggala, Christof MonzEMNLP Findings 2023 · 2 citations
- Prompting Language Models for Linguistic StructureTerra Blevins, Hila Gonen, Luke ZettlemoyerACL 2023 · 15 citations
