Lune

EMNLP Findings2024

Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft

Kranti Chalamalasetti, Sherzod Hakimov, David Schlangen

Abstract

In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Builder. Leveraging LLMs' in-context learning abilities, we use few-shot prompting techniques, that significantly improve performance over baseline methods. Additionally, we present a detailed analysis of the gaps in performance for future work.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • QLoRA: Efficient Finetuning of Quantized LLMs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • PaLM-E: An Embodied Multimodal Language Model cites Language Models are Few-Shot Learners
  • PaLM-E: An Embodied Multimodal Language Model cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Language Models are Few-Shot Learners cites Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  4. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  5. PaLM-E: An Embodied Multimodal Language ModelDanny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch et al.ICML 2023 · 2,601 citations

Related papers

  1. Learning to execute instructions in a Minecraft dialoguePrashant Jayannavar, Anjali Narayan-Chen, Julia HockenmaierACL 2020 · 22 citations
  2. Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks WorldGuande Wu, Chen Zhao, Cláudio T. Silva, He HeACL Findings 2024 · 3 citations
  3. CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language ProgrammingLi Feng, Ryan Yen, Yuzhe You, Mingming Fan et al.CHI 2024 · 28 citations
  4. Aligning Predictive Uncertainty with Clarification Questions in Grounded DialogKata Naszádi, Putra Manggala, Christof MonzEMNLP Findings 2023 · 2 citations
  5. Prompting Language Models for Linguistic StructureTerra Blevins, Hila Gonen, Luke ZettlemoyerACL 2023 · 15 citations
Dusk over the sea between two cliffs drawn in fine vertical lines