Lune

EMNLP Findings2024

AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations

Zhicheng Yang, Yinya Huang, Jing Xiong, Liang Feng, Xiaodan Liang, Yiwei Wang, Jing Tang

Abstract

Large Language Models prompting, such as using in-context demonstrations, is a mainstream technique for invoking LLMs to perform highperformance and solid complex reasoning (e.g., mathematical reasoning, commonsense reasoning), and has the potential for further human-machine collaborative scientific findings. However, current LLMs are delicate and elusive in prompt words and styles. And there is an unseen gap between LLM understanding and human-written prompts. This paper introduces AlignedCoT, an LLM-acquainted prompting technique that includes proficient "native-speaking" in in-context learning for the LLMs. Specifically, it achieves consistent and correct step-wise prompts in zero-shot scenarios by progressively probing, refining, and formatting the LLM chain of thoughts so that free from handcrafted few-shot demonstrations while maintaining the prompt quality. We conduct experiments on mathematical reasoning and commonsense reasoning. We find that LLMs with AlignedCoT perform significantly superior to them with human-crafted demonstrations. We further apply AlignedCoT for rewriting the GSM8K training set, resulting in a GSM8K-Align dataset. We observe its benefits for retrieval augmented generation. The code and data can be found at https://github.com/yangzhch6/AlignedCoT.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Calibrate Before Use: Improving Few-shot Performance of Language Models
  • Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Least-to-Most Prompting Enables Complex Reasoning in Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Calibrate Before Use: Improving Few-shot Performance of Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on9

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  3. Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein et al.ICML 2021 · 1,843 citations
  4. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  5. Least-to-Most Prompting Enables Complex Reasoning in Large Language ModelsDenny Zhou, Nathanael Schärli, Le Hou, Jason Wei et al.ICLR 2023 · 318 citations

Related papers

  1. Large Language Models are Zero-Shot ReasonersTakeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo et al.NeurIPS 2022 · 8,168 citations
  2. Instance-adaptive Zero-shot Chain-of-Thought PromptingXiaosong Yuan, Chen Shen, Shaotian Yan, Xiaofeng Zhang et al.NeurIPS 2024 · 46 citations
  3. Large Language Models as Analogical ReasonersMichihiro Yasunaga, Xinyun Chen, Yujia Li, Panupong Pasupat et al.ICLR 2024 · 155 citations
  4. Task-Level Thinking Steps Help Large Language Models for Challenging Classification TaskChunhui Du, Jidong Tian, Haoran Liao, Jindou Chen et al.EMNLP 2023 · 2 citations
  5. Automatic Chain of Thought Prompting in Large Language ModelsZhuosheng Zhang, Aston Zhang, Mu Li, Alex SmolaICLR 2023 · 234 citations
Dusk over the sea between two cliffs drawn in fine vertical lines