Lune

ACL Findings2025

Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training

Yihang Yao, Zhepeng Cen, Miao Li, William Han, Yuyou Zhang, Emerson Liu, Zuxin Liu, Chuang Gan, Ding Zhao

Abstract

Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentation, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insights into improving LLM robustness through structured dataset curation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models Can Be Easily Distracted by Irrelevant Context cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning cites MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
  • MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
  • Let's Verify Step by Step cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites Let's Verify Step by Step
  • MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Behavior Injection: Preparing Language Models for Reinforcement Learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Behavior Injection: Preparing Language Models for Reinforcement LearningZhepeng Cen, Yihang Yao, William Han, Zuxin Liu et al.NeurIPS 2025 · 9 citations

Builds on24

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
  3. Large Language Models Can Be Easily Distracted by Irrelevant ContextFreda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales et al.ICML 2023 · 970 citations
  4. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsLonghui Yu, Weisen Jiang, Han Shi, Jincheng Yu et al.ICLR 2024 · 637 citations
  5. MAmmoTH: Building Math Generalist Models through Hybrid Instruction TuningXiang Yue, Xingwei Qu, Ge Zhang, Yao Fu et al.ICLR 2024 · 558 citations

Related papers

  1. Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric AugmentationQianxi He, Qianyu He, Jiaqing Liang, Weikang Zhou et al.EMNLP 2025
  2. MuggleMath: Assessing the Impact of Query and Response Augmentation on Math ReasoningChengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong et al.ACL 2024 · 4 citations
  3. Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical ReasoningQiming Bao, Alex Yuxuan Peng, Zhenyun Deng, Wanjun Zhong et al.ACL Findings 2024 · 6 citations
  4. How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled BenchmarkMinglai Yang, Ethan Huang, Liang Zhang, Mihai Surdeanu et al.EMNLP 2025 · 2 citations
  5. GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem SolversQintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong et al.ACL 2024 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines