Hide and Seek with LLMs: An Adversarial Game for Sneaky Error Generation and Self-Improving Diagnosis
Rui Zou, Mengqi Wei, Yutao Zhu, Jirong Wen, Xin Zhao, Jing Chen
Abstract
Large Language Models (LLMs) excel in reasoning and generation across domains, but still struggle with identifying and diagnosing complex errors. This stems mainly from training objectives that prioritize correct answers, limiting exposure to and learning from errors. While recent studies have begun to address this by introducing error signals, most rely on shallow, static errors, restricting improvement in deep diagnostic ability. To overcome this, we propose Hide and Seek Game (HSG), a dynamic adversarial framework for error generation and diagnosis, and evaluate it on mathematical problem-solving. HSG involves two adversarial roles: Sneaky, which hides by generating subtle, deceptive reasoning errors, and Diagnosis, which seeks to accurately detect them. Through adversarial co-evolution, both error stealth and diagnostic precision are enhanced. Experiments on three mathematical reasoning datasets demonstrate that HSG significantly boosts error diagnosis, achieving 16.8%-31.4% higher accuracy than baselines like GPT-4o. We also release a challenging dataset of deceptive errors and diagnostic annotations as a benchmark for future research.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Large Language Models Cannot Self-Correct Reasoning Yet cites CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing cites Large Language Models Cannot Self-Correct Reasoning Yet
- Large Language Models Cannot Self-Correct Reasoning Yet cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Training language models to follow instructions with human feedback
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing cites Training language models to follow instructions with human feedback
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Training language models to follow instructions with human feedback
Builds on10
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Large Language Models Cannot Self-Correct Reasoning YetJie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng et al.ICLR 2024 · 858 citations
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive CritiquingZhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen et al.ICLR 2024 · 699 citations
Related papers
- SPC: Evolving Self-Play Critic via Adversarial Games for LLM ReasoningJiaqi Chen, Bang Zhang, Ruotian Ma, Peisong Wang et al.NeurIPS 2025 · 43 citations
- Improving Rationality in the Reasoning Process of Language Models through Self-playing GamePinzheng Wang, Juntao Li, Zecheng Tang, Haijia Gui et al.ICML 2025
- Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement LearningQihao Liu, Luoxin Ye, Wufei Ma, Yu-Cheng Chou et al.ICLR 2026 · 5 citations
- Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical ReasoningErxin Yu, Jing Li, Ming Liao, Qi Zhu et al.ACL 2025 · 4 citations
- Adversarial Training for Process Reward ModelsGurusha Juneja, Deepak Nathani, William WangICML 2026 · 2 citations
