Lune

ACL Findings2022

Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems

Zhongli Li, Wenxuan Zhang, Chao Yan, Qingyu Zhou, Chao Li, Hongzhi Liu, Yunbo Cao

Abstract

Math Word Problem (MWP) solving needs to discover the quantitative relationships over natural language narratives. Recent work shows that existing models memorize procedures from context and rely on shallow heuristics to solve MWPs. In this paper, we look at this issue and argue that the cause is a lack of overall understanding of MWP patterns. We first investigate how a neural network understands patterns only from semantics, and observe that, if the prototype equations like n 1 + n 2 are the same, most problems get closer representations and those representations apart from them or close to other prototypes tend to produce wrong solutions. Inspired by it, we propose a contrastive learning approach, where the neural network perceives the divergence of patterns. We collect contrastive examples by converting the prototype equation into a tree and seeking similar tree structures. The solving model is trained with an auxiliary objective on the collected examples, resulting in the representations of problems with similar prototypes being pulled closer. We conduct experiments 1 on the Chinese dataset Math23k and the English dataset MathQA. Our method greatly improves the performance in monolingual and multilingual settings.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

13 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Guiding Mathematical Reasoning via Mastering Commonsense Formula Knowledge cites Graph-to-Tree Learning for Solving Math Word Problems
  • Generalizing Math Word Problem Solvers via Solution Diversification cites Graph-to-Tree Learning for Solving Math Word Problems
  • Generalizing Math Word Problem Solvers via Solution Diversification cites Analogical Math Word Problems Solving with Enhanced Problem-Solution Association
  • Non-Autoregressive Math Word Problem Solver with Unified Tree Structure cites Graph-to-Tree Learning for Solving Math Word Problems
  • Analogical Math Word Problems Solving with Enhanced Problem-Solution Association cites Graph-to-Tree Learning for Solving Math Word Problems
  • A Symbolic Characters Aware Model for Solving Geometry Problems cites Graph-to-Tree Learning for Solving Math Word Problems
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers13

  1. Analogical Math Word Problems Solving with Enhanced Problem-Solution AssociationZhenwen Liang, Jipeng Zhang, Xiangliang ZhangEMNLP 2022 · 18 citations
  2. A Symbolic Characters Aware Model for Solving Geometry ProblemsMaizhen Ning, Qiu-Feng Wang, Kaizhu Huang, Xiaowei HuangACM MM 2023 · 18 citations
  3. Guiding Mathematical Reasoning via Mastering Commonsense Formula KnowledgeJiayu Liu, Zhenya Huang, Zhiyuan Ma, Qi Liu et al.KDD 2023 · 11 citations
  4. Generalizing Math Word Problem Solvers via Solution DiversificationZhenwen Liang, Jipeng Zhang, Lei Wang, Yan Wang et al.AAAI 2023 · 10 citations
  5. Non-Autoregressive Math Word Problem Solver with Unified Tree StructureYi Bin, Mengqun Han, Wenhao Shi, Lei Wang et al.EMNLP 2023 · 7 citations

Builds on6

  1. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  2. Hard Negative Mixing for Contrastive LearningYannis Kalantidis, Mert Bülent Sariyildiz, Noé Pion, Philippe Weinzaepfel et al.NeurIPS 2020 · 805 citations
  3. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
  4. Graph-to-Tree Learning for Solving Math Word ProblemsJipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin et al.ACL 2020 · 129 citations
  5. Improving BERT with Syntax-aware Local AttentionZhongli Li, Qingyu Zhou, Chao Li, Ke Xu et al.ACL Findings 2021 · 33 citations

Related papers

  1. Learning by Analogy: Diverse Questions Generation in Math Word ProblemZihao Zhou, Maizhen Ning, Qiufeng Wang, Jie Yao et al.ACL Findings 2023 · 25 citations
  2. Textual Enhanced Contrastive Learning for Solving Math Word ProblemsYibin Shen, Qianying Liu, Zhuoyuan Mao, Fei Cheng et al.EMNLP Findings 2022 · 8 citations
  3. Multi-View Reasoning: Consistent Contrastive Learning for Math Word ProblemWenqi Zhang, Yongliang Shen, Yanna Ma, Xiaoxia Cheng et al.EMNLP Findings 2022 · 17 citations
  4. Structure-Unified M-Tree Coding Solver for Math Word ProblemBin Wang, Jiangzhou Ju, Yang Fan, Xinyu Dai et al.EMNLP 2022 · 13 citations
  5. It Ain't Over: A Multi-aspect Diverse Math Word Problem DatasetJiwoo Kim, Youngbin Kim, Ilwoong Baek, JinYeong Bak et al.EMNLP 2023 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines