ATHENA: Mathematical Reasoning with Thought Expansion

Solving math word problems depends on how to articulate the problems, the lens through which models view human linguistic expressions. Real-world settings count on such a method even more due to the diverse practices of the same mathematical operations. Earlier works constrain available thinking processes by limited prediction strategies without considering their significance in acquiring mathematical knowledge. We introduce Attention-based THought Expansion Network Architecture (ATHENA) to tackle the challenges of real-world practices by mimicking human thought expansion mechanisms in the form of neural network propagation. A thought expansion recurrently generates the candidates carrying the thoughts of possible math expressions driven from the previous step and yields reasonable thoughts by selecting the valid pathways to the goal. Our experiments show that ATHENA achieves a new state-of-the-art stage toward the ideal model that is compelling in variant questions even when the informativeness in training examples is restricted. 1 Context The school playground was originally [80] meters long and [40] meters wide. Later when the school is remodeled, the length is increased by [10] meters and the width is increased by [15] meters. Train on an example of a question-solution pair under the context above. Question How many square meters is the original playground area? Solution (80 × 40) Test on variant questions that share the context above. Q0 How many times the length of the original playground was the width? DeductReasoner ATHENA UnbiasedMWP (80 + 10) × (40 + 15) -(80 × 40) (X) UnbiasedMWP (80 + 10) × (40 -15) (X) UnbiasedMWP (1:N) 80 ÷ 40 (O) UnbiasedMWP (1:N) 80 ÷ 40 (O) Q1 How many square meters is the current playground area? DeductReasoner ATHENA UnbiasedMWP (80 + 10) × (40 + 15) -(80 × 40) (X) UnbiasedMWP (80 + 10) × (40 + 15) (O) UnbiasedMWP (1:N) 80 × 40 (X) UnbiasedMWP (1:N) (80 + 10) × (40 + 15) (O) Q2 How many square meters are increased by the current playground area compared to the original one? DeductReasoner ATHENA UnbiasedMWP (80 + 10) × (40 + 15) -(80 × 40) (O) UnbiasedMWP (80 + 10) × (40 + 15) -(80 × 40) (O) UnbiasedMWP (1:N) 80 × 40 (X) UnbiasedMWP (1:N) (80 + 10) × (40 + 15) -(80 × 40) (O) An example with a lexically similar context to that of above from the UnbiasedMWP Context The school basketball court was [20] meters long and [12] meters wide. After the renovation, the length is increased by [8] meters, and the width increases by [3] meters. Question How many square meters are increased? Solution (20 + 8) × (12 + 3) -(20 × 12