Generalizing Reasoning Problems to Longer Lengths

❑ when trained on reasoning problems of smaller lengths/sizes, e.g., 345 + 67, ❑ the model struggles with problems of longer lengths, e.g., 1234 + 56789. ◼ A popular solution to improve reasoning is to use Chain of Thought (CoT) (Wei et al., 2022), ❑ CoT: providing intermediate reasoning steps ❑ However, Dziri et al. (2023) and others have shown that even with detailed CoT steps, the learned models still fail to generalize.