Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering

We propose Chain-of-Questions, a framework that trains a model to robustly answer multistep questions by generating and answering sub-questions. We obtain supervision for subquestions from human-annotated question decomposition meaning representation (QDMR), but QDMR does not include annotated answers to sub-questions. To overcome this technical challenge, we treat sub-answers as latent variables and infer them with a novel dynamic mixture of Hard-EM and MAPO. Chain-of-Questions is effective and robust, greatly outperforming strong neuro-symbolic methods by 9.0 F1 on a DROP contrast set and GPT-3.5 by 24.3 F1 on a HOTPOTQA adversarial set. Question Context Ground-truth QDMR Generated QDMR & Answers How many years after Pegu fell did the king die? (DROP) After the fall of Pegu in December 1599 ... but the king died during the campaign on 3 March 1606.