MVP-Tuning: Multi-View Knowledge Retrieval with Prompt Tuning for Commonsense Reasoning

Recent advances in pre-trained language models (PLMs) have facilitated the development of commonsense reasoning tasks. However, existing methods rely on multi-hop knowledge retrieval and thus suffer low accuracy due to embedded noise in the acquired knowledge. In addition, these methods often attain high computational costs and nontrivial knowledge loss because they encode the knowledge independently of the PLM, making it less relevant to the task and resulting in a poor local optimum. In this work, we propose Multi-View Knowledge Retrieval with Prompt Tuning (MVP-Tuning). Our MVP-Tuning leverages similar questionanswer pairs in training set to improve knowledge retrieval and employs a single prompttuned PLM to model knowledge and input text jointly. We conduct our experiments on five commonsense reasoning QA benchmarks to show that MVP-Tuning outperforms all other baselines in 4 out of 5 datasets with only as most 2% trainable parameters. The ensemble of our MVP-Tuning models even gets a new state-of-the-art performance on OpenBookQA and is ranked first place on the leaderboard 1 . Our code and data are available 2 . * Corresponding author. 1 The anonymous submission is in https:// leaderboard.allenai.org/open_book_qa/submission/ cdtvnvg4kc1nql1dnu3g 2 https://github.com/kochsnow/MVP-Tuning/ Input Question: <Q>: What are candles good for eliminating? A. shelf B. board C.church D.table E. dark Retrieved Question Answer Pairs: QA1: If I have a vintage, decorative light source in my possession, what is it likely to be? candle QA2: The power went out, so why did the family use a candle? emit light QA3: The person used a candle to navigate up the spiral staircase, where were they likely? light house