Crossing Variational Autoencoders for Answer Retrieval

Answer retrieval is to find the most aligned answer from a large set of candidates given a question. Learning vector representations of questions/answers is the key factor. Questionanswer alignment and question/answer semantics are two important signals for learning the representations. Existing methods learned semantic representations with dual encoders or dual variational auto-encoders. The semantic information was learned from language models or question-to-question (answer-to-answer) generative processes. However, the alignment and semantics were too separate to capture the aligned semantics between question and answer. In this work, we propose to cross variational auto-encoders by generating questions with aligned answers and generating answers with aligned questions. Experiments show that our method outperforms the state-of-theart answer retrieval method on SQuAD. Question Answer 𝑝(𝑦|𝑧 ! , 𝑧 " ) 𝑝(𝑞|𝑧 ! ) 𝑝(𝑎|𝑧 " ) 𝑧 ! 𝑝(𝑧 ! ) Encoder 𝑝(𝑧 ! |𝑞) 𝑧 " 𝑝(𝑧 " ) 𝑝(𝑧 " |𝑎) Question Answer Decoder 𝑝(𝑞|𝒛 𝒂 ) 𝑝(𝑎|𝒛 𝒒 ) 𝑝(𝑦|𝑧 ! , 𝑧 " ) 𝑝(𝑦|𝑧 ! , 𝑧 " ) Question Answer Question Answer Decoder Encoder Encoder Decoder Decoder Encoder 𝑝(𝑧 ! |𝑞) 𝑝(𝑧 " |𝑎)