Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation

Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. These tasks are formulated as a binary classification of responses given in a dialogue context, and models generally learn to make predictions based on context-response content similarity. However, over-reliance on content similarity makes the models less sensitive to the presence of inconsistencies, incorrect time expressions and other factors important for response appropriateness and coherence. We propose approaches for automatically creating adversarial negative training data to help ranking and evaluation models learn features beyond content similarity. We propose mask-and-fill and keyword-guided approaches that generate negative examples for training more robust dialogue systems. These generated adversarial responses have high content similarity with the contexts but are either incoherent, inappropriate or not fluent. Our approaches are fully data-driven and can be easily incorporated in existing models and datasets. Experiments on classification, ranking and evaluation tasks across multiple datasets demonstrate that our approaches outperform strong baselines in providing informative negative examples for training dialogue systems. 1 Error category Description Sample responses C-ent Incorrect entities or actors (R,G)

Incorrect subject or object of verbs or presence of one or more incorrect entities or coreference.

Context: I am so happy that you are doing okay.

Response: My friend is always happy.

Use of incorrect time expressions or tense of verbs. Context: What are you going to do on Monday? Response: Yesterday, I celebrated my daughter's wedding anniversary. C-cont Contradictory or extraneous details (R,G) Presence of details which make the response inconsistent within itself or contradict the context Context: A: I don't know why I bothered to come here. B: Did you enjoy your stay? Response: I enjoyed the concert a lot. C-speaker Incorrect speaker turn (R) The response is relevant to the conversation but from the wrong speaker. Context: What starting salary would you expect here? Response: If you work overtime, I will pay you extra salary. C-follow Does not directly address the context (R,G) The response does not follow immediately from the context. Context: What would you like for main course sir? Response: I know very well how to make noodles, and I taught one of my friends. C-strat Incorrect strategies (R,G) Use of incorrect dialogue act, emotion, persona or style Context: I can't find the paper clips. Response: Ok, great work. C-lang Poor language (G) Presence of poor grammar, incorrect sentence structures or repetitions Context: Do you have mixed drinks available here? Response: Yes. This order is divided by 16 divided for main main ones of order.