An Annotated Corpus of Reference Resolution for Interpreting Common Grounding

Common grounding is the process of creating, repairing and updating mutual understandings, which is a fundamental aspect of natural language conversation. However, interpreting the process of common grounding is a challenging task, especially under continuous and partially-observable context where complex ambiguity, uncertainty, partial understandings and misunderstandings are introduced. Interpretation becomes even more challenging when we deal with dialogue systems which still have limited capability of natural language understanding and generation. To address this problem, we consider reference resolution as the central subtask of common grounding and propose a new resource to study its intermediate process. Based on a simple and general annotation schema, we collected a total of 40,172 referring expressions in 5,191 dialogues curated from an existing corpus, along with multiple judgements of referent interpretations. We show that our annotation is highly reliable, captures the complexity of common grounding through a natural degree of reasonable disagreements, and allows for more detailed and quantitative analyses of common grounding strategies. Finally, we demonstrate the advantages of our annotation for interpreting, analyzing and improving common grounding in baseline dialogue systems. Introduction Common grounding is the process of creating, repairing and updating mutual understandings, which is a critical aspect of sophisticated human communication (Clark 1996) as well as a longstanding goal in dialogue modeling (Traum 1994) . Recently, there have been several new proposals of dialogue tasks which require advanced skills of common grounding under continuous and partially-observable context (Udagawa and Aizawa 2019; Haber et al. 2019) . Their main contributions include proposal of clear evaluation metrics based on task success rate, collection of large-scale datasets (thousands of dialogues) and introduction of complex ambiguity, uncertainty, partial understandings and misunderstandings which are minimally observed under traditional settings based on either categorical or fully-observable context. However, interpretation of the process of common grounding remains largely an open problem. Although a formal theory such as Poesio and Rieser (2010) can account