Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users

While most task-oriented dialogues assume conversations between the agent and one user at a time, dialogue systems are increasingly expected to communicate with multiple users simultaneously who make decisions collaboratively. To facilitate development of such systems, we release the Multi-User MultiWOZ dataset: task-oriented dialogues among two users and one agent. To collect this dataset, each user utterance from MultiWOZ 2.2 was replaced with a small chat between two users that is semantically and pragmatically consistent with the original user utterance, thus resulting in the same dialogue state and system response. These dialogues reflect interesting dynamics of collaborative decision-making in task-oriented scenarios, e.g., social chatter and deliberation. Supported by this data, we propose the novel task of multi-user contextual query rewriting: to rewrite a task-oriented chat between two users as a concise task-oriented query that retains only task-relevant information and that is directly consumable by the dialogue system. We demonstrate that in multi-user dialogues, using predicted rewrites substantially improves dialogue state tracking without modifying existing dialogue systems that are trained for single-user dialogues. Further, this method surpasses training a medium-sized model directly on multi-user dialogues and generalizes to unseen domains. 1 Q5-1. Does the summary contain additional information that is not present in the customer chat to the extent that the agent may perform an action or provide information that is not intended by the customers? (See the instructions regarding what is acceptable and what is not)

Yes: The summary contains additional information that is not present in the chat No: The summary contains no additional information

There are 5 colleges in the west. churchill college, clare college, clare hall, magdalene college, and queens' college.

Customer2 What is the entrance fee for Clare Hall? Customer1 There is no entrance fee.

It's open from 9 to 6 to the public. Do you need any other information?

Could you tell me the entrance fee for clare hall please?

Q1. Is the relationship between Customer1 and Customer2 really two customers making a decision together?

Yes: They are two customers making a decision together No: Their relationship is something else, e.g., one is a customer and the other is an agent Q2. Is the customer chat a realistic chat between humans?

Very much: Very realistic as a chat between humans Acceptable: Acceptable as a chat between humans Definitely not: Definitely not acceptable as a chat between humans Q3. Is the last utterance of the customer chat realistic as an utterance toward the agent?

Very much: Very realistic as an utterance toward the agent Acceptable: Acceptable as an utterance toward the agent Definitely not: Definitely not acceptable as an utterance toward the agent Q4-1. Is any important information in the customer chat missing in the summary to the extent that the agent may perform an action or provide information that is not intended by the customers? (See the instructions regarding what is acceptable and what is not) Yes: Some important information in the chat is missing in the summary No: No missing information Q4-2. If you chose "Yes", specify what information is missing. Q5-1. Does the summary contain additional information that is not present in the customer chat to the extent that the agent may perform an action or provide information that is not intended by the customers? (See the instructions regarding what is acceptable and what is not)