Reading Between the Lines: The One-Sided Conversation Problem

Conversational AI is constrained in many realworld settings where only one side of a dialogue can be recorded. We formalize the onesided conversation problem (1SC): inferring and learning from only one side of a conversation. We study two tasks: (1) reconstructing the missing speaker's turns and ( 2 ) generating summaries from one-sided transcripts. Evaluating models on MultiWOZ, DailyDialog, Spo-kenWOZ and Candor with both human A/B testing and LLM-as-a-judge metrics, we find that additional context improves reconstruction, and while large models generate promising reconstructions with prompting, smaller models require finetuning. Further, high-quality summaries can be generated without reconstructing missing turns. We present 1SC as a novel challenge and report promising results that mark a step toward privacy-aware conversational AI.