ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation

While LLMs increasingly assist individual users, there is a critical need for agents that can proactively manage complex, multi-party collaboration. However, the scarcity of systematic evaluation methods for these group dynamics limits the development of AI capable of effectively supporting teams. Here, we present PROMEDIATE 1 , the first testbed for evaluating proactive AI mediator agents in complex, multi-topic, multi-party negotiations. PROME-DIATE consists of two core components: (i) a simulation environment based on realistic negotiation cases with a plug-and-play proactive AI mediator, capable of flexibly deciding when and how to intervene; and (ii) a socio-cognitive evaluation framework with a new suite of metrics to measure consensus changes, intervention latency, mediator effectiveness, and intelligence. These components establish a systematic framework for assessing the capability of proactive AI agents in multi-party settings. Our results show that a socially intelligent mediator agent outperforms a generic baseline, via faster, better-targeted interventions. In the PROMEDI-ATE-Hard setting, our social mediator increases consensus change by 3.6 percentage points compared to the generic baseline (10.65% vs. 7.01%) while being 77% faster in response (15.98s vs. 3.71s). In conclusion, PROME-DIATE provides a rigorous, theory-grounded testbed to advance the development of proactive, socially intelligent agents. * Work done during Ziyi's internship at Microsoft Corp. 1 The code is available at https://github.com/ 👨⚖️ Mediator Hopkins HMO and PharmaCare Inc. (PCI) are negotiating a contract for Profelice, a new antidepressant. The negotiation involves multiple issues: pricing discounts, market share tiers, marketing support, formulary status, and contract length-with several options under each. The parties must balance cost control, market protection and patient care flexibility. Profelice is a breakthrough. Hopkins should lead adoption with premium discounts and exclusive formulary. Exclusive status is off the table. We need dual formulary status and aggressive discounts to protect patient cost. Dual status reduces our market protection. We need either exclusivity or a $20M volume commitment. Could we explore a phased approach-exclusive status for the first year with premium discounts, then transition to dual formulary? That works. Exclusive for 12 months with full discounts, then dual status as competitors enter. Consensus tracking Lee (HMO) Alex (PCI) Alex (PCI) Jamie (PCI)