Benchmarking and Learning Real-World Customer Service Dialogue

Existing benchmarks and training pipelines for industrial intelligent customer service (ICS) remain misaligned with real-world dialogue requirements, overemphasizing verifiable task success while under-measuring subjective service quality and realistic failure modes, leaving a gap between offline gains and deployable dialogue behavior. We close this gap with a benchmark-to-optimization loop: we first introduce OLABENCH, an ICS benchmark spanning retrieval-augmented generation, workflow-based systems, and agentic settings, which evaluates service capability, safety, and latency sensitivity; moreover, motivated by OLABENCH results showing state-of-the-art LLMs still fall short, we propose OLAMIND, which distills reusable reasoning patterns and service strategies from expert dialogues and applies staged exploration-exploitation reinforcement learning with instance-level rubric-aware guidance to improve model capability. OLA-MIND surpasses GPT-5.2 and Gemini 3 Pro on OLABENCH (83.64 vs. 70.58/70.84) and, in online A/B tests, delivers an average +23.67% issue resolution and -6.6% human transfer rate versus the baseline, bridging offline gains to deployment. Together, OLABENCH and OLA-MIND advance ICS systems toward more anthropomorphic, professional, and reliable deployment. The project page and evaluation are available at https://olamind-olabench. github.io .