Agentic Oversight via Dialectic Reasoning

Debate has emerged as a promising oversight mechanism for Large Language Models (LLMs) amid rising systemic complexity, particularly where models outperform human evaluators. Yet, Debate provides little verifiable evidence for its final judgments, and its scalability across domains remains largely unexplored. To make oversight grounded and scale as capabilities extend, we introduce an Agentic Oversight framework. Specifically, by using Dialectic Argumentation as a reasoning function, we extend this paradigm to multilingual and multimodal spaces. We employ a weak-to-strong oversight approach based on two expert models that evaluate and defend contesting answers, while a third blind judge determines the winner using Dialectic Argumentation. Experts argue only for belief-consistent answers, founding the Debate on disagreements. We experimented with six tasks on our framework in both multilingual and multimodal scenarios, experimenting with different models. After indepth analysis, we show that dialectic argumentation consistently outperforms single-expert baselines. Moreover, we demonstrate that dialectic judgements from a weaker model deliver argument-mediated supervision that, via finetuning, instils unsupervised reasoning signals in expert models. Q: What is dish called? Options: a. Paris. b. Choux pastry c. Pastafrolla d. Ravioli e. Sfogliatella Question Answers & Debate b. Choux pastry e. Sfogliatella Round 1 #Premises P1: Each piece has a distinct fan shell shape with many ne, curved ridges from laminated layers. P2: The crust looks hard-baked and aky, strata are visibly separated. P3: Filling is visible at the open side; icing sugar is over the ridged shell. P4: Some pieces show dark amarena in custard typical of the Sant'Agata variant. #Argument The shell-shaped, ridged, laminated exterior is characteristic of Sfogliatella. The custard-and-cherry nish ts the Neapolitan Sfogliatella Santarosa style. #Premises P1: The items are individual pastries dusted with icing sugar. P2: Each pastry is lled with a pale custard-like cream and dotted with dark cherries. P3: The exterior looks airy and layered, suggesting a light dough. #Argument These appear to be choux-based pastries because they are small, cream-lled, and nished with icing sugar features common to éclairs and cream pus. The structure seems light and layered, which aligns with a steam-leavened dough. The cherries are consistent with patisserie nishes on choux items. Round 2 #Agreements. Accepts icing sugar and custard/cherries as non-diagnostic by themselves. #Disagreements & Challenges C1 Surface morphology shows parallel lamination forming a fan; these are not random cracks the lines are evenly spaced and curve from a narrow tip to a broad mouth. C2 The edge shows compressed, crisp layers, smooth crust of choux. C3 Choux pastries are typically split éclairs or proteroles;[.....] with visible stratied pastry. #Counter-observations O1: The hand-fan geometry is canonical for sfogliatella. O2: The crust appears crackly, with separable sheets a sign of laminated butter dough. O3: The presence of amarena cherries in custard is consistent with Sant'Agata, a recognised variant. #Agreements. Both accept there is custard, cherries, icing sugar. #Disagreements & Challenges C1 (P1,P2): The 'layers' could simply be cracks from baking a light dough. C2 (P4): Cherries and custard are generic patisserie elements. #Counter-observations O1: The pastries seem light and hollow near the lling, akin to choux cavities. O2: The overall size and individual portioning resemble cream pus.