Principled Zero-shot Ranking Agents with Tournament Graphs

Selecting the top m from n items via expensive k-wise comparisons is central to settings ranging from LLM-based document reranking to crowdsourced evaluation and tournament design. Existing methods either rely on heuristics that discard comparison information, or exploit it at prohibitive cost. We introduce a tournament graph framework that provides a principled foundation for k-wise ranking. Our key observation is that each k-item comparison reveals an induced tournament of k 2 pairwise preferences; aggregating these into a global preference graph and computing its transitive closure yields many additional orderings without further oracle calls. We formalize when the current top-m output is certifiably determined and design a greedy query schedule that maximizes information gain towards identifying the top-m items. The framework also gracefully handles non-transitive preferences -cycles induced by real-world oracles -by collapsing them into equivalence classes that yield principled tiered rankings. Applied to LLM reranking across 14 benchmarks and 5 models, BLITZRANK achieves Pareto dominance over existing approaches: matching or exceeding accuracy while requiring 25-40% fewer tokens than comparable methods; against pairwise reranking, it achieves near-identical quality with 7× fewer tokens. Code available at https://github. com/ContextualAI/BlitzRank .