LOVBench: Ontology Ranking Benchmark
Niklas Kolbe, Pierre-Yves Vandenbussche, Sylvain Kubler, Yves Le Traon
Abstract
Ontology search and ranking are key building blocks to establish and reuse shared conceptualizations of domain knowledge on the Web. However, the effectiveness of proposed ontology ranking models is difficult to compare since these are often evaluated on diverse datasets that are limited by their static nature and scale. In this paper, we first introduce the LOVBench dataset as a benchmark for ontology term ranking. With inferred relevance judgments for more than 7000 queries, LOVBench is large enough to perform a comparison study using learning to rank (LTR) with complex ontology ranking models. Instead of relying on relevance judgments from a few experts, we consider implicit feedback from many actual users collected from the Linked Open Vocabularies (LOV) platform. Our approach further enables continuous updates of the benchmark, capturing the evolution of ontologies’ relevance in an ever-changing data community. Second, we compare the performance of several feature configurations from the literature using LOVBench in LTR settings and discuss the results in the context of the observed real-world user behavior. Our experimental results show that feature configurations which are (i) well-suited to the user behavior, (ii) cover all features types, and (iii) consider decomposition of features can significantly improve the ranking performance.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Related papers
- Can Clicks Be Both Labels and Features?: Unbiased Behavior Feature Collection and Uncertainty-aware Learning to RankTao Yang, Chen Luo, Hanqing Lu, Parth Gupta et al.SIGIR 2022 · 23 citations
- How do Online Learning to Rank Methods Adapt to Changes of Intent?Shengyao Zhuang, Guido ZucconSIGIR 2021 · 6 citations
- LT2R: Learning to Online Learning to Rank for Web SearchXiaokai Chu, Changying Hao, Shuaiqiang Wang, Dawei Yin et al.ICDE 2024 · 1 citation
- Learning to Rank with Multi-Criteria LLM-Judge AnnotationsNaghmeh Farzi, Laura DietzSIGIR 2026
- LRBench and Judge-R1: Principled Evaluation and Training of LLM-Based Judges for Long-Context ReasoningXinyi Zhao, Haoqi Hu, Ziyu Wang, Jinfeng XiaoACL Findings 2026
