BOLT: Benchmarking Open-World Learning for Text Classification

Text classification has long been a cornerstone of NLP, yet most prior work and benchmarks have been limited to closed-world settings, where all classes are assumed to be known in advance. In contrast, open-world learning has recently emerged as a critical paradigm for building more robust and realistic systems. However, existing benchmarks largely focus on out-of-distribution (OOD) detection, while overlooking broader challenges such as the discovery of novel categories. To address this gap, we introduce BOLT, a unified Benchmark and evaluation toolkit supporting Open-world Learning for Text classification. BOLTencompasses two representative tasks: Open-set Text Classification (OSTC), which requires models to classify in-distribution (ID) samples while rejecting OOD inputs, and Generalized Category Discovery (GCD), which aims to identify both known and novel categories from partially labeled corpora. We carefully curate 12 publicly available datasets spanning diverse domains and benchmark 22 methods, including 15 for OSTC and 7 for GCD, under a standardized protocol that explicitly accounts for varying labeled ratios and known class ratios. Our results reveal key challenges: most current methods tend to overfit training distributions and struggle to generalize to unseen classes. Moreover, by comparing our lightweight LLMbased variants with prior open-set baselines, we demonstrate the promise of leveraging LLMs for open-world text classification. BOLT provides standardized evaluation protocols that enable fair comparison and support future research in this emerging area. All datasets, baselines, and tools are available at https: //github.com/CNIC-DSL/BOLT . Task KCR LAR Evaluation Metrics Paper List OSTC 0.250, 0.500, 0.750 1.000 F1 DOC [(Shu et al., 2017)], DeepUnk [(Lin and Xu, 2019)] 0.250, 0.500, 0.750 1.000 ACC, F1, K-F1, N-F1 ADB [(Zhang et al., 2021a)], AB [(Lorenc et al., 2022)],KNNCon [(Zhou et al., 2022)] 0.667 1.000 ACC, F1, Recall SCL [(Zeng et al., 2021a)] 0.250, 0.750 1.000 ACC, F1, K-F1, N-F1 DyEn [(Zhou et al., 2023)] 0.250, 0.500, 0.750 0.100, 0.500, 1.000 ACC, F1, K-F1, N-F1 BOLT GCD 0.750 0.100 NMI, ARI, ACC DAL [(Zhang et al., 2021b)] 0.250 0.100 NMI, ARI, ACC GeoID [(Tang et al., 2024)] 0.750 0.075 ACC DPN [(An et al., 2023)] 0.750 0.100 H-score, K-ACC, N-ACC SDC [(An et al., 2025)], TAN [(An et al., 2024b)], LOOP [(An et al., 2024a)] 0.250, 0.500 0.100