NumColBERT: Non-Intrusive Numeracy Injection for Late-Interaction Retrieval Models

This study addresses the challenge of improving dense retrieval performance for queries containing numerical conditions, such as ''companies with more than one billion dollars in R&D expenditure.'' Although recent research has underscored the limitations of standard models in handling numeric information across domains such as finance, e-commerce, and medicine, existing solutions typically decompose queries into textual and numerical components and score them separately using dedicated methods. These approaches intrude upon late-interaction retrieval models such as ColBERT and incur considerable challenges in deployment, latency, and maintainability. To overcome these limitations, we propose NumColBERT, an inference-time non-intrusive method that enhances numerically conditioned retrieval while preserving the original late-interaction mechanism and providing unified scoring across textual and numerical content. Because NumColBERT retains the standard ColBERT indexing and MaxSim scoring pipeline, existing optimizations and ecosystem components developed for ColBERT can be directly reused, facilitating practical deployment. NumColBERT introduces a Numerical Gating Mechanism and a Numerical Contrastive Learning objective to enable numerical conditions to contribute more effectively to retrieval within the standard ColBERT scoring mechanism. The gating mechanism dynamically amplifies the influence of tokens carrying critical numerical constraints while suppressing context-neutral mentions such as model numbers or dates. The contrastive objective explicitly shapes the embedding space to reflect numerical magnitudes and conditions, enabling numerical values to be distinguished within the shared representation space. Experimental results show that NumColBERT substantially outperforms standard fine-tuning baselines and achieves accuracy that matches or exceeds that of prior approaches that rely on separate textual and numerical scoring. These findings demonstrate the feasibility of numerically conditioned retrieval with a non-intrusive inference pipeline and present a maintainable solution for real-world deployment.