Structure-Aware Quantized Retrieval for Long-Document Question Answering

Long-document question answering is challenging because relevant evidence is often scattered across distant sections. Traditional longdocument QA/RAG pipelines often suffer from context fragmentation, retrieving locally plausible but structurally misaligned passages. We present the Hierarchical Quantized Document Retriever (HQDR), a framework that aligns hierarchical graph representations with a universal token vocabulary and integrates explicit structure into retrieval. By grounding continuous structural features in a fixed, discrete semantic space, HQDR captures universal hierarchical patterns rather than overfitting to specific layouts. We further propose a hybrid scoring mechanism that decouples semantic matching from structural alignment. Extensive experiments on QASPER and Natural Questions demonstrate that HQDR consistently outperforms strong baselines and exhibits superior robustness when transferring across datasets with distinct structural characteristics.