FlashANNS: GPU-Driven Asynchronous I/O Pipelining for Eliminating Storage-Compute Bottlenecks in Billion-Scale Similarity Search

Approximate Nearest Neighbor Search (ANNS) enables efficient similarity retrieval in high-dimensional vector spaces, and becomes a fundamental component of upper-layer workloads ranging from recommendation systems to retrieval-augmented generation (RAG). Modern ANNS systems integrate SSDs to support terabytescale vector datasets, primarily employing cluster-indexing and graph-indexing. However, cluster-indexing ANNS systems suffer from suboptimal query throughput because of the coarse-grained vector indexing, while graph-indexing systems suffer from suboptimal performance due to two inherent limitations: 1) failing to overlap SSD accesses with distance computation processes and 2) poor I/O performance due to long tail latency. To address these challenges, we present FlashANNS, a GPU-accelerated out-of-core graph-based ANNS system through I/O-compute overlapping. Our core insight lies in the careful orchestration of I/O and computation through three key innovations: 1) Dependency-relaxed asynchronous pipeline with rigorous theoretical convergence guarantee: FlashANNS decouples I/O-computation dependencies to fully overlap between GPU distance calculations and SSD data transfers; 2) Query-grained concurrent SSD access: FlashANNS implement a lock-free I/O stack with query-grained concurrency control, to avoid I/O performance degradation due to long tail latency; and 3) Computation-I/O balanced graph degree selection, which ensures optimal balance between computational load and storage access latency across different hardware characteristics, different datasets, and different query requirements. We implement FlashANNS and compare it with three state-of-the-art out-of-core ANNS systems (SPANN, DiskANN, and FusionANNS). Experimental results demonstrate that at the same ≥95% recall@10 accuracy, our method achieves 2.7–5.9× higher query throughput compared to existing SOTA methods with a single SSD, and further achieves 3.9–12.2× query throughput improvement in the multi-SSD configurations.