Decoding Intent from Fragments: Structure-Aware Late Interaction for Concept-Level EEG-to-Text Retrieval
Rui Lin, Bo Xu, Quanhao Zhu, Boling Zhu, Chenyuan Wang, Liang Zhao
Abstract
While retrieval-based paradigms have emerged as a more viable alternative to generative approaches for EEG-to-Text decoding, existing methods typically compress continuous EEG signals into holistic vectors. However, real-world brain-computer interaction is fundamentally driven by discrete conceptual units, and current methods that compress continuous EEG signals into holistic vectors struggle to capture the granularity of these sparse, fragmented intents. To address this limitation, we introduce a novel task: Brain Concept-to-Sentence Retrieval (BCSR), designed to decode intent from fragmented EEG signals. We propose SparkRetriever, a structure-aware retrieval model tailored for aligning discrete EEG concepts with natural language sentences. Departing from traditional dual-tower architectures, SparkRetriever incorporates a fine-grained late interaction mechanism. By preserving EEG tokens as independent semantic anchors, it enables precise multi-to-multi matching with text tokens. Furthermore, to tackle signal discreteness and cross-subject variability, we construct a robust semantic manifold leveraging Multi-Positive Contrastive Learning and Subject Embeddings. Extensive experiments on the ZuCo benchmark demonstrate that SparkRetriever significantly outperforms state-of-the-art baselines, achieving relative improvements ranging from 23.4% to 33.7% across multiple metrics. Our analysis reveals that multi-positive supervision acts as the primary driver of performance, effectively mitigating the bias inherent in single-sentence alignment.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Related papers
- Neuro-SPO: Physically-Grounded and Affect-Aligned EEG-to-Keyword DecodingZihua Xu, C. L. Philip Chen, Tong ZhangKDD 2026
- Towards Brain Passage Retrieval: An Investigation of EEG Query RepresentationsNiall McGuire, Yashar MoshfeghiSIGIR 2025 · 5 citations
- Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked AutoencoderJiaqi Wang, Zhenxi Song, Zhengyu Ma, Xipeng Qiu et al.ACL 2024
- Open Vocabulary Electroencephalography-to-Text Decoding and Zero-Shot Sentiment ClassificationZhenhailong Wang, Heng JiAAAI 2022 · 122 citations
- Visual Decoding and Reconstruction via EEG Embeddings with Guided DiffusionDongyang Li, Chen Wei, Shiying Li, Jiachen Zou et al.NeurIPS 2024 · 164 citations
