CerebraGloss: Instruction-Tuning a Large Vision-Language Model for Fine-Grained Clinical EEG Interpretation
Wei Gu, Tianming Luo, Qiran Zhang, Mohan Ye, Xiao Shen, Wenxin Chen, Yunhuan Li, Yichen Zhang, Jing Hong, Bao-Liang Lu, Wei-Long Zheng
摘要
Interpreting clinical electroencephalography (EEG) is a laborious, subjective process, and existing computational models are limited to narrow classification tasks rather than holistic interpretation. A key bottleneck for applying powerful Large Vision-Language Models (LVLMs) to this domain is the scarcity of datasets pairing EEG visualizations with fine-grained, expert-level annotations. We address this by introducing CerebraGloss, an instruction-tuned LVLM for nuanced EEG interpretation. We first introduce a novel, automated data generation pipeline, featuring a bespoke YOLO-based waveform detector, to programmatically create a large-scale corpus of EEG-text instruction data. Using this data, we develop CerebraGloss, the first model of its kind capable of unified, generative analysis-performing tasks from detailed waveform description to multi-turn, context-aware dialogue. To evaluate this new capability, we construct and release CerebraGloss-Bench, a comprehensive benchmark for open-ended EEG interpretation. CerebraGloss demonstrates strong performance, surpassing leading LVLMs, including proprietary models like GPT-5, on this benchmark and achieving a new state-of-the-art on the TUSZ seizure detection task. Models, benchmark and tools are available at https://github.com/iewug/CerebraGloss .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper7
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 被引用 11,349 次
- Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIWei-Bang Jiang, Li-Ming Zhao, Bao-Liang LuICLR 2024 · 被引用 298 次
- EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG SignalsGuangyu Wang, Wenchao Liu, Yuhong He, Cong Xu 等NeurIPS 2024 · 被引用 267 次
- Improved Baselines with Visual Instruction TuningHaotian Liu, Chunyuan Li, Yuheng Li, Yong Jae LeeCVPR 2024
- Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data GenerationYue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta 等ACL 2025
相关 Paper
- EEG Agent: A Unified Framework for Automated EEG Analysis Using Large Language ModelsSha Zhao, Mingyi Peng, Haiteng Jiang, Tao Li 等AAAI 2026
- Seizure-Semiology-Suite(): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology UnderstandingLina Zhang, Jiarui Cui, Tonmoy Monsoor, Peizheng Li 等ICML 2026
- OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical TasksZhihao Peng, Cheng Wang, Shengyuan Liu, Zhiying Liang 等CVPR 2026 · 被引用 7 次
- NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG SignalsWeibang Jiang, Yansen Wang, Bao-Liang Lu, Dongsheng LiICLR 2025
- ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG InterpretationJiarui Jin, Haoyu Wang, Xingliang Wu, Xiaocheng Fang 等ICML 2026 · 被引用 11 次
