Lune

ACL2026Top-tier venue

Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection

Minghui Jia, Qichao Zhang, Ali Luo, Linjing Li, Shuo Ye, Hailing Lu, Wen Hou, Dongbin Zhao

Abstract

Due to the limited generalization and interpretability of deep learning classifiers, The final vetting of rare celestial object candidates still relies on expert visual inspection--a manually intensive process. In this process, astronomers leverage specialized tools to analyze spectra and construct reliable catalogs. However, this practice has become the primary bottleneck, as it is fundamentally incapable of scaling with the data deluge from modern spectroscopic surveys. To bridge this gap, we propose Spec-o3, a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection via interleaved multimodal chain-of-thought reasoning. Spec-o3 is trained with a two-stage post-training recipe: cold-start supervised fine-tuning on expert inspection trajectories followed by outcome-based reinforcement learning on rare-type verification tasks. Evaluated on five rare-object identification tasks from LAMOST, Spec-o3 establishes a new State-of-the-Art, boosting the macro-F1 score from 28.3 to 76.5 with a 7B parameter base model and outperforming both proprietary VLMs and specialized deep models. Crucially, the agent demonstrates strong generalization to unseen inspection tasks across survey shifts (from LAMOST to SDSS/DESI). Expert evaluations confirm that its reasoning traces are coherent and physically consistent, supporting transparent and trustworthy decision-making. Code, data, and models are available at https://github.com/Maxwell-Jia/spec-o3.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Visual Instruction Tuning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  • SGLang: Efficient Execution of Structured Language Model Programs cites Efficient Memory Management for Large Language Model Serving with PagedAttention
  • DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning cites Visual Instruction Tuning
  • DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning cites BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on13

  1. Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
  2. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
  3. SGLang: Efficient Execution of Structured Language Model ProgramsLianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun et al.NeurIPS 2024 · 1,586 citations
  4. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
  5. DeepEyes: Incentivizing "Thinking with Images" via Reinforcement LearningZiwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao et al.ICLR 2026 · 321 citations

Related papers

  1. InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual SearchKaican Li, Lewei Yao, Jiannan Wu, Tiezheng YU et al.ICLR 2026 · 10 citations
  2. Ophiuchus: Incentivizing Tool-augmented ''Think with Images'' for Joint Medical Segmentation, Understanding and ReasoningYankai Jiang, Yujie Zhang, Peng Zhang, Wenjie Li et al.ICML 2026
  3. Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual SearchXin Lai, Junyi Li, Wei Li, Tao Liu et al.ICLR 2026 · 124 citations
  4. SpotAgent: Grounding Visual Geo-localization in Large Vision-Language Models through Agentic ReasoningFurong Jia, Ling Dai, Wenjin Deng, Fan Zhang et al.KDD 2026 · 6 citations
  5. Learning to Select Visual Tools from ExperienceZeyi Huang, Yuyang Ji, Anirudh Sundara Rajan, Zefan Cai et al.CVPR 2026
Dusk over the sea between two cliffs drawn in fine vertical lines