Lune

ACL Findings2025

'No' Matters: Out-of-Distribution Detection in Multimodality Multi-Turn Interactive Dialogue Download PDF

Rena Wei Gao, Xuetong Wu, Siwen Luo, Caren Han, Feng Liu

Abstract

Out-of-distribution (OOD) detection in multimodal contexts is essential for identifying deviations in different modalities, particularly for interactive dialogue systems in real-life interactions, where the systems are usually infeasible to deploy large language models (LLMs) to generate responses due to data privacy and ethical issues. This paper aims to improve label detection that involves multiround long dialogues by efficiently detecting OOD dialogues and images. We introduce a novel scoring framework named Dialogue Image Aligning and Enhancing Framework (DIAEF) that integrates the visual language models with the novel proposed scores that detect OOD in two key scenarios (1) mismatches between the dialogue and image input pair and (2) input pairs with previously unseen labels. Our experimental results, derived from various benchmarks, demonstrate that integrating image and multi-round dialogue OOD detection is more effective with previously unseen labels than using either modality independently. In the presence of mismatched pairs, our proposed score effectively identifies these mismatches and demonstrates strong robustness in long dialogues, with user case as evidence to demonstrate the effectiveness of proposed function for applications. This approach enhances domain-aware, adaptive conversational agents and establishes baselines for future studies. 1 1 Code can be found in https://github.com/ wfyitf/multimodal_ood.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Exploring the Limits of Out-of-Distribution Detection cites Learning Transferable Visual Models From Natural Language Supervision
  • Scaling Out-of-Distribution Detection for Real-World Settings cites Exploring the Limits of Out-of-Distribution Detection
  • BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation cites Learning Transferable Visual Models From Natural Language Supervision
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models cites BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsJunnan Li, Dongxu Li, Silvio Savarese, Steven C. H. HoiICML 2023 · 7,873 citations
  3. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
  4. Scaling Out-of-Distribution Detection for Real-World SettingsDan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou et al.ICML 2022 · 653 citations
  5. Exploring the Limits of Out-of-Distribution DetectionStanislav Fort, Jie Ren, Balaji LakshminarayananNeurIPS 2021 · 443 citations

Related papers

  1. Concept Matching with Agent for Out-of-Distribution DetectionYuxiao Lee, Xiaofeng Cao, Jingcai Guo, Wei Ye et al.AAAI 2025 · 8 citations
  2. Negative Label Guided OOD Detection with Pretrained Vision-Language ModelsXue Jiang, Feng Liu, Zhen Fang, Hong Chen et al.ICLR 2024 · 73 citations
  3. TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center LearningJinglun Li, Xinyu Zhou, Kaixun Jiang, Lingyi Hong et al.ACM MM 2024 · 1 citation
  4. Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal RepresentationsJeonghyeon Kim, Sangheum HwangCVPR 2025
  5. ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and ReasoningWenjie Zhu, Yabin Zhang, Xin Jin, Wenjun Zeng et al.CVPR 2026 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines