FactVerse: A Benchmark for Factual Consistency in Interleaved Image-Text Generation

Interleaved multimodal understanding and generation-where models can interactively comprehend and produce images and text in arbitrary orders-has emerged as a key research direction in generative Multimodal Large Language Models(MLLMs). Such interleaved image-text content plays an increasingly important role in information dissemination. However, the compounded persuasive power of multimodal narratives also raises the risk of factual misinformation. Despite this, existing benchmarks lack effective mechanisms to evaluate factual consistency in interleaved image-text content. To bridge this gap, we introduce Fact-Verse, a benchmark dedicated to evaluating factual consistency in interleaved image-text generation. FactVerse comprises 3,000 humanverified instances across four categories and 50 domains, supporting both English and Chinese. We also establish a multi-dimensional evaluation framework designed to rigorously assess factual consistency. Experiments demonstrate that our framework achieves high alignment with human judgments, significantly outperforming existing evaluation methods. Furthermore, our analysis reveals systematic deficiencies in current models, offering critical insights for future design.