TextGround4M: A Prompt-Aligned Dataset for Layout-Aware Text Rendering
Dongxing Mao, Yilin Wang, Linjie Li, Zhengyuan Yang, Alex Jinpeng Wang
摘要
Despite recent advances in text-to-image (T2I) generation, models still struggle to accurately render prompt-specified text with correct spatial layout—especially in multi-span, structured settings. This challenge is driven not only by the lack of datasets that align prompts with the exact text and layout expected in the image, but also by the absence of effective metrics for evaluating layout quality. To address these issues, we introduce TextGround4M, a large-scale dataset of over 4 million prompt-image pairs, each annotated with span-level text grounded in the prompt and corresponding bounding boxes. This enables fine-grained supervision for layout-aware, prompt-grounded text rendering. Building on this, we propose a lightweight training strategy for autoregressive T2I models that appends layout-aware span tokens during training, without altering model architecture or inference behavior. We further construct a benchmark with stratified layout complexity to evaluate both open-source and proprietary models in a zero-shot setting. In addition, we introduce two layout-aware metrics to address the long-standing lack of spatial evaluation in text rendering. Our results show that models trained on TextGround4M outperform strong baselines in text fidelity, spatial accuracy, and prompt consistency, highlighting the importance of fine-grained layout supervision for grounded T2I generation.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper11
- Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 被引用 6,759 次
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari 等ICML 2024 · 被引用 3,620 次
- Muse: Text-To-Image Generation via Masked Generative TransformersHuiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot 等ICML 2023 · 被引用 751 次
- Composer: Creative and Controllable Image Synthesis with Composable ConditionsLianghua Huang, Di Chen, Yu Liu, Yujun Shen 等ICML 2023 · 被引用 371 次
- GlyphControl: Glyph Conditional Control for Visual Text GenerationYukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang 等NeurIPS 2023 · 被引用 163 次
相关 Paper
- TextAtlas5M: A Large-Scale Dataset for Long Text Image GenerationDongxing Mao, Alex Jinpeng Wang, weiming Han, Jiawei Zhang 等ICML 2026
- TextDiffuser: Diffusion Models as Text PaintersJingye Chen, Yupan Huang, Tengchao Lv, Lei Cui 等NeurIPS 2023 · 被引用 290 次
- InstanceGen: Image Generation with Instance-level InstructionsEtai Sella, Yanir Kleiman, Hadar Averbuch-ElorSIGGRAPH 2025 · 被引用 2 次
- GOAL: Grounded text-to-image Synthesis with Joint Layout Alignment TuningYaqi Li, Han Fang, Zerun Feng, Kaijing Ma 等ACM MM 2024 · 被引用 1 次
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image GenerationJiayu Xiao, Henglei Lv, Liang Li, Shuhui Wang 等ICLR 2024 · 被引用 31 次
