FreeText: Training-Free Text Rendering via Attention Localization and Spectral Glyph Injection
RuiQiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang
摘要
Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multiline layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically require costly retraining or rigid external layout constraints, which can degrade aesthetics and limit flexibility. We propose FreeText, a training-free, plug-and-play framework that improves text rendering by exploiting intrinsic mechanisms of Diffusion Transformer (DiT) models. FreeText decomposes the problem into where to write and what to write. For where to write, we localize writing regions by reading token-wise spatial attribution from endogenous image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For what to write, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and suppress semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while largely preserving semantic alignment and aesthetic quality, with modest inference overhead.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper10
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 被引用 5,568 次
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari 等ICML 2024 · 被引用 3,620 次
- Vision Transformers Need RegistersTimothée Darcet, Maxime Oquab, Julien Mairal, Piotr BojanowskiICLR 2024 · 被引用 769 次
- Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion ModelsHila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf 等SIGGRAPH 2023 · 被引用 438 次
- Blended Latent DiffusionOmri Avrahami, Ohad Fried, Dani LischinskiSIGGRAPH 2023 · 被引用 339 次
相关 Paper
- AMO Sampler: Enhancing Text Rendering with OvershootingXixi Hu, Keyang Xu, Bo Liu, Qiang Liu 等CVPR 2025
- ViType: High-Fidelity Visual Text Rendering via Glyph-Aware Multimodal DiffusionLishuai Gao, Jun-Yan He, Yingsen Zeng, Yujie Zhong 等AAAI 2026
- Prompt Reinjection: Alleviating Prompt Forgetting in Multimodal Diffusion TransformersYuxuan Yao, Yuxuan Chen, Hui Li, Kaihui Cheng 等ICML 2026
- FreeCus: Free Lunch Subject-Driven Customization in Diffusion TransformersYanbing Zhang, Zhe Wang, Qin Zhou, Mengping YangICCV 2025
- EasyText: Controllable Diffusion Transformer for Multilingual Text RenderingRunnan Lu, Yuxuan Zhang, Jiaming Liu, Haofan Wang 等AAAI 2026 · 被引用 20 次
