Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation
Dahee Kwon, Haeun Lee, Jaesik Choi
Abstract
Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text–image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on16
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari et al.ICML 2024 · 3,620 citations
- MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and EditingMingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan et al.ICCV 2023 · 770 citations
- Reliable Fidelity and Diversity Metrics for Generative ModelsMuhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi et al.ICML 2020 · 553 citations
- Prompt-to-Prompt Image Editing with Cross-Attention ControlAmir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman et al.ICLR 2023 · 361 citations
Related papers
- On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion TransformersOmer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-OrSIGGRAPH 2026 · 1 citation
- Letting Trajectories Spread: Quality-Preserving Control for Diverse Flow MatchingJingxuan Wu, Zhenglin Wan, Xingrui Yu, Yuzhe YANG et al.ICML 2026 · 1 citation
- It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion ModelsAnne Harrington, A. Sophia Koepke, Shyamgopal Karthik, Trevor Darrell et al.CVPR 2026 · 12 citations
- GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image GenerationYe Zhu, Kaleb Newman, Johannes Lutzeyer, Adriana Romero-Soriano et al.ICML 2026 · 1 citation
- SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive GenerationYoungwoo Shin, Jiwan Hur, Junmo KimICLR 2026 · 1 citation
