Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation
Dahee Kwon, Haeun Lee, Jaesik Choi
摘要
Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text–image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper16
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari 等ICML 2024 · 被引用 3,620 次
- MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and EditingMingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan 等ICCV 2023 · 被引用 770 次
- Reliable Fidelity and Diversity Metrics for Generative ModelsMuhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi 等ICML 2020 · 被引用 553 次
- Prompt-to-Prompt Image Editing with Cross-Attention ControlAmir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman 等ICLR 2023 · 被引用 361 次
相关 Paper
- On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion TransformersOmer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-OrSIGGRAPH 2026 · 被引用 1 次
- Letting Trajectories Spread: Quality-Preserving Control for Diverse Flow MatchingJingxuan Wu, Zhenglin Wan, Xingrui Yu, Yuzhe YANG 等ICML 2026 · 被引用 1 次
- It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion ModelsAnne Harrington, A. Sophia Koepke, Shyamgopal Karthik, Trevor Darrell 等CVPR 2026 · 被引用 12 次
- GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image GenerationYe Zhu, Kaleb Newman, Johannes Lutzeyer, Adriana Romero-Soriano 等ICML 2026 · 被引用 1 次
- SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive GenerationYoungwoo Shin, Jiwan Hur, Junmo KimICLR 2026 · 被引用 1 次
