CVPR Findings2026
LLM Guided Multi Style Typography and Layout Generation via Dynamic Direct Preference Optimization
Chen Fu, Shengzhou Yi, Ling Xiao, Toshihiko Yamasaki
Abstract
Recent progress in large language models (LLMs) has created new opportunities to connect linguistic intent with visual layout and typography design. We present UniLayout Generator, a unified LLM-guided framework for multi-style typography and layout generation that translates naturallanguage prompts into explicit design parameters, including font attributes, spatial hierarchy, and composition. By using an LLM as a planner, our framework enables controllable and content-aware typography generation across languages. To further improve readability, accessibility, and visual quality, we introduce Dynamic Direct Preference Optimization (D-DPO), a fine-tuning scheme that adaptively reweights preference pairs based on multimodal evaluation signals, including OCR-F1, LPIPS, WCAG contrast compliance, and human aesthetic judgments. Experiments on public layout benchmarks show that UniLayout Generator with D-DPO consistently improves readability and accessibility while maintaining strong visual quality compared with competitive baselines. These results highlight the value of explicit language-guided planning and multi-objective preference alignment for generative typography.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- LayoutGPT: Compositional Visual Planning and Generation with Large Language Models cites LayoutTransformer: Layout Generation and Completion with Self-attention
- Diffusion Model Alignment Using Direct Preference Optimization cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- LayoutPrompter: Awaken the Design Ability of Large Language Models cites LayoutTransformer: Layout Generation and Completion with Self-attention
Builds on9
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- LayoutGPT: Compositional Visual Planning and Generation with Large Language ModelsWeixi Feng, Wanrong Zhu, Tsu-Jui Fu, Varun Jampani et al.NeurIPS 2023 · 462 citations
- LayoutTransformer: Layout Generation and Completion with Self-attentionKamal Gupta, Justin Lazarow, Alessandro Achille, Larry Davis et al.ICCV 2021 · 184 citations
- Diffusion Model Alignment Using Direct Preference OptimizationBram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou et al.CVPR 2024 · 89 citations
- LayoutPrompter: Awaken the Design Ability of Large Language ModelsJiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang et al.NeurIPS 2023 · 71 citations
Related papers
- AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language ModelsSohan Patnaik, Rishabh Jain, Balaji Krishnamurthy, Mausoom SarkarCVPR 2025
- Uni-Layout: Integrating Human Feedback in Unified Layout Generation and EvaluationShuo Lu, Yanyin Chen, Wei Feng, Jiahao Fan et al.ACM MM 2025 · 1 citation
- AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference ModelsQi Liu, Jingqing Ruan, Hao Li, Haodong Zhao et al.ACL Findings 2025 · 2 citations
- mDPO: Conditional Preference Optimization for Multimodal Large Language ModelsFei Wang, Wenxuan Zhou, James Y. Huang, Nan Xu et al.EMNLP 2024 · 11 citations
- Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference OptimizationYurui Pan, Ke Xu, Bo PengACL Findings 2026
