Freestyle Layout-to-Image Synthesis
Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang
Abstract
Layout "train bush grass railroad" "lego train bush grass railroad" "an ink painting of train bush grass railroad" "warehouse bush grass railroad" Layout "bench cat building bush furniture grass ground pavement roof sky tree window" "bench tabby cat building bush furniture grass ground pavement roof sky tree window" "a sketch of bench cat building bush furniture grass ground pavement roof sky tree window" "bench unicorn building bush furniture grass ground pavement roof sky tree window" Figure 1 . Freestyle Layout-to-Image Synthesis (FLIS) results generated by using our model. Each has two kinds of inputs: a layout of semantic masks (on the 1st column), and a text (on the top of each result). For each layout, we show three example results with edited texts (3rd-5th columns). They validate that our model is able to introduce new attributes (3rd column), styles (4th column), and objects (5th column), which are all unseen during training, in the synthesized images. The generated hornless unicorn is due to the layout constraint.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 88a8bc8a-b7cb-46f8-b4d8-71c2c4de5615Cited by top-tier papers33
- FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation ModelsLihe Yang, Xiaogang Xu, Bingyi Kang, Yinghuan Shi et al.NeurIPS 2023 · 94 citations
- SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image GenerationChengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai et al.AAAI 2024 · 30 citations
- SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified FlowChaoyang Wang, Xiangtai Li, Lu Qi, Henghui Ding et al.NeurIPS 2024 · 25 citations
- SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion ModelTao Wu, Xuewei Li, Zhongang Qi, Di Hu et al.AAAI 2024 · 24 citations
- Adversarial Supervision Makes Layout-to-Image Diffusion Models ThriveYumeng Li, Margret Keuper, Dan Zhang, Anna KhorevaICLR 2024 · 20 citations
Builds on28
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 11,743 citations
Related papers
- Semantic Image Synthesis with Unconditional GeneratorJungwoo Chae, Hyunin Cho, Sooyeon Go, Kyungmook Choi et al.NeurIPS 2023 · 6 citations
- LoCo: Training-Free Layout-to-Image Synthesis with Localized ConstraintsPeiang Zhao, Han Li, Ruiyang Jin, S. Kevin ZhouACM MM 2025 · 2 citations
- Image Synthesis From Reconfigurable Layout and StyleWei Sun, Tianfu WuICCV 2019 · 160 citations
- ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image IntermediaryZeqi Gu, Yin Cui, Zhaoshuo Li, Fangyin Wei et al.CVPR 2025
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image TranslationNarek Tumanyan, Michal Geyer, Shai Bagon, Tali DekelCVPR 2023
