Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models
Stephen Brade, Bryan Wang, Maurício Sousa, Sageev Oore, Tovi Grossman
Abstract
Text-to-image generative models have demonstrated remarkable capabilities in generating high-quality images based on textual prompts. However, crafting prompts that accurately capture the user’s creative intent remains challenging. It often involves laborious trial-and-error procedures to ensure that the model interprets the prompts in alignment with the user’s intention. To address these challenges, we present Promptify, an interactive system that supports prompt exploration and refinement for text-to-image generative models. Promptify utilizes a suggestion engine powered by large language models to help users quickly explore and craft diverse prompts. Our interface allows users to organize the generated images flexibly, and based on their preferences, Promptify suggests potential changes to the original prompt. This feedback loop enables users to iteratively refine their prompts and enhance desired features while avoiding unwanted ones. Our user study shows that Promptify effectively facilitates the text-to-image workflow, allowing users to create visually appealing images on their first attempt while requiring significantly less cognitive load than a widely-used baseline tool.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 8eaa0db1-a74b-454a-b9ed-53033b0fbcc0Cited by top-tier papers69
- Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-CreationSangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li et al.CHI 2024 · 143 citations
- ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis TestingIan Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg et al.CHI 2024 · 141 citations
- CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIDaEun Choi, Sumin Hong, Jeongeon Park, John Joon Young Chung et al.CHI 2024 · 116 citations
- PromptCharm: Text-to-Image Generation through Multi-modal Prompting and RefinementZhijie Wang, Yuheng Huang, Da Song, Lei Ma et al.CHI 2024 · 111 citations
- When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design TaskYuanning Han, Ziyi Qiu, Jiale Cheng, Ray LCCHI 2024 · 103 citations
Builds on16
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
Related papers
- PromptMagician: Interactive Prompt Engineering for Text-to-Image CreationYingchaojie Feng, Xingbo Wang, Kamkwai Wong, Sijia Wang et al.IEEE VIS 2023 · 127 citations
- Design Guidelines for Prompt Engineering Text-to-Image Generative ModelsVivian Liu, Lydia B. ChiltonCHI 2022 · 586 citations
- A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image SynthesisNailei Hei, Qianyu Guo, Zihao Wang, Yan Wang et al.AAAI 2024 · 11 citations
- SketchFlex: Facilitating Spatial-Semantic Coherence in Text-to-Image Generation with Region-Based SketchesHaichuan Lin, Yilin Ye, Jiazhi Xia, Wei ZengCHI 2025 · 25 citations
- Is It AI or Is It Me? Understanding Users' Prompt Journey with Text-to-Image Generative AI ToolsAtefeh Mahdavi Goloujeh, Anne Sullivan, Brian MagerkoCHI 2024 · 88 citations
