SheetDesigner: MLLM-Powered Spreadsheet Layout Generation with Rule-Based and Vision-Based Reflection
Qin Chen, Yuanyi Ren, Xiaojun Ma, Mugeng Liu, Shi Han, Dongmei Zhang
Abstract
Spreadsheets are critical to data-centric tasks, with rich, structured layouts that enable efficient information transmission. Given the time and expertise required for manual spreadsheet layout design, there is an urgent need for automated solutions. However, existing automated layout models are ill-suited to spreadsheets, as they often (1) treat components as axis-aligned rectangles with continuous coordinates, overlooking the inherently discrete, gridbased structure of spreadsheets; and (2) neglect interrelated semantics, such as data dependencies and contextual links, unique to spreadsheets. In this paper, we first formalize the spreadsheet layout generation task, supported by a seven-criterion evaluation protocol and a dataset of 3,326 spreadsheets. We then introduce SheetDesigner, a zero-shot and trainingfree framework using Multimodal Large Language Models (MLLMs) that combines rule and vision reflection for component placement and content population. SheetDesigner outperforms five baselines by at least 22.6%. We further find that through vision modality, MLLMs handle overlap and balance well but struggle with alignment, necessitates hybrid rule and visual reflection strategies. Our codes and data is available at Github.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers2
- DV-World: Benchmarking Data Visualization Agents in Real-World ScenariosJinxiang Meng, Shaoping Huang, Fangyu Lei, Jingyu Guo et al.ICML 2026 · 1 citation
- Representation Interventions Enable Lifelong Knowledge Memory Control in LLMsXuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu et al.ACL 2026
Builds on13
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- LayoutTransformer: Layout Generation and Completion with Self-attentionKamal Gupta, Justin Lazarow, Alessandro Achille, Larry Davis et al.ICCV 2021 · 184 citations
- Constrained Graphic Layout Generation via Latent OptimizationKotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, Kota YamaguchiACM MM 2021 · 80 citations
- LayoutPrompter: Awaken the Design Ability of Large Language ModelsJiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang Yang et al.NeurIPS 2023 · 71 citations
- Coarse-to-Fine Generative Modeling for Graphic LayoutsZhaoyun Jiang, Shizhao Sun, Jihua Zhu, Jian-Guang Lou et al.AAAI 2022 · 54 citations
Related papers
- Seeing is Improving: Visual Feedback for Iterative Text Layout RefinementJunrong Guo, Shancheng Fang, Yadong Qu, Hongtao XieCVPR 2026 · 2 citations
- SheetAgent: Towards a Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language ModelsYibin Chen, Yifu Yuan, Zeyu Zhang, Yan Zheng et al.WWW 2025 · 13 citations
- SheetCopilot: Bringing Software Productivity to the Next Level through Large Language ModelsHongxin Li, Jingran Su, Yuntao Chen, Qing Li et al.NeurIPS 2023 · 75 citations
- Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format ReasoningHouxing Ren, Mingjie Zhan, Zimu Lu, Ke Wang et al.ACL 2026
- Encoding Spreadsheets for Large Language ModelsHaoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong et al.EMNLP 2024 · 4 citations
