Component-based Reusable UI Code Generation for Complex Websites via Semantic Segmentation and Fine-grained Feedback
Jingyu Xiao, Jiantong Qin, Shuoqi Li, Man Ho Lam, Yuxuan Wan, Jen-tse Huang, Yintong Huo, Michael R. Lyu
摘要
Multimodal Large Language Models (MLLMs) have demonstrated strong performance on the UI-to-code task, which aims to generate UI code from design mock-ups. However, when applied to long and complex websites, they often struggle with fragmented segmentation, redundant code generation for repetitive components, and frequent UI inconsistencies. To systematically investigate and address these challenges, we introduce ComUIBench, a new multi-page complex webpage benchmark with component annotations, designed to evaluate MLLMs' ability to generate reusable UI code in realistic website scenarios. Building upon this benchmark, we propose ComUICoder, a component-based UI code generation framework that emphasizes semantic-aware segmentation, code reuse, and fine-grained refinement. Specifically, ComUICoder incorporates (1) Hybrid Semantic-aware Block Segmentation for accurate UI semantic coherent block detection, (2) Visual-aware Graph-based Block Merge to consolidate structurally similar components within and across webpages for reusable implementation, and (3) Priority-based Element-wise Feedback to refine generated code and reduce element-level inconsistencies. Extensive experiments demonstrate that ComUICoder significantly improves overall generation quality and code reusability on complex multipage websites. Our datasets and code are publicly available at https://github.com/WebPAI/ComUICoder.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- MLLM-Based UI2Code Automation Guided by UI Layout InformationFan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen 等ISSTA 2025 · 被引用 5 次
- WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code GenerationRabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li 等EMNLP 2025
- EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code GenerationJingyu Xiao, Zhongyi Zhang, Yuxuan Wan, Yintong Huo 等FSE 2026
- Divide-and-Conquer: Generating UI Code from ScreenshotsYuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang 等FSE 2025 · 被引用 10 次
- Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive PrototypingJingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang 等ASE 2025 · 被引用 1 次
