Deterministic Component Mining for Multi-Framework UI2Code Generation
Zixiong Yang, Linxiao Li, Jiaye Lin, Binrui Wu, Xiaoyu Kang, Jiechao Gao
摘要
Automating User Interface (UI) generation substantially improves productivity and accelerates development by reducing engineering time and manual effort. Despite recent progress of Multimodal Large Language Models (MLLMs) in UI2Code, most existing approaches focus on a single HTML/CSS form and fail to systematically incorporate front-end frameworks such as React, Vue, and Angular. Moreover, their outputs are often verbose and hard to reuse at the component level. To address those issues, we propose Deterministic Component Mining (DCM), a multi-stage pipeline that couples MLLMs with a compact intermediate representation to enable multi-framework and component-oriented code generation. Firstly, a lightweight structure model predicts the representation of the DOM tree in JSON format, capturing the coarse layout from a webpage screenshot. Subsequently, we formulate deterministic rules to normalize the predicted DOM tree and mine reusable components with repetitive patterns via structural hashing and clustering, thereby yielding a portable intermediate representation. Finally, we employ a framework-conditioned prompting strategy governed by a binding specification and a file-block protocol to emit HTML/React/Vue/Angular code with explicit component props and repeat constructs. Extensive experiments demonstrate that DCM significantly outperforms baselines on automatic evaluation metrics and component-level reuse, while delivering consistent gains in multi-framework portability and overall code structural quality.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper16
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language TasksJiannan Wu, Muyan Zhong, Sen Xing, Zeqiang Lai 等NeurIPS 2024 · 被引用 179 次
- ENCODER: Entity Mining and Modification Relation Binding for Composed Image RetrievalZixu Li, Zhiwei Chen, Haokun Wen, Zhiheng Fu 等AAAI 2025 · 被引用 59 次
- WebCode2M: A Real-World Dataset for Code Generation from Webpage DesignsYi Gui, Zhen Li, Yao Wan, Yemin Shi 等WWW 2025 · 被引用 38 次
- ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image RetrievalZixu Li, Yupeng Hu, Zhiwei Chen, Mingyu Zhang 等CVPR 2026 · 被引用 16 次
相关 Paper
- MulFCoder: Framework-conditioned Multi-agent for MLLM-based Multi-framework Front-end Code GenerationJie Wu, Haoran Ma, Shisong Tang, Yulin Xu 等ICML 2026
- Component-based Reusable UI Code Generation for Complex Websites via Semantic Segmentation and Fine-grained FeedbackJingyu Xiao, Jiantong Qin, Shuoqi Li, Man Ho Lam 等KDD 2026 · 被引用 1 次
- DeclarUI: Bridging Design and Development with Automated Declarative UI Code GenerationTing Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun 等FSE 2025 · 被引用 13 次
- MLLM-Based UI2Code Automation Guided by UI Layout InformationFan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen 等ISSTA 2025 · 被引用 5 次
- Widget2Code: From Visual Widgets to UI Code via Multimodal LLMsHouston H. Zhang, Tao Zhang, Baoze Lin, Yuanqi Xue 等CVPR 2026 · 被引用 9 次
