WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code

With the rapid advancement of Generative AI technology, Multimodal Large Language Models(MLLMs) have the potential to act as AI software engineers capable of executing complex web application development. Considering that the model requires a confluence of multidimensional sub-capabilities to address the challenges of various development phases, constructing a multi-view evaluation framework is crucial for accurately guiding the enhancement of development efficiency. However, existing benchmarks usually fail to provide an assessment of subcapabilities and focus solely on webpage generation outcomes. In this work, we draw inspiration from the principles of software engineering and further propose WebUIBench, a benchmark systematically designed to evaluate MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI-to-Code. WebUIBench comprises 21K high-quality question-answer pairs derived from over 0.7K real-world websites. The extensive evaluation of 29 mainstream MLLMs uncovers the skill characteristics and various weakness that models encountered during the development process. Question: Please locate the object element with the text content of the [Business] A. <img> and <input> C. <input> and <button> B. <img> and <button> D. None of the above Question: Please select the HTML tag that appears in the screenshot of the webpage Question: Please provide the text in the red box in this screenshot of the webpage? Question: Please provide the color of the text [Herboren] in RGB format? Choice： Question: Please select the font style of the text [Herboren] A. Normal B. Italic Choice: Corrupted Code： Question：There is a syntax error in the following HTML code. Please correct it Code： Question：Please add the attribute height "160px" and width "120px" to <span> Correct or Unrelated Code： <div class=""> <img src=""/> </div> Question: Please check whether the front-end code snippet matches a certain part of the webpage screenshot Question: Divide the image evenly into 16 regions…Please give the area number of the text [Prodotti] A .