ACL Findings2025
ProBench: Judging Multimodal Foundation Models on Open-ended Multi-domain Expert Tasks
Yan Yang, Dongxu Li, Haoning Wu, Bei Chen, Liu Liu, Liyuan Pan, Junnan Li
Abstract
Query: i want you to write a Rshiny code in rstudio to generate above visualization. Can you do that? Coding; Screenshots and UI Elements Task sub-field: Code Generation
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on10
- Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng et al.SOSP 2023 · 1,016 citations
- AlpacaFarm: A Simulation Framework for Methods that Learn from Human FeedbackYann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang et al.NeurIPS 2023 · 948 citations
- Deduplicating Training Data Makes Language Models BetterKatherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang et al.ACL 2022 · 844 citations
- ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical ReasoningAhmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq R. Joty et al.ACL Findings 2022 · 389 citations
- MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding BenchmarkXiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang et al.ACL 2025 · 377 citations
Related papers
- Lux: Always-on Visualization Recommendations for Exploratory Dataframe WorkflowsDoris Jung Lin Lee, Dixin Tang, Kunal Agarwal, Thyne Boonmark et al.VLDB 2022 · 61 citations
- PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic ContextXinyun Chen, Linyuan Gong, Alvin Cheung, Dawn SongACL 2021
- Integrated Development Environment with Interactive Scatter Plot for Examining Statistical ModelingKentaro Asai, Tsukasa Fukusato, Takeo IgarashiCHI 2020 · 3 citations
- B2: Bridging Code and Interactive Visualization in Computational NotebooksYifan Wu, Joseph M. Hellerstein, Arvind SatyanarayanUIST 2020 · 71 citations
- Visualization by exampleChenglong Wang, Yu Feng, Rastislav Bodík, Alvin Cheung et al.POPL 2020 · 36 citations
