Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications through Visual Programming
Ruofei Du, Na Li, Jing Jin, Michelle Carney, Scott Miles, Maria Kleiner, Xiuxiu Yuan, Yinda Zhang, Anuva Kulkarni, Xingyu Liu, Ahmed Sabie, Sergio Orts-Escolano
Abstract
In recent years, there has been a proliferation of multimedia applications that leverage machine learning (ML) for interactive experiences. Prototyping ML-based applications is, however, still challenging, given complex workflows that are not ideal for design and experimentation. To better understand these challenges, we conducted a formative study with seven ML practitioners to gather insights about common ML evaluation workflows.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 11ac5309-e642-4823-afec-6ef970a78533Cited by top-tier papers7
- Jigsaw: Supporting Designers to Prototype Multimodal Applications by Chaining AI Foundation ModelsDavid Chuan-En Lin, Nikolas MartelaroCHI 2024 · 22 citations
- Evaluating Navigation and Comparison Performance of Computational Notebooks on Desktop and in Virtual RealitySungwon In, Eric Krokos, Kirsten Whitley, Chris North et al.CHI 2024 · 17 citations
- ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout TransitionXun Qian, Feitong Tan, Yinda Zhang, Brian Moreno Collins et al.CHI 2024 · 14 citations
- Canvil: Designerly Adaptation for LLM-Powered User ExperiencesK. J. Kevin Feng, Q. Vera Liao, Ziang Xiao, Jennifer Wortman Vaughan et al.CHI 2025 · 14 citations
- InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMsZhongyi Zhou, Jing Jin, Vrushank Phadnis, Xiuxiu Yuan et al.CHI 2025 · 10 citations
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Searching for MobileNetV3Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le et al.ICCV 2019 · 9,163 citations
- Digging Into Self-Supervised Monocular Depth EstimationClément Godard, Oisin Mac Aodha, Michael Firman, Gabriel J. BrostowICCV 2019 · 2,416 citations
- Scaling Laws for Reward Model OveroptimizationLeo Gao, John Schulman, Jacob HiltonICML 2023 · 963 citations
- Understanding Deep Networks via Extremal Perturbations and Smooth MasksRuth Fong, Mandela Patrick, Andrea VedaldiICCV 2019 · 480 citations
Related papers
- Analyzing Collaborative Challenges and Needs of UX Practitioners when Designing with AI/MLMeena Devii Muralikumar, David W. McDonaldCSCW 2024 · 6 citations
- AutoML in The Wild: Obstacles, Workarounds, and ExpectationsYuan Sun, Qiurong Song, Xinning Gui, Fenglong Ma et al.CHI 2023 · 28 citations
- Establishing Heuristics for Improving the Usability of GUI Machine Learning Tools for Novice UsersAsma Z. Yamani, Haifa Abdullah Al-Shammare, Malak BaslymanCHI 2024 · 3 citations
- Zeno: An Interactive Framework for Behavioral Evaluation of Machine LearningÁngel Alexander Cabrera, Erica Fu, Donald Bertucci, Kenneth Holstein et al.CHI 2023 · 51 citations
- How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?Agathe Balayn, Natasa Rikalo, Christoph Lofi, Jie Yang et al.CHI 2022 · 22 citations
