Crepe: A Mobile Screen Data Collector Using Graph Query
Yuwen Lu, Meng Chen, Qi Zhao, Victor V. Cox, Yang Yang, Meng Jiang, Jay Brockman, Tamara Kay, Toby Jia-Jun Li
Abstract
Collecting mobile datasets remains challenging for academic researchers due to limited data access and technical barriers. Commercial organizations often possess exclusive access to mobile data, leading to a "data monopoly" that restricts the independence of academic research. Existing open-source mobile data collection frameworks primarily focus on mobile sensing data rather than screen content, which is crucial for various research studies. We present Crepe, a no-code Android app that enables researchers to collect information displayed on screen through simple demonstrations of target data. Crepe utilizes a novel Graph Query technique which augments the structures of mobile UI screens to support flexible identification, location, and collection of specific data pieces. The tool emphasizes participants' privacy and agency by providing full transparency over collected data and allowing easy opt-out. We designed and built Crepe for research purposes only and in scenarios where researchers obtain explicit consent from participants. Code for Crepe will be open-sourced to support future academic research data collection.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers2
- Uncovering Disparities in Rideshare Drivers’ Earning and Work Patterns: A Case Study of ChicagoHy Dang, Yuwen Lu, Jason Spicer, Tamara Kay et al.CSCW 2026
- DroidRetriever: A Transparent and Steerable Automation System for Collaborative Mobile Information SeekingYiheng Bian, Yunpeng Song, Guiyu Ma, Rongrong Zhu et al.CHI 2026
Builds on7
- Enabling Conversational Interaction with Mobile UI using Large Language ModelsBryan Wang, Gang Li, Yang LiCHI 2023 · 149 citations
- Screen2Words: Automatic Mobile UI Summarization with Multimodal LearningBryan Wang, Gang Li, Xin Zhou, Zhourong Chen et al.UIST 2021 · 97 citations
- Screen2Vec: Semantic Embedding of GUI Screens and GUI ComponentsToby Jia-Jun Li, Lindsay Popowski, Tom M. Mitchell, Brad A. MyersCHI 2021 · 72 citations
- Bargaining with the Black-Box: Designing and Deploying Worker-Centric Tools to Audit Algorithmic ManagementDan Calacci, Alex PentlandCSCW 2022 · 57 citations
- An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and BehaviorsChaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye et al.CHI 2024 · 25 citations
Related papers
- Abandon All Hope Ye Who Enter Here: A Dynamic, Longitudinal Investigation of Android's Data Safety SectionIoannis Arkalakis, Michalis Diamantaris, Serafeim Moustakas, Sotiris Ioannidis et al.USENIX Security 2024 · 13 citations
- PTPDroid: Detecting Violated User Privacy Disclosures to Third-Parties of Android AppsZeya Tan, Wei SongICSE 2023 · 20 citations
- Re-identification of mobile devices using real-time bidding advertising networksKeen Sung, Jianyi Huang, Mark D. Corner, Brian Neil LevineMobiCom 2020 · 6 citations
- Understanding Worldwide Private Information Collection on AndroidYun Shen, Pierre-Antoine Vervier, Gianluca StringhiniNDSS 2021
- PriviAware: Exploring Data Visualization and Dynamic Privacy Control Support for Data Collection in Mobile Sensing ResearchHyunsoo Lee, Yugyeong Jung, Hei Yiu Law, Seolyeong Bae et al.CHI 2024 · 11 citations
