Restoring Execution Environments of Jupyter Notebooks
Jiawei Wang, Li Li, Andreas Zeller
Abstract
More than ninety percent of published Jupyter notebooks do not state dependencies on external packages. This makes them non-executable and thus hinders reproducibility of scientific results. We present SnifferDog, an approach that 1) collects the APIs of Python packages and versions, creating a database of APIs; 2) analyzes notebooks to determine candidates for required packages and versions; and 3) checks which packages are required to make the notebook executable (and ideally, reproduce its stored results). In its evaluation, we show that SnifferDog precisely restores execution environments for the largest majority of notebooks, making them immediately executable for end users.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers14
- The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-LargeSumon Biswas, Mohammad Wardat, Hridesh RajanICSE 2022 · 64 citations
- Subtle Bugs Everywhere: Generating Documentation for Data Wrangling CodeChenyang Yang, Shurui Zhou, Jin L. C. Guo, Christian KästnerASE 2021 · 25 citations
- Knowledge-Based Environment Dependency Inference for Python ProgramsHongjie Ye, Wei Chen, Wensheng Dou, Guoquan Wu et al.ICSE 2022 · 21 citations
- LExecutor: Learning-Guided ExecutionBeatriz Souza, Michael PradelFSE 2023 · 16 citations
- Conflict-aware Inference of Python Compatible Runtime Environments with Domain Knowledge GraphWei Cheng, Xiangrong Zhu, Wei HuICSE 2022 · 16 citations
Builds on4
- Assessing and Restoring Reproducibility of Jupyter NotebooksJiawei Wang, Tzu-yang Kuo, Li Li, Andreas ZellerASE 2020 · 68 citations
- Watchman: monitoring dependency conflicts for Python library ecosystemYing Wang, Ming Wen, Yepang Liu, Yibo Wang et al.ICSE 2020 · 65 citations
- Automated Third-Party Library Detection for Android Applications: Are We There Yet?Xian Zhan, Lingling Fan, Tianming Liu, Sen Chen et al.ASE 2020 · 55 citations
- Exploring how deprecated Python library APIs are (not) handledJiawei Wang, Li Li, Kui Liu, Haipeng CaiFSE 2020 · 52 citations
Related papers
- Restoring the Executability of Jupyter Notebooks by Automatic Upgrade of Deprecated APIsChenguang Zhu, Ripon K. Saha, Mukul R. Prasad, Sarfraz KhurshidASE 2021 · 12 citations
- Automated Modernization of Machine Learning Engineering Notebooks for ReproducibilityBihui Jin, Kaiyuan Wang, Pengyu NieISSTA 2026
- Fixing dependency errors for Python build reproducibilitySuchita Mukherjee, Abigail Almanza, Cindy Rubio-GonzálezISSTA 2021 · 55 citations
- Fine-Grained Lineage for Safer Notebook InteractionsStephen Macke, Aditya G. Parameswaran, Hongpu Gong, Doris Jung Lin Lee et al.VLDB 2021 · 46 citations
- Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data ScientistsIan Drosos, Titus Barik, Philip J. Guo, Robert DeLine et al.CHI 2020 · 110 citations
