Vessel: A Taxonomy of Reproducibility Issues for Container Images

Build reproducibility of container images is essential to ensure that deployed systems will work as expected and have not been tampered with. However, bit-by-bit reproducibility of container images is almost never achievable due to external factors, and it is also very slow and labor intensive to determine the causes and severity of reproducibility failures. In this paper, we present a taxonomy of reproducibility issues for container images, as well as a tool, Vessel Diff, to help automatically categorize the type and severity of reproducibility failures in container images. We analyzed a set of open source repositories where container images are built to find common patterns and configure our tool to properly categorize failures. Our analysis shows that approximately 87% of their reproducibility failures were automatically classified by the tool according to our taxonomy. However, the vast majority of these failures were caused by trivial issues and not non-trivial issues, which could cause noticeable changes in execution of container applications and are more difficult to detect. These results highlight the need for additional research and tooling to detect, classify, and fix reproducibility issues, especially those that can lead to major failures.