Trust, But Verify: A Longitudinal Analysis Of Android OEM Compliance and Customization

Nowadays, more than two billions of mobile devices run Android OS. At the core of this success are the open source nature of the Android Open Source Project and vendors' ability to customize the code base and ship it on their own devices. While the possibility of customizations is beneficial to vendors, they can potentially lead to compatibility and security problems. To prevent these problems, Google developed a set of requirements that must be satisfied for a vendor to brand its devices as "Android," and recently introduced Project Treble as an effort to partition vendor customizations. These requirements are encoded as part of a textual document (called Compatibility Definition Document, or CDD) and various automated tests. This paper performs the first longitudinal study on Android OEM customizations. We first built a dataset of 2,907 ROMs, spanning across 42 different vendors, and covering Android versions from 1.6 to 9.0 (years 2009-2020). We then developed an analysis framework and pipeline to extract each ROM's customization layers and evaluate it across several metrics. For example, we analyze ROMs to determine whether they are compliant with respect to the various requirements and whether their customizations negatively affect the security posture of the overall device. In the process, we focus on various aspects, ranging from security hardening of binaries, SELinux policies, Android init scripts, and kernel security hardening techniques. Our results are worrisome. We found 579 over 2,907 ( ∼ 20%) of the ROMs have at least one violation for the CDD related to their Android version -incredibly, 11 of them are branded by Google itself. Some of our findings suggest that vendors often go out of their way to bypass or "comment out" safety nets added by the Android security team. In other cases, we found ROMs that modify init scripts to launch at boot outdated versions (with known CVEs and public POCs) of programs as root and reachable from a remote attacker (e.g., tcpdump). This paper shows that Google's efforts are not enough, and we offer several recommendations on how to improve the compliance check pipelines.