Interpreting Emergent Features in Deep Learning-based Side-channel Analysis

Side-channel analysis (SCA) poses a real-world threat by exploiting unintentional physical signals to extract secret information from secure devices. Evaluation labs also use the same techniques to certify device security. In recent years, deep learning has emerged as a prominent method for SCA, achieving state-ofthe-art attack performance at the cost of interpretability. Understanding how neural networks extract secrets is crucial for security evaluators aiming to defend against such attacks, as only by understanding the attack can one propose better countermeasures. In this work, we apply mechanistic interpretability to neural networks trained for SCA, revealing how models exploit what leakage in side-channel traces. We focus on sudden jumps in performance to reverse engineer learned representations, ultimately recovering secret masks and moving the evaluation process from blackbox to white-box. Our results show that mechanistic interpretability can scale to realistic SCA settings, even when relevant inputs are sparse, model accuracies are low, and side-channel protections prevent standard input interventions. extract input features, i.e., individual shares s i related to device internal randomness, from model activations, providing a path to move from black-box to white-box evaluations. The overall analysis process is illustrated in Figure 1.

To summarize, our main contributions are:

• We explore the feasibility of applying MI in a challenging real-world setting where input interventions to features are not possible due to SCA countermeasures.

• By investigating the changes in model outputs during sudden jumps in model performance, we find how networks combine leakage in DLSCA.

• We directly retrieve the internal secret share values by applying activation patches 3 to intermediate layer activations across several targets.

• We provide more detailed insights into the specific physical leakage that neural networks exploit for widely used (DL)SCA benchmark datasets. Notably, we do this without assuming a priori mask knowledge [54,35] or requiring custom architectures [52,53].

• We find identical structures emerging during sudden generalizations for models trained on side-channel traces captured on different implementations and in different SCA domains (electromagnetic vs. power), providing further evidence for the weak universality hypothesis [7].