Cryptanalytic Extraction of Deep Neural Networks with Non-linear Activations

Deep neural networks (DNNs) are today’s central machine learning engines, yet their parameters represent valuable intellectual prop- erty exposed to extraction through black-box queries. While existing cryptanalytic attacks have primarily targeted ReLU-based architectures, this work extends model-stealing techniques to a broad class of non-linear activation functions, including GELU, SiLU, SELU, Sigmoid, and oth- ers. We present the first universal black-box attack capable of recovering both weights and biases from networks whose activations converge to lin- ear behavior outside narrow non-linear regions. Our method generalizes prior geometric approaches by leveraging higher-order derivatives and ad- jacent linear zone analysis, bypassing the need for non-differentiability. We show that, for several activations, neuron signatures can be recov- ered more easily than in the ReLU case, and we further demonstrate that activation functions themselves can be identified when not publicly known. Our results broaden the scope of cryptanalytic model extraction, revealing that the secrecy of activation functions or smoothness of nonlin- earities does not provide effective protection against black-box recovery attacks.