Speeding up detection of SHA-1 collision attacks using unavoidable attack conditions

Counter-cryptanalysis, the concept of using cryptanalytic techniques to detect cryptanalytic attacks, was first introduced by Stevens at CRYPTO 2013 with a hash collision detection algorithm. That is, an algorithm that detects whether a given single message is part of a colliding message pair constructed using a cryptanalytic collision attack on MD5 or SHA-1. The concept's utility was proven when it was used to expose the then-unknown cryptanalytic collision attack exploited by the Flame espionage supermalware.

So far there is a significant cost: to detect collision attacks against SHA-1 (respectively MD5) costs the equivalent of hashing the message 15 (respectively 224) times. In this paper we present a significant performance improvement for collision detection based on the new concept of unavoidable conditions. Unavoidable conditions are conditions that are necessary for all feasible attacks in a certain attack class. As such they can be used to quickly dismiss particular attack classes that may have been used in the construction of the message. To determine an unavoidable condition one must rule out any feasible variant attack where this condition might not be necessary, otherwise adversaries aware of counter-cryptanalysis could easily bypass this improved collision detection with a carefully chosen variant attack. We provide a formal model for unavoidable conditions for collision attacks on MD5-like compression functions.

Furthermore, based on a conjecture solidly supported by the current state of the art, we show how we can determine such unavoidable conditions for SHA-1. We have implemented the improved SHA-1 collision detection using such unavoidable conditions and which is about 16 times faster than without our unavoidable condition improvements. We have measured that overall our implemented SHA-1 with collision detection is only a factor 1.96 slower, on average, than SHA-1. Our work is very timely given the recently announced first SHA-1 collision proving that SHA-1 is now practically broken.