Adversarial examples for fast hash functions

Adversarial examples for fast hash functions represent a concern in the realm of cryptography and data integrity. Hash functions encode data of arbitrary length into fixed-size values, aiming to ensure distinct inputs yield distinct outputs, albeit with a very small probability. This property enables the construction of efficient hash tables, where no two data points collide in the same bucket.

Examples of fast hash functions include xxHash, komihash, HighwayHash, SpookyHash, aHash, and t1ha2. These hashes often prioritize speed over robustness, especially for adversarial inputs, where they might collide more frequently than expected. Despite this, most hash functions strive to be resilient, preventing quadratic slowdowns in algorithms and denial-of-service attacks.

A hash is deemed b-bit universal if inputs of length L collide with a probability no greater than L * 2^(-b) for all L. This universal property provides a provable guarantee. Claude Fable was used to analyze popular hashes, revealing that many perform poorly, at least 20 bits below expectations. Some hashes have published proofs, which were analyzed and verified in Lean.

The findings were shared with maintainers upstream before publication. However, only attacks involving large sets of inputs that collide with high probability are considered worth fixing. The study discovered flooding-grade key-free multicollisions for various hashes, highlighting the need for research into faster, provable hash functions.

Several hashes show similar bad patterns across their families, suggesting a knee-jerk reaction may not involve merely switching to "cryptographically secure" hashes like SHA or using AES native instructions.