Anatomy of a HashDoS Attack: Equivalent Substrings & Meet-in-the-Middle
How does an attacker generate $2^{16} = 65{,}536$ colliding strings in milliseconds?
- Equivalent Two-Char Blocks (Java
hashCode/ DJB2): In Java's polynomial hash $h(s) = \sum s[i] \cdot 31^{L-1-i} \bmod 2^{32}$, any two-character block $c_1 c_2$ contributes $31 c_1 + c_2$. Since'A' * 31 + 'a' = 65 * 31 + 97 = 2112and'B' * 31 + 'B' = 66 * 31 + 66 = 2112, the substrings"Aa"and"BB"have the exact same hash value! Concatenating $k = 16$ choices of{"Aa", "BB"}produces $2^{16} = 65{,}536$ distinct 32-byte strings with identical hash codes in $O(1)$ effort—and because every character comparison checks the full string prefix before failing, total comparison work reaches $\Theta(N^2 \cdot L)$! - Seed-Independent Differential Multicollisions (MurmurHash / CityHash): Non-cryptographic hashes like MurmurHash3 and CityHash64 were designed for statistical uniformity (passing SMHasher avalanche tests), not adversarial preimage resistance. Using internal state differentials, attackers construct pairs of blocks $(\Delta_1, \Delta_2)$ that cancel each other inside the mixing rounds regardless of the 32-bit/64-bit startup seed.
| Defense Mechanism | Worst-Case $N$ Insertions | Short-Key Latency (16B) | Adversarial Security Guarantee | Production Adoption |
|---|---|---|---|---|
| Unkeyed Polynomial (DJB2 / FNV-1a) | Θ(N²) | ~3 ns | None (Trivial 2-char collisions) | Legacy C / Pre-2012 runtimes |
| Seeded Non-Crypto (Murmur3 / WyHash) | Θ(N²) (Differential) | ~4 ns | Breaks under differential/seed-recovery | Internal trusted keys / C++ Abseil |
| Keyed ARX PRF (SipHash-1-3 / 2-4) | O(N) Expected w.h.p. | ~9–14 ns | 128-bit cryptographic PRF security | Rust std::collections::HashMap, Python, Linux, Redis |
| Treeified Buckets (Red-Black Chain) | O(N log N) Deterministic | ~5 ns (Normal) | Deterministic $O(\log N)$ per bucket | Java 8+ java.util.HashMap (TREEIFY_THRESHOLD=8) |
Formal Specification: SipHash ARX State Machine & PRF Indistinguishability
SipHash-$c$-$d$ Architecture (Aumasson & Bernstein, 2012): Initializes four 64-bit internal state lanes $(v_0, v_1, v_2, v_3)$ by XORing a 128-bit secret key $(k_0, k_1)$ obtained from OS entropy (getrandom / /dev/urandom) with four ASCII constant words ("somepseudorandomlygeneratedbytes"). For each 8-byte message block $m_i$: set $v_3 \gets v_3 \oplus m_i$, execute $c$ compression SipRounds, and set $v_0 \gets v_0 \oplus m_i$. After the final length-padded block, set $v_2 \gets v_2 \oplus \text{0xff}$, execute $d$ finalization SipRounds, and output $v_0 \oplus v_1 \oplus v_2 \oplus v_3$.
PRF Bound: Without knowing $(k_0, k_1)$, no probabilistic polynomial-time adversary (even with adaptive timing oracle access) can distinguish SipHash-2-4 outputs from a truly random oracle, bounding $N$-key insertion cost to $O(N)$ with probability $1 - 2^{-64}$.
Visualizing the SipRound Add-Rotate-XOR (ARX) Butterfly
Witness how an unkeyed polynomial hash (h = 31·h + c) degenerates to O(N²) under an adversarial HashDoS collision payload, how keyed SipHash-2-4 ARX rounds & Carter-Wegman universal families enforce Pr[collision] ≤ 1/M, and how the Birthday Paradox curve 1 − e^(−k(k−1)/2M) crosses 50% at k ≈ 1.177√M.
Invariant & State
Hardware Reality: ARX Instructions vs SHA-256 on Short Strings
Why didn't language runtimes simply use HMAC-SHA256 for hash tables? SHA-256 processes 64-byte (512-bit) blocks with 64 rounds per block, meaning a typical 12-byte map key ("user_id_4921") wastes $80\%$ of its block capacity on padding and takes ~150 CPU cycles. SipHash-2-4 operates on 8-byte words using pure 64-bit Add-Rotate-XOR (ARX) instructions (ADD, ROL, XOR) that execute in 4 cycles per SipRound on superscalar 64-bit pipelines, completing a 16-byte string hash in ~30 cycles with zero S-box cache-timing side channels.
Production C++20 Implementation: RFC-Compliant SipHash-2-4
// C++20: Reference SipHash-2-4 (Aumasson & Bernstein) for HashDoS-Resistant Tables
#include <cstdint>
#include <cstddef>
#include <bit>
#include <cstring>
class SipHash24 {
uint64_t k0_, k1_;
static constexpr void sipRound(uint64_t& v0, uint64_t& v1, uint64_t& v2, uint64_t& v3) noexcept {
v0 += v1; v1 = std::rotl(v1, 13) ^ v0; v0 = std::rotl(v0, 32);
v2 += v3; v3 = std::rotl(v3, 16) ^ v2;
v0 += v3; v3 = std::rotl(v3, 21) ^ v0;
v2 += v1; v1 = std::rotl(v1, 17) ^ v2; v2 = std::rotl(v2, 32);
}
public:
constexpr SipHash24(uint64_t k0, uint64_t k1) noexcept : k0_(k0), k1_(k1) {}
uint64_t operator()(const uint8_t* data, size_t len) const noexcept {
uint64_t v0 = k0_ ^ 0x736f6d6570736575ULL;
uint64_t v1 = k1_ ^ 0x646f72616e646f6dULL;
uint64_t v2 = k0_ ^ 0x6c7967656e657261ULL;
uint64_t v3 = k1_ ^ 0x7465646279746573ULL;
const size_t blocks = len / 8;
for (size_t i = 0; i < blocks; ++i) {
uint64_t m = 0;
std::memcpy(&m, data + i * 8, sizeof(uint64_t));
v3 ^= m;
sipRound(v0, v1, v2, v3);
sipRound(v0, v1, v2, v3); // c = 2 compression rounds
v0 ^= m;
}
// Pack remaining 0..7 bytes in little-endian + (len mod 256) in top byte [56..63]
uint64_t last = static_cast<uint64_t>(len & 0xff) << 56;
const uint8_t* tail = data + blocks * 8;
for (size_t r = 0; r < (len & 7); ++r) {
last |= static_cast<uint64_t>(tail[r]) << (r * 8);
}
v3 ^= last;
sipRound(v0, v1, v2, v3);
sipRound(v0, v1, v2, v3);
v0 ^= last;
v2 ^= 0xffULL;
for (int d = 0; d < 4; ++d) sipRound(v0, v1, v2, v3); // d = 4 finalization rounds
return v0 ^ v1 ^ v2 ^ v3;
}
};
Failure Modes & Adversarial Pitfalls
- Exposing Hash Iteration Order to Untrusted Clients: Even if a server uses SipHash with a random 128-bit secret key, if an API serializes hash map keys in bucket iteration order and returns them to the user, an attacker can mount a timing/ordering side-channel attack to learn which keys share buckets or recover the secret key. Keep seeds private or randomize per-table state.
- Using Unkeyed
fxhash/fnvon Network-Facing Maps: Rust'srustc-hash(FxHashMap) and C++'sstd::unordered_mapuse fast unkeyed hashes. WhileFxHashMapis $2\times$ faster on integer keys inside a compiler AST, exposing it to HTTP headers, JSON keys, or query parameters invites immediate $\Theta(N^2)$ HashDoS.