DSA DSA In Depth
0%
lesson sos-dpzeta-transformfwht

Sum Over Subsets (SOS) DP & Fast Walsh-Hadamard Transform (FWHT) for Bitwise Convolutions

7 min read Module 6.3: Advanced DP State Spaces & Algebraic Optimizations Updated Oct 23, 2026
📄 Cheat Sheet
Sum Over Subsets (SOS DP) & Fast Zeta Transform Fundamentals
Vivek Gupta Sum Over Subsets (SOS DP) & Fast Zeta Transform Fundamentals Watch on YouTube ↗

Explains how processing the boolean hypercube dimension-by-dimension reduces O(3^N) submask aggregation to O(N * 2^N) in 6 lines of code.

0:00 / --:--

1. From $O(3^N)$ Submasks to $O(N \cdot 2^N)$ Hypercube Prefix Sums (SOS DP)

Given an array $A[0..2^N - 1]$, we want to compute the Submask Zeta Transform (Sum Over Subsets) for every mask $m$:

$$F[m] = \sum_{s \subseteq m} A[s]$$

Think of an $N$-bit integer $m$ not as a scalar, but as an $N$-tuple of coordinates $(b_{N-1}, \dots, b_1, b_0) \in \{0, 1\}^N$. Then $s \subseteq m$ simply means $s_i \le m_i$ in every coordinate $i \in \{0..N-1\}$—so $F[m]$ is literally an $N$-dimensional prefix sum over a $\underbrace{2 \times 2 \times \dots \times 2}_{N}$ grid!

Core Invariant — SOS DP Dimension-by-Dimension Invariant

Define $dp[m][i]$ as the sum of $A[s]$ over all submasks $s \subseteq m$ that are allowed to differ from $m$ only in the lowest $i + 1$ bits (bits $0..i$), while matching $m$ on bits $i+1..N-1$:
1. If bit $i$ of $m$ is $0$, then bit $i$ of $s$ must also be $0$: $dp[m][i] = dp[m][i-1]$.
2. If bit $i$ of $m$ is $1$, then bit $i$ of $s$ can be either $1$ (matching $m$) or $0$ (matching $m \oplus 2^i$):
$$dp[m][i] = dp[m][i-1] + dp[m \oplus 2^i][i-1]$$ Because $dp[m][i]$ only reads $dp[\dots][i-1]$, the second dimension $i$ compresses 100% in-place! Furthermore, reversing the $+\to -$ sign computes the Fast Möbius Inversion (recovering $A$ from $F$ by Inclusion-Exclusion in $O(N 2^N)$)!

2. Bitwise Convolutions (OR, AND, XOR) via Fast Walsh-Hadamard Transform (FWHT)

Suppose we want to compute the bitwise convolution $C[k] = \sum_{i \circledast j = k} A[i] \cdot B[j]$ for $\circledast \in \{|, \&, \oplus\}$ in $O(N \cdot 2^N)$ instead of $O(4^N)$:

  • Bitwise OR Convolution ($i \mathbin{|} j = k$): Notice $(i \mathbin{|} j) \subseteq k \iff i \subseteq k \text{ and } j \subseteq k$. Therefore, the Submask Zeta Transform turns OR-convolution into pointwise multiplication: $\hat{C}[m] = \hat{A}[m] \cdot \hat{B}[m]$, followed by Fast Möbius Inversion (subtracting dp[m ^ (1 << i)])!
  • Bitwise AND Convolution ($i \mathbin{\&} j = k$): $(i \mathbin{\&} j) \supseteq k \iff i \supseteq k \text{ and } j \supseteq k$. Use the Supermask Zeta Transform (when bit $i$ of $m$ is $0$, add dp[m | (1 << i)]), multiply pointwise, and invert by subtracting!
  • Bitwise XOR Convolution ($i \oplus j = k$): Apply the Fast Walsh-Hadamard Transform (FWHT)—an $N$-dimensional size-2 DFT with butterfly $(u, v) \leftarrow (u + v,\; u - v)$—multiply pointwise, and apply the inverse butterfly $(u, v) \leftarrow ((u + v)/2,\; (u - v)/2)$!
flowchart LR AB["Arrays A, B
Size 2^N"] -->|"Forward Transform O(N 2^N)
OR: Submask SOS
AND: Supermask SOS
XOR: Hadamard (u+v, u-v)"| Hat["Transformed A_hat, B_hat"] Hat -->|"Pointwise Product O(2^N)
C_hat[m] = A_hat[m] * B_hat[m]"| Chat["C_hat"] Chat -->|"Inverse Transform O(N 2^N)
OR/AND: Mobius (-)
XOR: Hadamard / 2^N"| C["Convolution C = A * B"]

Hardware Reality — Why Outer Loop bit + Butterfly Stride Beats Outer Loop mask

In SOS DP, you must put for (int i = 0; i < N; ++i) as the outer loop and for (int mask = 0; mask < (1 << N); ++mask) as the inner loop! First, putting mask on the outside is mathematically incorrect for in-place updates when dp[m] aggregates over $\max$ or $+$. Second, with i on the outside, the butterfly step processes contiguous power-of-2 blocks of size $2^i$, enabling unconditional branchless inner loops (for (int b = 0; b < (1 << N); b += 1 << (i + 1)) for (int j = 0; j < (1 << i); ++j)) that the compiler auto-vectorizes with AVX2!

3. Complexity Matrix

SOS DP (Sum Over Subsets) Hypercube, Digit DP & Broken Profile Tiling
O(N·2^N) SOS / Broken Profile

Trace dimension-by-dimension SOS DP bit-flips F[mask] += F[mask ^ (1<<i)] without overcounting, Broken Profile staircase wavefront domino tiling, and Digit DP tight-prefix automata.

Step 1 / 7
Custom Input:
Timeline

Invariant & State

SOS DP Invariant: After outer loop dimension i, F[mask] aggregates all submasks that may differ from mask only in bits 0..i (zero duplicate paths!).
🔗 Live Pointer & State Transition Graph — SOS DP (Sum Over Subsets) Hypercube, Digit DP & Broken Profile Tiling Active Pointer Jump/Transition Matched/Optimal
00₂ F[00]… [0] Dim i=0 F[01]… [1] 10₂ F[10]… [2] Dim i=1 F[11]… [3]
[0]F[00]=a₀00₂
[1]F[01]=a₀+a₁ (bit 0)Dim i=0 merged
[2]F[10]=a₂10₂
[3]F[11]=a₂+a₃ (bit 0) → +F[01] (bit 1)Dim i=1 → ∑ all 4
3^N Submask Loop
O(3^N) = 59,049 (N=10)
SOS DP N·2^N
O(N·2^N) = 10,240 (N=10)
Update Rule
if (mask & (1<<i)) F[mask] += F[mask^(1<<i)]
[Step 1] Swept dimensions i = 0..N-1; each hypercube edge is traversed once in O(N · 2^N).
Bitwise Transform / Convolution Naive Complexity Submask Complexity SOS / FWHT Complexity Time at $N = 20$ ($2^{20} \approx 10^6$)
Sum Over Subsets ($\sum_{s \subseteq m} A[s]$) O(4^N) O(3^N) ($3.5 \times 10^9$) O(N * 2^N) ($2 \times 10^7$) ~15 ms (vs. 4.5 sec for $3^N$)
Sum Over Supersets ($\sum_{s \supseteq m} A[s]$) O(4^N) O(3^N) O(N * 2^N) ~15 ms
Bitwise OR / AND Convolution O(4^N) O(3^N) O(N * 2^N) ~45 ms (3 passes)
Bitwise XOR Convolution (FWHT) O(4^N) O(4^N) O(N * 2^N) ~45 ms (3 passes)

4. Reference C++20 Implementation

#include <cstdint>
#include <vector>

// 1. Branchless In-Place Sum Over Subsets (Fast Zeta) & Inverse (Fast Mobius) in O(N * 2^N)
// invert = false: F[m] = sum_{s subseteq m} A[s]
// invert = true:  Recovers A[m] from F[m] via Inclusion-Exclusion
void sos_submask_transform(std::vector<long long>& f, int n, bool invert = false) {
    const int total = 1 << n;
    for (int i = 0; i < n; ++i) {
        const int half = 1 << i;
        const int step = half << 1;
        // Branchless butterfly: skips all masks where bit i is 0 without an if!
        for (int base = 0; base < total; base += step) {
            for (int j = 0; j < half; ++j) {
                if (!invert) {
                    f[base + half + j] += f[base + j];
                } else {
                    f[base + half + j] -= f[base + j];
                }
            }
        }
    }
}

// 2. Bitwise OR Convolution: C[k] = sum_{i | j == k} A[i] * B[j] in O(N * 2^N)
std::vector<long long> bitwise_or_convolution(std::vector<long long> a, std::vector<long long> b, int n) {
    sos_submask_transform(a, n, false);
    sos_submask_transform(b, n, false);
    for (int m = 0; m < (1 << n); ++m) a[m] *= b[m];
    sos_submask_transform(a, n, true); // Fast Mobius Inversion recovers exact OR-convolution!
    return a;
}

// 3. Fast Walsh-Hadamard Transform (FWHT) for Bitwise XOR Convolution: C[k] = sum_{i ^ j == k} A[i] * B[j]
void fwht_xor(std::vector<long long>& a, int n, bool invert = false) {
    const int total = 1 << n;
    for (int i = 0; i < n; ++i) {
        const int half = 1 << i;
        const int step = half << 1;
        for (int base = 0; base < total; base += step) {
            for (int j = 0; j < half; ++j) {
                long long u = a[base + j];
                long long v = a[base + half + j];
                a[base + j] = u + v;
                a[base + half + j] = u - v;
            }
        }
    }
    if (invert) {
        for (long long& x : a) x >>= n; // Divide by 2^n
    }
}

Production Pitfall — Swapping the bit and mask Loop Order in SOS DP

Never write for (int mask = 0; mask < (1 << n); ++mask) for (int i = 0; i < n; ++i)! When mask is the outer loop, f[mask ^ (1 << i)] has already been completely summed across all $n$ dimensions instead of just dimensions $0..i-1$, causing submasks that differ in multiple bits to be double-, triple-, or $k!$-counted! Dimension i must be the outer loop.

Check yourself Question 1 of 4
Concept Check Core Invariant Check recall

In Sum Over Subsets (SOS) DP (`for i = 0..N-1: for mask = 0..2^N-1: if (mask & (1<<i)) dp[mask] += dp[mask ^ (1<<i)]`), why can we overwrite `dp[mask]` in-place without reading stale or doubly-counted values?

⚡ Interactive Challenge · Staff Level

Algorithmic Design & Complexity Check

Analyze the invariant, asymptotic bound, and hardware behavior before selecting.

You are given an array of $M$ integers $a_1, \dots, a_M \in [0, 2^{20}-1]$ and want to count the number of ordered pairs $(x, y)$ whose bitwise AND is zero ($a_x \mathbin{\&} a_y = 0$). How does SOS DP solve this in $O(M + N 2^N)$ time ($N = 20$)?

Key Takeaways

Sum Over Subsets (SOS) DP treats an N-bit mask as an N-dimensional {0, 1}^N hypercube and computes 1D prefix sums along each bit dimension i = 0..N-1.
SOS DP reduces submask and supermask aggregation from O(3^N) to O(N * 2^N) time and runs 100% in-place in a single array of size 2^N.
Reversing addition to subtraction in SOS DP computes the Fast Mobius Inversion (Inclusion-Exclusion over the boolean lattice) in O(N * 2^N).
Bitwise OR, AND, and XOR convolutions reduce to O(2^N) pointwise multiplication via Submask Zeta, Supermask Zeta, and Fast Walsh-Hadamard Transform (FWHT).
Was this lesson helpful? Spot an issue or have a suggestion? Let us know on GitHub.
Report an issue