DSA DSA In Depth
0%
lesson bit-manipulationtwos-complementxor

Twos-Complement Arithmetic, Bitmasks, Brian Kernighan's Trick & XOR

7 min read Module 1.2: Contiguous Arrays, Bit Manipulation & Pointer Idioms Updated Oct 23, 2026
📄 Cheat Sheet
MIT 6.172 Lecture 3: Bit Hacks, Two's Complement & SWAR Word-Level Parallelism
MIT OpenCourseWare (Charles Leiserson) MIT 6.172 Lecture 3: Bit Hacks, Two's Complement & SWAR Word-Level Parallelism Watch on YouTube ↗

Charles Leiserson derives two's-complement identities, x & (x - 1) lowest-set-bit clearing, branchless min/max, popcount masks, and De Bruijn O(1) bitscan.

0:00 / --:--

Two's Complement Ring Algebra & Lowest-Bit Identities

In a $w$-bit two's-complement register, a bit vector $b_{w-1} b_{w-2} \dots b_1 b_0$ represents the signed integer value:

$$x = -b_{w-1} 2^{w-1} + \sum_{i=0}^{w-2} b_i 2^i$$

Why did two's complement universally triumph over sign-magnitude and ones' complement? Because addition, subtraction, and multiplication use the exact same ALU circuitry for both signed and unsigned integers modulo $2^w$, and zero has a unique representation. Suppose a non-zero integer $x$ has its lowest set bit ($1$) at index $k$, so its binary representation splits into an arbitrary prefix $P$, bit $k$ equal to $1$, and $k$ trailing zeros: $x = [P \;\vert\; 1 \;\vert\; 00\dots0_k]$. Observe what happens under subtraction of $1$ and under negation ($\sim x + 1$):

  • Subtracting 1 ($x - 1$): Borrowing from bit $k$ flips bit $k$ to $0$ and turns all $k$ trailing zeros into $1$s, leaving prefix $P$ untouched: $x - 1 = [P \;\vert\; 0 \;\vert\; 11\dots1_k]$.
  • Brian Kernighan's Clear-Lowest-Set-Bit ($x \mathrel{\&} (x - 1)$): ANDing $x$ with $x - 1$ preserves prefix $P$ and zeroes out bit $k$ and everything below it: $x \mathrel{\&} (x - 1) = [P \;\vert\; 0 \;\vert\; 00\dots0_k]$. This clears the lowest set bit in $O(1)$ time, and equals $0$ for $x > 0$ if and only if $x$ is a power of two!
  • Isolating the Lowest Set Bit ($x \mathrel{\&} -x$): Bitwise NOT gives $\sim x = [\sim P \;\vert\; 0 \;\vert\; 11\dots1_k]$. Adding $1$ ripples a carry through the $k$ trailing ones and sets bit $k$: $-x = \sim x + 1 = [\sim P \;\vert\; 1 \;\vert\; 00\dots0_k]$. Therefore, $x \mathrel{\&} -x = [00\dots0 \;\vert\; 1 \;\vert\; 00\dots0_k] = 2^k$, isolating the lowest set bit (the fundamental primitive of Fenwick Trees).
Algebraic Invariant: $\mathbb{F}_2^w$ XOR Vector Space & Submask Enumeration

Bitwise XOR ($\oplus$) is addition modulo $2$ in the vector space $\mathbb{F}_2^w$: it is commutative ($a \oplus b = b \oplus a$), associative ($a \oplus (b \oplus c) = (a \oplus b) \oplus c$), has identity $0$ ($a \oplus 0 = a$), and every element is its own inverse ($a \oplus a = 0$). Consequently, XORing a stream where every element appears twice except one cancels all pairs to $0$, leaving the unique element.

Even more remarkably, if two unique elements $u \neq v$ appear once while all others appear twice, the total XOR is $S = u \oplus v \neq 0$. Extracting the lowest set bit $lowbit = S \mathrel{\&} -S$ identifies a bit position where $u$ and $v$ provably differ—partitioning the stream into two disjoint subsets (those with $lowbit$ set vs. clear) that isolate $u$ and $v$ in $O(N)$ time and $O(1)$ space!

Essential Bit Manipulation Primitives & x86 BMI1/BMI2 Instructions

Operation Bitwise Expression x86-64 Hardware Instruction Latency / Throughput Algorithmic Application
Clear lowest set bit x & (x - 1) BLSR reg, reg (BMI1) 1 cycle / 0.5 CPI Kernighan popcount, power-of-2 test
Isolate lowest set bit ($2^k$) x & -x (or x & (~x + 1)) BLSI reg, reg (BMI1) 1 cycle / 0.5 CPI Fenwick Tree (BIT), Single Number III
Mask lowest $1$ and trailing $0$s x ^ (x - 1) BLSMSK reg, reg (BMI1) 1 cycle / 0.5 CPI Trailing zero mask generation
Next submask of mask $M$ s = (s - 1) & M DEC + AND 1 cycle $O(3^n)$ Subset-of-Subset DP
Count trailing zeros ($k$) std::countr_zero(x) TZCNT (BMI1) 1–3 cycles Bitboard bitscan, Gosper's hack

Visualizing Two's-Complement Bit Isolation & $O(3^n)$ Submask Jumping

Bit Decomposition of x = 0b10110100 (Lowest Set Bit at k = 2) x = Prefix P: 1 0 1 1 0 Bit k: 1 Trailing: 0 0 x - 1 = Prefix P: 1 0 1 1 0 Bit k: 0 Flipped: 1 1 -x (~x+1) = ~Prefix: 0 1 0 0 1 Bit k: 1 Trailing: 0 0 x & (x-1) = [Prefix P | 0 | 00] (Clears lowest 1) x & -x = [00000 | 1 | 00] (Isolates lowest 1) Submask jump s = (s - 1) & M clears all bits outside M in 1 step, enumerating all pairs (M, s) in Sum C(n,k)2^k = 3^n!

Why Enumerating All Submasks of All Masks Takes $O(3^n)$ Instead of $O(4^n)$

flowchart LR A["Outer Loop:
For mask M = 0 .. 2^n - 1"] --> B["Inner Loop:
For s = M; s > 0; s = (s - 1) & M"] B --> C["If M has k set bits,
inner loop runs 2^k times"] C --> D["Binomial Theorem:
Sum C(n, k) * 2^k = (1 + 2)^n = 3^n"]
Hardware Reality: Signed Overflow UB, INT_MIN Negation, and Shift Width Modulo Traps

In C and C++, signed integer overflow is Undefined Behavior (UB)—even though C++20 standardized two's-complement representation! Specifically:

  • Negating INT32_MIN ($-2^{31} = \text{0x80000000}$) in signed arithmetic overflows because $+2^{31}$ is unrepresentable in 32-bit signed two's complement (its bitwise negation $+1$ wraps back to 0x80000000). Always cast to an unsigned type (uint32_t / uint64_t) before applying 0u - ux or bit manipulation.
  • Shifting a $w$-bit integer by $\ge w$ bits (e.g., 1ULL << 64 or 1 << 32) is Undefined Behavior in C/C++, and on x86-64 the hardware SHL instruction masks the shift count to $63$ (or $31$), turning 1ULL << 64 into 1ULL << 0 == 1 instead of 0!
Two's Complement Bit Tricks (x & -x, x & (x-1)) & 64-Bit SWAR Bitset
O(1) ALU / O(N/64) Bitset

Inspect register-level bit patterns for Two's complement negation -x = ~x + 1, lowest set bit isolation x & -x, Kernighan's popcount loop, and 64-bit word-parallel bitsets.

Step 1 / 7
Custom Input:
Timeline

Invariant & State

In Two's Complement, -x = ~x + 1 inverts all bits left of the lowest set bit and preserves the lowest set bit 2^k, so x & -x = 2^k.
🌳 Live 2D Hierarchical Tree Topology — Two's Complement Bit Tricks (x & -x, x & (x-1)) & 64-Bit SWAR Bitset Visited/Active Pivot/Root Resolved
Trailing x … [0] Bitwise ~x … [1] Carry st -x=~x… [2] LSB = 2³ x &am… [3]
[0]x = 01011000₂ (+88)Trailing 1000₂
[1]~x = 10100111₂Bitwise NOT
[2]-x=~x+1= 10101000₂ (-88)Carry stops at bit 3
[3]x & -x = 00001000₂ (8)LSB = 2³ = 8
Input x
88 (01011000₂)
Negation -x
-88 (10101000₂)
Isolated LSB (x & -x)
8 (00001000₂)
[Step 1] Isolated lowest set bit 2³ = 8 via x & -x (used in Fenwick Trees and bitmask enumeration).

Production Implementation: Single Number III, Gosper's Hack & $O(3^n)$ Submask DP

#include <bit>
#include <cstdint>
#include <utility>
#include <vector>

// Partition an array where two elements appear once and all others appear twice
std::pair<int, int> single_number_two_uniques(const std::vector<int>& nums) noexcept {
  std::uint32_t xor_all = 0;
  for (int x : nums) {
    xor_all ^= static_cast<std::uint32_t>(x);
  }
  // Isolate lowest differing bit using unsigned two's complement negation (safe for INT_MIN!)
  const std::uint32_t diff_bit = xor_all & (0u - xor_all);

  std::uint32_t group_a = 0, group_b = 0;
  for (int x : nums) {
    const auto ux = static_cast<std::uint32_t>(x);
    if ((ux & diff_bit) != 0) {
      group_a ^= ux;
    } else {
      group_b ^= ux;
    }
  }
  return {static_cast<int>(group_a), static_cast<int>(group_b)};
}

// Gosper's Hack: Compute next lexicographical bitmask with the exact same popcount
constexpr std::uint64_t next_combination_mask(std::uint64_t x) noexcept {
  const std::uint64_t c = x & (0ULL - x); // lowest set bit
  const std::uint64_t r = x + c;          // ripple carry through lowest block of 1s
  return (((r ^ x) >> 2) / c) | r;        // shift remaining 1s to the bottom
}

// Enumerate all submasks of all n-bit masks in exactly O(3^n) steps
void enumerate_all_submasks(int n) noexcept {
  const std::uint32_t total = 1u << n;
  for (std::uint32_t mask = 0; mask < total; ++mask) {
    for (std::uint32_t sub = mask; sub > 0; sub = (sub - 1) & mask) {
      // Process non-empty submask sub of mask
    }
    // Process sub == 0 here if empty submask is needed
  }
}
Production Pitfall: C Operator Precedence of Bitwise &, ^, | vs. == and !=

By a notorious historical accident in early B and C (before && and || were separated from & and |), Dennis Ritchie gave relational and equality operators (==, !=, <, >) higher operator precedence than bitwise &, ^, and |. As a result, writing if (mask & bit != 0) is parsed by the compiler as if (mask & (bit != 0))—which evaluates mask & 1 and silently tests the wrong bit! Always parenthesize bitwise tests: if ((mask & bit) != 0), and always write 1ULL << k instead of 1 << k when $k \ge 31$.

Check yourself Question 1 of 4
Concept Check Isolating vs Clearing the Lowest Set Bit recall

In two's-complement arithmetic, let $x$ be a non-zero integer whose lowest set bit ($1$) is at bit position $k$ (having value $2^k$).

⚡ Interactive Challenge · Staff Level

Algorithmic Design & Complexity Check

Analyze the invariant, asymptotic bound, and hardware behavior before selecting.

Why are all three expressions buggy or Undefined Behavior in C/C++?

Key Takeaways

In two's-complement arithmetic (-x == ~x + 1), x & (x - 1) clears the lowest set bit (BLSR) and x & -x isolates the lowest set bit (BLSI) in a single ALU cycle.
To avoid signed integer overflow UB on INT_MIN (-2^31), perform bitwise negation and bit hacks on unsigned types: ux & (0u - ux).
Iterating s = (s - 1) & mask visits only valid submasks of mask; summed over all 2^n masks, this takes sum C(n,k) 2^k = 3^n total operations instead of 4^n.
Always wrap bitwise masks in parentheses before equality comparisons ((x & mask) != 0) and use 1ULL << k to prevent 32-bit shift Undefined Behavior.
Was this lesson helpful? Spot an issue or have a suggestion? Let us know on GitHub.
Report an issue