DSA DSA In Depth
0%
lesson matrix-exponentiationtropical-semiringlinear-recurrence

Matrix Exponentiation & Min-Plus (Tropical) Semiring Matrix Multiplication for Fixed-Length Paths

7 min read Module 6.3: Advanced DP State Spaces & Algebraic Optimizations Updated Oct 23, 2026
📄 Cheat Sheet
Matrix Exponentiation in Dynamic Programming
Errichto Algorithms Matrix Exponentiation in Dynamic Programming Watch on YouTube ↗

Demonstrates how any time-invariant 1D linear recurrence or fixed-step graph DP can be compiled into a transition matrix and fast-forwarded to step 10^18 in O(K^3 log N) time.

0:00 / --:--

1. Formal Invariant — Semiring Associativity & Transition Operators

Invariant: Semiring Matrix Exponentiation

Let $(S, \oplus, \otimes)$ be a semiring where $\oplus$ is commutative and associative with identity $\mathbb{0}$, $\otimes$ is associative with identity $\mathbb{1}$, and $\otimes$ distributes over $\oplus$: $a \otimes (b \oplus c) = (a \otimes b) \oplus (a \otimes c)$. Define matrix multiplication $C = A \star B$ for $K \times K$ matrices by:

$$C_{i,j} = \bigoplus_{m=0}^{K-1} \Big( A_{i,m} \otimes B_{m,j} \Big)$$

  • Counting / Linear Recurrence Ring $(+, \times, 0, 1)$: $(A^N)_{i,j}$ counts the weighted sum of walks of length $N$ from $i$ to $j$.
  • Tropical Min-Plus Semiring $(\min, +, +\infty, 0)$: $(A^{\otimes N})_{i,j} = \min_{m} (A^{\otimes (N-1)}_{i,m} + A_{m,j})$ equals the minimum weight of a walk with exactly $N$ edges from $i$ to $j$.

Because $\star$ is associative, $A^{\otimes N}$ can be computed in $O(K^3 \log N)$ operations via repeated squaring.

2. Binary Exponentiation Pipeline

flowchart LR Init["Result = Identity Matrix I
Base = Transition Matrix T
Exponent = N"] --> CheckBit{"N > 0 ?"} CheckBit -->|"Yes, N & 1 == 1"| MultRes["Result = Result * Base
(Standard or Min-Plus)"] CheckBit -->|"Yes, N & 1 == 0"| SquareBase MultRes --> SquareBase["Base = Base * Base
N >>= 1"] SquareBase --> CheckBit CheckBit -->|"No (N == 0)"| Output["Return Result * dp[0]"]

3. State-Space Compilation Tricks

  • Order-$K$ Linear Recurrences: If $f(n) = c_1 f(n-1) + c_2 f(n-2) + \dots + c_K f(n-K)$, the state vector is $[f(n-1), f(n-2), \dots, f(n-K)]^T$ and the companion matrix has $[c_1, c_2, \dots, c_K]$ in row $0$ and $1$s on the subdiagonal.
  • Affine Constants & Polynomial Terms: If $f(n) = c_1 f(n-1) + d \cdot n + e$, augment the state vector with $[n, 1]^T$ using the binomial expansion $(n+1) = 1 \cdot n + 1 \cdot 1$.
  • At Most $N$ Edges vs Exactly $N$ Edges: In the tropical $(\min, +)$ semiring, adding a zero-weight self-loop $A_{i,i} = 0$ at every vertex allows a walk to 'wait' at vertex $i$, converting 'shortest walk of exactly $N$ edges' into 'shortest walk of at most $N$ edges'.
  • Multiple Queries on the Same Graph: Precompute powers of two $T^{2^0}, T^{2^1}, \dots, T^{2^{60}}$ in $O(K^3 \log N)$ time once. Then each query for walk length $N_q$ multiplies a $1 \times K$ vector by at most $\log_2 N_q$ matrices in $O(K^2 \log N_q)$ time instead of $O(K^3 \log N_q)$!

4. Hardware & Cache Optimization

Loop Reordering (i-k-j) & Modulo Reduction

In a $K \times K$ matrix product, the textbook i-j-k loop reads B[k][j] down column j with stride $K$, causing cache misses and blocking SIMD vectorization. Permuting the loops to i-k-j hoisted A[i][k] into a register and streams C[i][j] and B[k][j] sequentially across row j, yielding a 4x–8x speedup! For modular arithmetic with $M = 10^9 + 7$, accumulating into uint64_t and reducing modulo $M^2 \approx 10^{18}$ every 16 inner iterations eliminates $93\%$ of slow div instructions.

5. Complexity Matrix

Convex Hull Trick (CHT) & Li Chao Tree Lower Envelope
O(N) CHT / O(log X) Li Chao

Visualize linear DP transitions dp[i] = min_j(m_j·x_i + b_j) as a lower envelope of lines, popping dominated middle lines when intersect(L1, L3) ≤ intersect(L1, L2).

Step 1 / 8
Custom Input:
Timeline

Invariant & State

Hull stack [L1(y=-x+14), L2(y=-2x+19)] with decreasing slopes m1=-1 > m2=-2. Crossover at x = 5.0.
x(L1,L2) = 5.0
Active Hull
[L1, L2]
Intersection x(L1, L2)
5.0
Optimal for x < 5
L1
[Step 1] Pushed L1 and L2 onto lower hull; L2 becomes optimal for queries x > 5.0.
Algorithm Preprocessing Single Target $N$ $Q$ Queries after Precompute
Iterative DP ($N$ steps, $K$ states) O(1) O(N * K^2) O(Q * N * K^2)
Semiring Matrix Exponentiation O(1) O(K^3 log N) O(Q * K^3 log N)
Precomputed Powers $T^{2^b}$ + Vector-Matrix Product O(K^3 log N) O(K^2 log N) O(Q * K^2 log N)

6. Production-Grade C++20 Implementation (Both Standard & Min-Plus Semirings)

#include <vector>
#include <cstdint>
#include <limits>
#include <algorithm>

// Tropical Min-Plus Matrix Multiplication for Shortest Walk of Exactly N Edges
struct MinPlusMatrix {
    static constexpr int64_t INF = std::numeric_limits<int64_t>::max() / 4;
    int n;
    std::vector<int64_t> mat; // Flat 1D row-major array for cache locality

    explicit MinPlusMatrix(int dim, int64_t fill_val = INF)
        : n(dim), mat(dim * dim, fill_val) {}

    static MinPlusMatrix identity(int dim) {
        MinPlusMatrix I(dim, INF);
        for (int i = 0; i < dim; ++i) I(i, i) = 0; // Tropical multiplicative identity = 0
        return I;
    }

    int64_t& operator()(int r, int c) { return mat[r * n + c]; }
    int64_t operator()(int r, int c) const { return mat[r * n + c]; }

    // Cache-friendly i-k-j loop order
    MinPlusMatrix operator*(const MinPlusMatrix& other) const {
        MinPlusMatrix res(n, INF);
        for (int i = 0; i < n; ++i) {
            for (int k = 0; k < n; ++k) {
                int64_t a_ik = (*this)(i, k);
                if (a_ik >= INF) continue;
                for (int j = 0; j < n; ++j) {
                    int64_t cand = a_ik + other(k, j);
                    if (cand < res(i, j)) {
                        res(i, j) = cand;
                    }
                }
            }
        }
        return res;
    }
};

MinPlusMatrix power(MinPlusMatrix base, int64_t exp) {
    MinPlusMatrix res = MinPlusMatrix::identity(base.n);
    while (exp > 0) {
        if (exp & 1) res = res * base;
        base = base * base;
        exp >>= 1;
    }
    return res;
}

Production Pitfall — Mixing Up Identity Matrices Between Standard and Min-Plus Semirings

In standard $(+, \times)$ matrix exponentiation, the identity matrix has $1$ on the diagonal and $0$ off-diagonal. In the tropical $(\min, +)$ semiring, the 'addition' operation is $\min$ (whose identity is $+\infty$) and the 'multiplication' operation is $+$ (whose identity is $0$). Therefore, the tropical identity matrix has $0$ on the diagonal and $+\infty$ off-diagonal! Initializing a tropical matrix with $0$ off-diagonal silently creates free $0$-cost teleportation edges between every pair of vertices.

Check yourself Question 1 of 4
Concept Check Core Invariant Check recall

In the Tropical Min-Plus Semiring $(\min, +)$ used for finding minimum-weight walks of exactly $N$ edges via matrix exponentiation, how must the $K \times K$ identity matrix $I$ be initialized?

⚡ Interactive Challenge · Staff Level

Algorithmic Design & Complexity Check

Analyze the invariant, asymptotic bound, and hardware behavior before selecting.

Why does permuting the triple nested loop of matrix multiplication from textbook `i-j-k` to `i-k-j` yield a 4x–8x wall-clock speedup in C++?

Key Takeaways

Any time-invariant linear DP over K states can be represented as matrix multiplication over a semiring (S, oplus, otimes) and accelerated to O(K^3 log N) via binary exponentiation.
In the tropical (min, +) semiring, (A^N)_{u,v} computes the minimum weight walk of exactly N edges from u to v; its identity matrix has 0 on the diagonal and +INF off-diagonal.
Precomputing powers A^{2^b} allows answering Q walk-length queries in O(K^2 log N) per query by multiplying a 1 x K state vector instead of two K x K matrices.
Using a flat 1D row-major vector and the i-k-j loop order maximizes L1 cache hits and enables auto-vectorization.
Was this lesson helpful? Spot an issue or have a suggestion? Let us know on GitHub.
Report an issue