DSA DSA In Depth
0%
lesson stacksshunting-yardparsing

Call Stacks, Dijkstra's Shunting-Yard Algorithm & AST Evaluation

7 min read Module 1.3: Linked Structures, Monotonic Stacks/Queues & Cache-Conscious Lists Updated Oct 23, 2026
📄 Cheat Sheet
Dijkstra's Shunting-Yard Algorithm & Stack Expression Evaluation
Comp Sci in 5 Dijkstra's Shunting-Yard Algorithm & Stack Expression Evaluation Watch on YouTube ↗
0:00 / --:--

Infix Ambiguity vs. Reverse Polish Notation (Postfix)

Why is an infix expression like $3 + 4 \times 2 \uparrow 3 \uparrow 2$ ambiguous without precedence and associativity rules, whereas its Reverse Polish Notation (postfix) equivalent 3 4 2 3 2 ^ ^ * + requires zero parentheses and zero precedence rules? Because in postfix notation, every operator appears immediately after its operands, encoding the exact post-order traversal of the Abstract Syntax Tree (AST)!

Evaluating an RPN token stream is a single branch-free stack machine pass: scan tokens left to right; push numbers onto an operand stack; when a binary operator $\circ$ arrives, pop the right operand $b$, pop the left operand $a$, and push $(a \circ b)$.

Parsing Invariant: Dijkstra's Shunting-Yard Precedence & Associativity Condition

When converting an infix token stream into RPN using an operator stack op_stack, incoming operators must wait on the stack until their right operand has been completely parsed. When a new binary operator $o_1$ arrives and the top of op_stack is a non-parenthesis operator $o_2$, under what exact condition must $o_2$ be popped and emitted before pushing $o_1$?

$$\text{pop } o_2 \iff \text{prec}(o_2) > \text{prec}(o_1) \;\lor\; \big(\text{prec}(o_2) == \text{prec}(o_1) \land \text{isLeftAssociative}(o_1)\big)$$

Notice how a single boolean flip handles associativity: for left-associative subtraction $a - b - c = (a - b) - c$, the second $-$ sees $\text{prec}(-) == \text{prec}(-)$ with left-associativity and pops the first $-$, emitting a b - c -. For right-associative exponentiation $2 \uparrow 3 \uparrow 2 = 2^{(3^2)} = 512$, the second $\uparrow$ sees $\text{prec}(\uparrow) == \text{prec}(\uparrow)$ with right-associativity and does not pop the first $\uparrow$, pushing onto the stack to emit 2 3 2 ^ ^!

Operator Precedence, Associativity & Unary Disambiguation

Operator Token Precedence Level Associativity Arity Shunting-Yard Pop Condition against Stack Top $o_2$
Unary - (negation) / ! 4 (Highest) Right-to-Left 1 (Unary) Pop $o_2$ only if $\text{prec}(o_2) > 4$
Exponentiation ^ 3 Right-to-Left 2 (Binary) Pop $o_2$ if $\text{prec}(o_2) > 3$ (strictly greater!)
Multiply / Divide *, /, % 2 Left-to-Right 2 (Binary) Pop $o_2$ if $\text{prec}(o_2) \ge 2$
Add / Subtract +, - 1 (Lowest) Left-to-Right 2 (Binary) Pop $o_2$ if $\text{prec}(o_2) \ge 1$

Visualizing AST Post-Order Equivalence & Two-Stack On-the-Fly Evaluation

Expression: 3 + 4 * 2 ^ 3 ^ 2 === 3 + (4 * (2 ^ (3 ^ 2))) = 2051 Operand Stack (vals) 2 (top) 3 2 4, 3 Operator Stack (ops) ^ (right-assoc) ^ (not popped!) * + Unary vs Binary '-' Detection Rule: A '-' token is UNARY negation iff it occurs: 1. At the very beginning of the expression, OR 2. Immediately after '(' or another operator! Pop Order Note: rhs = vals.pop(); lhs = vals.pop();

Shunting-Yard Token Dispatch & Reduction Pipeline

flowchart TD A["Read Next Token T"] --> B{"Token Type?"} B -- "Number" --> C["vals.push(T)"] B -- "'('" --> D["ops.push('(')"] B -- "')'" --> E["While ops.top() != '('
apply_top_op(vals, ops)
Then pop '('"] B -- "Operator o1" --> F["While ops not empty && ops.top() != '('
&& (prec(o2) > prec(o1) || (prec== && left_assoc))
apply_top_op(vals, ops)"] F --> G["ops.push(o1)"]
Hardware Reality: The Hardware Return Stack Buffer (RSB) & Explicit Heap Stacks

When a program executes a machine CALL instruction, x86-64 and ARM64 CPUs push the return address not only onto the architectural stack in L1d ([RSP]), but also into a dedicated 16-to-32 entry hardware cache inside the branch predictor called the Return Stack Buffer (RSB)—allowing RET instructions to predict their jump target in 0 cycles! However, each OS thread's call stack is fixed to a contiguous 1 MB–8 MB virtual memory guard region. Recursive AST evaluation or DFS on a degenerate tree of depth $D = 200,000$ overflows the OS thread stack (SIGSEGV stack overflow), whereas an explicit std::vector-backed state stack grows dynamically on the heap up to available RAM and uses $4\times$ less memory per frame than an ABI function call frame.

Monotonic Stack (Histogram) & Monotonic Queue (Sliding Window Max)
O(N) Single Pass

Every index is pushed once and popped at most once. Inspect how maintaining a strict monotonic invariant resolves Next Greater/Smaller boundaries and sliding window extrema in O(N) time.

Step 1 / 7
Custom Input:
Timeline

Invariant & State

Stack heights strictly non-decreasing: [2]
🔗 Live Pointer & State Transition Graph — Monotonic Stack (Histogram) & Monotonic Queue (Sliding Window Max) Active Pointer Jump/Transition Matched/Optimal
curr 2 [0] 1 [1] 5 [2] 6 [3] 2 [4] 3 [5]
[0]2curr
[1]1
[2]5
[3]6
[4]2
[5]3
Monotonic Increasing Stack: [0(h=2)]
Scan Index i
0 (h = 2)
Stack Depth
1
Popped Rectangles
0
Max Rectangle Area
0 (0)
[Step 1] h[0]=2 maintains increasing order; pushed index 0 onto stack.

Production Implementation: Shunting-Yard Evaluator with Unary Negation & Exponentiation

#include <cctype>
#include <cstdint>
#include <string_view>
#include <vector>

class ExpressionEvaluator {
  enum class Op : char { Add = '+', Sub = '-', Mul = '*', Div = '/', Pow = '^', Neg = '~', LParen = '(' };

  static int precedence(Op op) noexcept {
    switch (op) {
      case Op::Add: case Op::Sub: return 1;
      case Op::Mul: case Op::Div: return 2;
      case Op::Pow:               return 3;
      case Op::Neg:               return 4;
      default:                    return 0;
    }
  }

  static bool is_right_assoc(Op op) noexcept {
    return op == Op::Pow || op == Op::Neg;
  }

  static std::int64_t ipow(std::int64_t base, std::int64_t exp) noexcept {
    if (exp < 0) return 0;
    std::int64_t res = 1;
    while (exp > 0) {
      if (exp & 1) res *= base;
      base *= base;
      exp >>= 1;
    }
    return res;
  }

  static void apply_op(std::vector<std::int64_t>& vals, Op op) noexcept {
    if (op == Op::Neg) {
      vals.back() = -vals.back();
      return;
    }
    // CRITICAL: First popped is RIGHT operand, second popped is LEFT operand!
    const std::int64_t rhs = vals.back(); vals.pop_back();
    const std::int64_t lhs = vals.back(); vals.pop_back();
    switch (op) {
      case Op::Add: vals.push_back(lhs + rhs); break;
      case Op::Sub: vals.push_back(lhs - rhs); break;
      case Op::Mul: vals.push_back(lhs * rhs); break;
      case Op::Div: vals.push_back(lhs / rhs); break;
      case Op::Pow: vals.push_back(ipow(lhs, rhs)); break;
      default: break;
    }
  }

public:
  static std::int64_t evaluate(std::string_view expr) {
    std::vector<std::int64_t> vals;
    std::vector<Op> ops;
    bool expect_unary = true; // True at start, after '(', or after any operator

    for (std::size_t i = 0; i < expr.size();) {
      if (std::isspace(static_cast<unsigned char>(expr[i]))) {
        ++i;
        continue;
      }
      if (std::isdigit(static_cast<unsigned char>(expr[i]))) {
        std::int64_t num = 0;
        while (i < expr.size() && std::isdigit(static_cast<unsigned char>(expr[i]))) {
          num = num * 10 + (expr[i] - '0');
          ++i;
        }
        vals.push_back(num);
        expect_unary = false;
      } else if (expr[i] == '(') {
        ops.push_back(Op::LParen);
        expect_unary = true;
        ++i;
      } else if (expr[i] == ')') {
        while (!ops.empty() && ops.back() != Op::LParen) {
          apply_op(vals, ops.back());
          ops.pop_back();
        }
        if (!ops.empty()) ops.pop_back(); // Pop '('
        expect_unary = false;
        ++i;
      } else {
        Op cur_op = static_cast<Op>(expr[i]);
        if (cur_op == Op::Sub && expect_unary) {
          cur_op = Op::Neg;
        }
        while (!ops.empty() && ops.back() != Op::LParen) {
          const int p_top = precedence(ops.back());
          const int p_cur = precedence(cur_op);
          if (p_top > p_cur || (p_top == p_cur && !is_right_assoc(cur_op))) {
            apply_op(vals, ops.back());
            ops.pop_back();
          } else {
            break;
          }
        }
        ops.push_back(cur_op);
        expect_unary = true;
        ++i;
      }
    }

    while (!ops.empty()) {
      apply_op(vals, ops.back());
      ops.pop_back();
    }
    return vals.empty() ? 0 : vals.back();
  }
};
Production Pitfall: Reversed Operand Pop Order & Unary Minus Precedence

Two classic bugs break stack-based expression evaluators:

  1. Reversed Pop Order on Non-Commutative Operators: Because operands are pushed left-to-right onto the LIFO vals stack, the first value popped (vals.back()) is the right-hand operand (rhs), and the second value popped is the left-hand operand (lhs)! Reversing them leaves + and * unaffected but computes rhs - lhs and rhs / lhs for subtraction, division, and exponentiation.
  2. Replacing Unary -x with 0 - x When Exponentiation Is Present: Pushing a dummy 0 before - works if only +, -, *, / exist, but fails on 2 ^ -3 or -2 ^ 2 because binary - has precedence 1 while unary negation has higher precedence. Track a boolean state expect_unary (true at start of string, after (, or after any operator) to dispatch - as a dedicated unary operator Op::Neg.
Check yourself Question 1 of 4
Concept Check Shunting-Yard Right-Associative Precedence Rule recall

In Dijkstra's Shunting-Yard algorithm (or Pratt parsing), suppose the incoming operator is `^` (exponentiation, precedence 3, right-associative) and the operator at the top of the operator stack is also `^` (precedence 3). Should the stack top be popped before pushing the incoming `^`?

⚡ Interactive Challenge · Staff Level

Algorithmic Design & Complexity Check

Analyze the invariant, asymptotic bound, and hardware behavior before selecting.

Why does each recursive function call consume 32–64+ bytes of stack space when the logical state is only a 4-byte vertex ID?

Key Takeaways

Reverse Polish Notation (postfix) encodes the exact post-order traversal of an Abstract Syntax Tree, eliminating parentheses and precedence rules during stack evaluation.
In Dijkstra's Shunting-Yard algorithm, an incoming operator o1 pops stack-top operator o2 iff prec(o2) > prec(o1) OR (prec(o2) == prec(o1) and o1 is left-associative).
When evaluating a binary operator from an operand stack, the first pop yields the right-hand operand (rhs) and the second pop yields the left-hand operand (lhs).
Unary operators (such as unary minus) are disambiguated in O(1) by checking whether the previous token was the start of input, '(', or another operator.
Was this lesson helpful? Spot an issue or have a suggestion? Let us know on GitHub.
Report an issue