Module 2.2: Strategic Exploration

Curriculum Alignment: docs/plan/02_phase2_engineering_of_reasoning.md
Topic Scope: Tree of Thoughts (ToT), Decision Trees & Backtracking, Self-Consistency Voting, LLM-as-a-Judge
Level: Advanced AI Engineering / Architecture

1. Why Linear Sequences Fail: The Need for Graph Search

Standard Chain-of-Thought is fundamentally a greedy search with a branching factor of 1 and zero backtracking:

State s_{0} \overset{greedy}{\to} s_{1} \overset{greedy}{\to} s_{2} \dots \overset{greedy}{\to} s_{n}

If the model makes a single calculation or assumption error at step $s_{1}$ , it suffers from compounding hallucination: every subsequent token reinforces the flawed premise, with no mechanism to undo the mistake.

2. The Tree of Thoughts (ToT) Framework

Tree of Thoughts (ToT) elevates language model inference into a deliberate search algorithm over a state space:

The 4 Pillars of ToT:

Thought Decomposition: Partitioning the complex problem into discrete, evaluable reasoning units (e.g. one design decision or logical constraint per node).
Thought Generator ( $G (s)$ ): Emitting $k$ distinct candidate thoughts for the current state $s$ .
State Evaluator ( $V (s)$ ): A separate heuristic or evaluator LLM scoring the viability of each path (e.g. Sure, Maybe, Impossible or scores from $1 -- 10$ ).
Search Algorithm: Orchestrating tree traversal via Breadth-First Search (BFS) or Depth-First Search (DFS) with explicit backtracking when a node evaluates poorly.

3. Self-Consistency: Stochastic Ensembling

When solving problems with a single ground truth, greedy decoding ( $T = 0.0$ ) can get trapped in suboptimal local minima.

Self-Consistency replaces greedy decoding with ensemble sampling:

Sample $N$ independent reasoning paths using a moderate temperature ( $T = 0.6 - 0.8$ ).
Extract the final answer from each reasoning path.
Compute the majority vote (or use an LLM judge to reconcile discrepancies).

\hat{y} = \arg max_{a \in A} \sum_{i = 1}^{N} 1_{{y_{i} = a}}

Majority Vote vs. LLM-as-a-Judge

Majority Vote: Appropriate for deterministic, discrete outputs (integers, exact dates, predefined category strings).
LLM-as-a-Judge: Prompts an independent evaluator model to review divergent reasoning chains, weigh trade-offs, and arbitrate which logical justification is soundest.

TIP

Engineering Trade-off: Self-Consistency increases reliability on mathematical and logical proofs by $15 -- 25 %$ , but multiplies inference cost and token consumption by $N \times$ . Use it selectively for critical decision nodes.

Conceptual Mindmap: Strategic Exploration

4. Curated Reading & Canonical References

Resource	Canonical Reference & Link	Specific Focus Areas
Primary Curriculum Book	AI Agents and Applications (Google Drive)	Chapter 2, Section 2.5 (p. 74): "Beyond Chain of Thought: Tree of Thought" — overcoming token-by-token greedy forward generation through state evaluation and multi-path branching.
Foundational ToT Paper	Tree of Thoughts: Deliberate Problem Solving (Yao et al., 2023)	Search heuristics ( $G (s)$ generator, $V (s)$ evaluator), BFS/DFS decision trees, and backtracking.
Self-Consistency Paper	Self-Consistency Improves Chain of Thought (Wang et al., 2022)	Majority voting consensus over stochastic sampling trajectories ( $T > 0$ ).

5. Active Recall (Module 2.2 Flashcards)

Strategic SearchClick or press Space to flip ↺

How does Tree of Thoughts (ToT) solve the 'compounding hallucination' problem of standard CoT?

Strategic Search • AnswerClick to flip back ↻

Standard CoT uses greedy search with branching factor 1; if a mistake is made at step 1, every subsequent token reinforces the error. ToT explicitly generates multiple candidate thoughts per state, evaluates their viability with an evaluator model, and backtracks when a path is deemed unviable.

💡 Architect Takeaway: ToT transforms generation from linear prediction into a search problem over a state tree.

Consensus MechanicsClick or press Space to flip ↺

Why does Self-Consistency sample at temperature T=0.7 instead of greedy T=0.0?

Consensus Mechanics • AnswerClick to flip back ↻

At T=0.0, the model is completely deterministic and produces identical reasoning paths every time (diversity = 0). Sampling at T=0.6-0.8 generates diverse, independent reasoning chains; taking a majority vote across diverse correct paths filters out idiosyncratic stochastic errors.

💡 Architect Takeaway: Self-Consistency leverages diversity to find consensus on high-probability truths.

6. Hands-on Engineering Exercises

Exercise 2.2: The Multi-Path Planner Drill

Goal: Build the conceptual blueprint for your Multi-Path Planner deliverable.
Problem: Design a software feature implementation (e.g. designing a high-reliability distributed idempotency layer or job scheduling engine).
Requirements:
1. Brainstorm 3 distinct implementation approaches using Tree of Thoughts.
2. Implement an evaluation rubric (Latency, Fault Tolerance, Complexity) scored by an evaluator LLM.
3. Formulate a final synthesis justifying the chosen path.

Module 2.2: Strategic Exploration ​

1. Why Linear Sequences Fail: The Need for Graph Search ​

2. The Tree of Thoughts (ToT) Framework ​

The 4 Pillars of ToT: ​

3. Self-Consistency: Stochastic Ensembling ​

Majority Vote vs. LLM-as-a-Judge ​

Conceptual Mindmap: Strategic Exploration ​

4. Curated Reading & Canonical References ​

5. Active Recall (Module 2.2 Flashcards) ​