Module 2.3: Interactive Reasoning — The ReAct Framework

Curriculum Alignment: [docs/plan/02_phase2_engineering_of_reasoning.md](file:///Users/huychau/Documents/working/training/ai/docs/plan/02_phase2_engineering_of_reasoning.md)
Topic Scope: Thought-Action-Observation Loop, Stop Sequences, Tool Parameter Marshalling, Self-Correction
Level: Senior Architect / Advanced AI Engineering

---

1. Systems Perspective: Closed-Loop Control Systems

Pure reasoning (CoT, ToT) is open-loop: the model calculates answers based solely on its internal frozen training weights. If information is outdated, private, or incorrect, the reasoning is compromised.

ReAct (Reason + Act) transforms the LLM into a closed-loop feedback controller:

Why Coupling "Reasoning" with "Acting" is Critical:

• Action without Reasoning (pure function calling): The model greedily selects tools without understanding context, often leading to invalid parameters or wrong tool selection.
• Reasoning without Action (pure CoT): The model speculates about facts it does not know, leading to hallucination.
• ReAct Synergy: The model uses "Thoughts" to dynamically adjust its plan based on the actual "Observations" returned by external APIs.

---

2. Engineering Failure Modes in ReAct Pipelines

When implementing production ReAct agents, watch for these 3 critical systems failure modes:

1. The Observation Hallucination Bug

If the LLM is not halted immediately after emitting an Action, it will invent (hallucinate) the tool's return value:

Action: search_database(user_id="492")
Observation: Found user 492 with email "fake@example.com"  <-- THE MODEL HALLUCINATED THIS!

• Architectural Fix: Use Stop Sequences. Configure the LLM sampling call with stop=["Observation:", "\nObservation"]. As soon as the model outputs the Action: ... line, the runtime stops generation, executes the real Python function, appends the real observation, and re-prompts the model.

2. Observation Context Flooding

A tool (such as a web scraper or SQL query) returns 100,000 characters of raw data, immediately exceeding context limits or triggering the "Lost in the Middle" degradation.

• Architectural Fix: Middleware truncation. Truncate observations to strict token bounds (e.g. max 1,500 tokens) or pass them through a summarizer before appending to context.

3. Tool Parsing Drift

The model emits slightly malformed JSON for tool parameters (e.g. single quotes instead of double quotes, trailing commas).

• Architectural Fix: Pydantic schema validation paired with an automated retry loop that feeds the validation error back to the model as an observation.

---

3. Curated Reading & Canonical References

Resource	Canonical Reference & Link	Specific Focus Areas
Primary Curriculum Book	AI Agents and Applications (Google Drive)	Chapter 2, Section 2.5 ("Interactive Reasoning"): Tool orchestration, observation integration, and error recovery.
Foundational ReAct Paper	ReAct: Synergizing Reasoning and Acting (Yao et al., 2022)	Thought-Action-Observation loops, overcoming sycophancy, and external grounded reasoning.
Framework Standard	LangChain Agent Architecture	State machine loops, tool invocation protocols, and stop sequences.
Production Architecture	FastAPI LangGraph Production Template (GitHub)	Asynchronous agent runtime, middleware pipelines, and structured tool handling.

---

4. Senior Exercises

Exercise 2.3: The Self-Correcting Research Assistant

• Goal: Build the foundation for your Self-Correcting Research Assistant deliverable.
• Task: Implement a ReAct loop in Python with access to two tools: search_api and calculator.
• Challenge:
  • Feed the agent a query where the first search returns an ambiguous result (e.g. multiple people sharing the same name).
  • Ensure the agent evaluates the observation in its next "Thought" and refines its search query autonomously before concluding.