Phase 2 Lab Guide: Building the Deliverables

Curriculum Source: [docs/plan/02_phase2_engineering_of_reasoning.md](file:///Users/huychau/Documents/working/training/ai/docs/plan/02_phase2_engineering_of_reasoning.md)
Evaluation Skill: [lab-evaluator](file:///Users/huychau/Documents/working/training/ai/.agents/skills/lab-evaluator/SKILL.md)

---

🎯 Required Deliverables Specification

When you are ready to write the code for Phase 2, you will implement the following 5 deliverables:

---

1. The Multi-Path Planner (Tree of Thoughts)

• Objective: Implement a Tree of Thoughts pipeline in Python to plan a complex software architecture feature (e.g. migrating an auth service to OAuth2/OIDC with zero downtime).
• Requirements:
  • Automatically brainstorm at least 3 distinct technical approaches.
  • Implement an evaluator heuristic or LLM-as-a-Judge scoring each approach against:
    • Availability & Zero-Downtime Feasibility
    • Implementation Complexity
    • Security & Compliance
  • Output the final chosen architecture with an executive justification.

---

2. The Efficiency Report (CoT vs. Chain of Draft)

• Objective: Produce a documented comparative benchmark showing the token and latency savings of Chain of Draft.
• Requirements:
  • Solve a multi-step capacity planning or logic problem across 10 iterations using standard verbose Chain-of-Thought.
  • Solve the identical problem across 10 iterations using Chain of Draft (sketch-like scratchpad).
  • Produce a Markdown report comparing:
    • Total completion tokens consumed
    • Mean latency (seconds)
    • Accuracy percentage
    • Empirical token savings percentage (target $\ge 50\mathrm{\%}$).

---

3. The Self-Correcting Research Assistant (ReAct)

• Objective: Build an autonomous, closed-loop ReAct agent in Python.
• Requirements:
  • Implements the cyclic loop: Thought $\to$ Action $\to$ Observation $\to$ Thought.
  • Configures stop sequences (stop=["Observation:", "\nObservation"]) to prevent observation hallucinations.
  • Demonstrates self-correction: when given an ambiguous query where the initial search returns irrelevant or conflicting data, the agent evaluates the observation and refines its search query autonomously before answering.

---

4. The Meta-Refiner (Meta-Prompting)

• Objective: Build a meta-prompting system that converts vague, underspecified natural language requests into structured, hardened prompts.
• Requirements:
  • Architect LLM analyzes user requests for missing boundary conditions, edge cases, and required schemas.
  • Generates a production-ready, XML-delimited prompt contract ready for downstream execution.

---

5. Alignment Audit Proof (Goal Hijacking Defense)

• Objective: Implement an AlignmentCheck semantic guardrail.
• Requirements:
  • Evaluates whether an agent's candidate tool action logically aligns with the root user intent ($G_0$).
  • When fed an adversarial tool observation attempting to redirect the task (e.g. exfiltrating data or dropping a table), the guardrail catches the goal drift and immediately aborts execution.
  • Produces a verifiable trace showing the detected violation.