Module 1.3: Systematic Prompt Structure & In-Context Learning

Curriculum Alignment: docs/plan/01_phase1_engine_and_prompting.md
Topic Scope: Systematic Prompt Components, Zero-Shot, Few-Shot & One-Shot Prompting, LangChain PromptTemplate, Pydantic Schemas
Level: Advanced AI Engineering / Architecture

1. The 6-Component Systematic Prompt Architecture

Natural language instructions are inherently prone to misunderstanding and format drift. To achieve production reliability, prompts must be organized into six systematic components:

text

┌────────────────────────────────────────────────────────┐
│ 1. PERSONA                                             │
│    Who the model is: role, tone, and operational limits│
├────────────────────────────────────────────────────────┤
│ 2. CONTEXT                                             │
│    Background scenario, domain rules, and axioms       │
├────────────────────────────────────────────────────────┤
│ 3. INSTRUCTION                                         │
│    The primary directive or transformation task        │
├────────────────────────────────────────────────────────┤
│ 4. INPUT CONTAINER                                     │
│    Untrusted user payload enclosed in explicit tags    │
├────────────────────────────────────────────────────────┤
│ 5. STEPS                                               │
│    Explicit procedural logic the model must follow     │
├────────────────────────────────────────────────────────┤
│ 6. OUTPUT FORMAT                                       │
│    Exact schema, JSON keys, or structure required      │
└────────────────────────────────────────────────────────┘

2. In-Context Learning (ICL): Zero-Shot vs. Few-Shot vs. One-Shot

In-Context Learning (ICL) allows an LLM to adapt to new tasks during inference without altering any underlying model weights.

The Mechanics of Few-Shot Learning

When you provide demonstration pairs in the prompt, the model's self-attention layers compute cross-attention over these examples.
This acts as activation steering: the demonstrations form a temporary latent context vector that strongly biases token generation toward the desired style, taxonomy, and syntactic structure.

Comparison Matrix

Paradigm	Definition & Best Use	Pros	Failure Modes
Zero-Shot	Relying purely on the model's pre-trained weights without providing any examples. Best for standard categorization and general text manipulation.	Zero token overhead; lowest cost and latency.	Sensitive to subtle wording changes; high variance in output formatting.
One-Shot	Providing exactly one demonstration pair. Best for illustrating output formatting and schema alignment.	Minimal token overhead; immediately clarifies complex formatting requirements.	The model can easily overfit to the single example's specific contents.
Few-Shot (3–5)	Providing multiple diverse demonstration pairs. Best for nuanced domain taxonomy and edge-case handling.	High accuracy and reliability on edge cases; consistent tone and formatting.	Increases prompt token count and latency; order of examples can introduce recency bias.

TIP

Few-Shot Best Practices:

Balance Class Frequencies: If classifying tickets into 3 categories (Bug, Feature, Question), provide an equal number of examples for each class to avoid frequency bias.
Include Hard Negatives: Include examples showing what not to classify into a given category.

3. Structural Templating: LangChain PromptTemplate

In production code, avoid raw string concatenation or manual f-strings. Use LangChain's PromptTemplate to parameterize prompts cleanly:

python

"""
Structural Prompting via LangChain and Pydantic v2
Demonstrates parameterized template construction with isolated input containers.
"""
from enum import Enum
from langchain_core.prompts import ChatPromptTemplate
from pydantic import BaseModel, ConfigDict, Field

class TicketCategory(str, Enum):
    AUTHENTICATION_ISSUE = "AUTHENTICATION_ISSUE"
    BILLING_DISPUTE = "BILLING_DISPUTE"
    PERFORMANCE_DEGRADATION = "PERFORMANCE_DEGRADATION"
    FEATURE_REQUEST = "FEATURE_REQUEST"
    UNKNOWN = "UNKNOWN"

class SupportTriageResult(BaseModel):
    """Target output schema."""
    model_config = ConfigDict(frozen=True, extra="forbid")
    
    category: TicketCategory = Field(description="Primary category of the ticket.")
    urgency: int = Field(ge=1, le=5, description="Urgency score from 1 to 5.")
    reasoning: str = Field(description="Concise justification for the classification.")

def build_systematic_triage_prompt() -> ChatPromptTemplate:
    """Builds a structured prompt template using LangChain."""
    system_message = """You are the Sentinel Triage Engine, an automated classifier for enterprise support.

<operational_context>
Analyze incoming support tickets and categorize them into the defined schema.
</operational_context>

<execution_steps>
1. Read the customer input contained strictly within <customer_input>...</customer_input>.
2. Determine the core issue and assign one of the predefined categories.
3. Assess urgency: 5 (critical outage) down to 1 (minor query).
4. Return strictly valid JSON conforming to the requested schema.
</execution_steps>

<few_shot_examples>
Example 1:
<customer_input>Database latency spiked to 4500ms right after the migration in us-east-1.</customer_input>
Output: {"category": "PERFORMANCE_DEGRADATION", "urgency": 5, "reasoning": "High database latency impacting production."}

Example 2:
<customer_input>Can you add dark mode to the dashboard settings?</customer_input>
Output: {"category": "FEATURE_REQUEST", "urgency": 2, "reasoning": "UI customization request with non-blocking priority."}
</few_shot_examples>
"""

    human_message = """Analyze the support ticket below:
<customer_input>
{customer_text}
</customer_input>
"""

    return ChatPromptTemplate.from_messages([
        ("system", system_message),
        ("human", human_message)
    ])

### Dynamic Few-Shot Separation: LangChain `FewShotPromptTemplate` (*AI Agents and Applications*, Ch. 2, p. 71)

As demonstrated in Listing 2.1 of the textbook, hardcoding few-shot examples into string templates is an anti-pattern. LangChain's `FewShotPromptTemplate` allows you to define examples as data dictionaries, format each example via an `example_prompt`, and inject them dynamically:

```python
"""
Dynamic Few-Shot Injection via FewShotPromptTemplate
Adapted from AI Agents and Applications (Chapter 2, Listing 2.1)
"""
from langchain_core.prompts.few_shot import FewShotPromptTemplate
from langchain_core.prompts.prompt import PromptTemplate

# 1. Define demonstration pairs as data dictionaries
classification_examples = [
    {"input": "Server returned HTTP 504 Gateway Timeout during peak traffic.", "category": "PERFORMANCE_DEGRADATION"},
    {"input": "User unable to reset password via SSO link.", "category": "AUTHENTICATION_ISSUE"},
    {"input": "Double charged for enterprise subscription invoice.", "category": "BILLING_DISPUTE"},
]

# 2. Define how each individual example is rendered
example_prompt = PromptTemplate(
    input_variables=["input", "category"],
    template="Input: {input}\nCategory: {category}"
)

# 3. Assemble the dynamic FewShotPromptTemplate
few_shot_prompt = FewShotPromptTemplate(
    examples=classification_examples,
    example_prompt=example_prompt,
    prefix="Classify each support ticket into the correct category.\n\nExamples:",
    suffix="\nNow classify the following ticket:\nInput: {ticket_text}\nCategory:",
    input_variables=["ticket_text"]
)

Conceptual Mindmap: Systematic Prompt Structure

4. Curated Reading & Canonical References

Resource	Canonical Reference & Link	Specific Focus Areas
Primary Curriculum Book	AI Agents and Applications (Google Drive)	Chapter 2 (pp. 54–78): Section 2.3 (Prompt templates, p. 59), Section 2.5 (In-context learning & FewShotPromptTemplate, pp. 67–74), Section 2.6 (Prompt structure, pp. 75–78).
Systematic Survey	The Prompt Report (Schulhoff et al., 2024)	Cited in textbook Ch. 2: Systematic analysis of prompt components and delimiter effectiveness.
Industry Research	Patronus AI: Advanced Prompt Engineering	Structural prompting, output anchoring, and preventing schema degradation.
DAIR.AI Guide	Prompt Engineering Guide — Few-Shot Prompting	Mitigating recency bias, selecting representative demonstration distributions.
Model Vendor Guide	Anthropic: Use XML Tags to Isolate Content	Establishing hard boundaries between instructions and untrusted data containers.

5. Active Recall (Module 1.3 Flashcards)

Prompt ArchitectureClick or press Space to flip ↺

Why must user-provided text be wrapped in explicit XML delimiters like <customer_input>?

Prompt Architecture • AnswerClick to flip back ↻

Delimiters create an unambiguous syntactic boundary between the model's instructions and untrusted user data, preventing instruction hijacking.

💡 Architect Takeaway: Never concatenate raw user input directly into system instruction strings.

In-Context LearningClick or press Space to flip ↺

How does Few-Shot demonstration work inside the transformer without updating weights?

In-Context Learning • AnswerClick to flip back ↻

Few-shot examples act as activation steering: self-attention heads compute cross-attention across demonstration pairs, forming a temporary latent context vector that biases output token distribution.

💡 Architect Takeaway: Always balance few-shot class frequencies to avoid introducing model bias.

6. Hands-on Engineering Exercises

Exercise 1.3: Structural Prompt Conversion

Task: Convert a vague natural language prompt (e.g. "Review this code and tell me if it is good or has bugs") into a systematic 6-part LangChain ChatPromptTemplate.
Requirements:
- Define Persona, Context, Instruction, Input Container, Steps, and Output Format.
- Include 2 balanced few-shot demonstration pairs.
- Enforce JSON output validation using Pydantic.

Module 1.3: Systematic Prompt Structure & In-Context Learning ​

1. The 6-Component Systematic Prompt Architecture ​

2. In-Context Learning (ICL): Zero-Shot vs. Few-Shot vs. One-Shot ​

The Mechanics of Few-Shot Learning ​

Comparison Matrix ​

3. Structural Templating: LangChain PromptTemplate ​

Conceptual Mindmap: Systematic Prompt Structure ​

4. Curated Reading & Canonical References ​

5. Active Recall (Module 1.3 Flashcards) ​