Module 4.6: Compulsory Security — Provenance & Inversion Defense

Curriculum Alignment: docs/plan/04_phase4_context_engineering_agentic_rag.md
Topic Scope: Document Provenance, Corpus Poisoning Defense (BadRAG / PoisonedRAG), Embedding Inversion Defense (Vec2Text)
Level: Advanced AI Engineering / Architecture

---

1. Threat Vectors Unique to Vector Search & RAG

RAG pipelines introduce attack surfaces that do not exist in standard web applications. Attackers do not need to exploit software vulnerabilities in your code; they can exploit the semantic retrieval layer itself through two primary attack vectors:

1. Corpus Poisoning (BadRAG / PoisonedRAG): An attacker injects a subtly crafted malicious document into the shared enterprise knowledge base. When a user asks a target query, the poisoned document is engineered to achieve the highest semantic cosine similarity, injecting malicious instructions or false facts directly into the agent's context.
2. Embedding Inversion (Vec2Text Attacks): Attackers with read access to raw vector embeddings use inverse neural networks (like Morris et al.'s Vec2Text) to mathematically reconstruct the original cleartext from 1536-dimensional vector floats, completely bypassing database encryption.

┌────────────────────────────────────────────────────────────────────────┐
│ RAG SECURITY THREAT MATRIX                                             │
├────────────────────┬────────────────────┬──────────────────────────────┤
│ ATTACK VECTOR      │ EXPLOIT MECHANISM  │ MANDATORY DEFENSE PRIMITIVE  │
├────────────────────┼────────────────────┼──────────────────────────────┤
│ 1. Corpus          │ Ingesting rogue    │ Cryptographic Document       │
│    Poisoning       │ documents to rank  │ Provenance & Signature       │
│    (BadRAG)        │ top on queries     │ Verification (HMAC/RSA)      │
├────────────────────┼────────────────────┼──────────────────────────────┤
│ 2. Embedding       │ Reconstructing     │ Differential Privacy &       │
│    Inversion       │ cleartext from     │ Gaussian Noise Perturbation  │
│    (Vec2Text)      │ dense vector floats│ at Vector Ingestion          │
└────────────────────┴────────────────────┴──────────────────────────────┘

---

2. Document Provenance & Corpus Poisoning Defense

To prevent unauthorized or adversarial documents from being indexed, enterprise RAG pipelines must enforce Cryptographic Document Provenance:

The Ingestion Security Contract:

• Every document must be cryptographically signed by an authenticated identity (uploader_id, organization_id, timestamp, content_sha256).
• A digital signature (e.g. RSA or HMAC-SHA256) is computed over the content and stored in the vector database metadata.
• At retrieval time, unverified or unsigned documents are strictly discarded before reaching the prompt context.

---

3. Embedding Inversion Defense (Vec2Text Attacks)

Dense vector embeddings (e.g. OpenAI text-embedding-3-small) were long assumed to be "one-way hashes". However, research demonstrates that deep generative models can reconstruct up to 90% of original words and sensitive PII directly from raw vectors.

Defense Mechanism: Noise Perturbation

By injecting calibrated, micro-scale Gaussian noise ($ϵ\sim \mathcal{N}(0,{\sigma }^2)$) into the embedding vector before saving it to the vector database:

• Retrieval Utility Preserved: Cosine similarity is minimally affected ($\approx 0.999$ correlation with original ranking).
• Inversion Reconstruction Blocked: The micro-perturbation disrupts the high-frequency latent dimensions that inverse models rely on to reconstruct exact characters and token sequences.

---

4. Production Python Implementation: Hardened RAG Ingestion Pipeline

"""
Hardened Ingestion Pipeline
Demonstrates Cryptographic Document Provenance and Embedding Perturbation.
"""
import hmac
import hashlib
import time
import numpy as np
from pydantic import BaseModel, ConfigDict, Field

SHARED_SIGNING_KEY = b"enterprise-internal-ca-secret-2026"

class ProvenanceMetadata(BaseModel):
    """Immutable audit contract for indexed documents."""
    model_config = ConfigDict(frozen=True, extra="forbid")
    
    document_id: str
    uploader_id: str
    timestamp: float
    content_hash: str
    signature: str

def generate_document_signature(document_id: str, content: str, uploader_id: str) -> ProvenanceMetadata:
    """Signs document content cryptographically."""
    content_hash = hashlib.sha256(content.encode()).hexdigest()
    ts = time.time()
    
    # Message = doc_id + uploader + ts + content_hash
    message = f"{document_id}:{uploader_id}:{ts}:{content_hash}".encode()
    sig = hmac.new(SHARED_SIGNING_KEY, message, hashlib.sha256).hexdigest()
    
    return ProvenanceMetadata(
        document_id=document_id,
        uploader_id=uploader_id,
        timestamp=ts,
        content_hash=content_hash,
        signature=sig
    )

def verify_document_provenance(meta: ProvenanceMetadata, content: str) -> bool:
    """Verifies that document has not been tampered with or injected by an attacker."""
    computed_hash = hashlib.sha256(content.encode()).hexdigest()
    if computed_hash != meta.content_hash:
        return False
        
    message = f"{meta.document_id}:{meta.uploader_id}:{meta.timestamp}:{meta.content_hash}".encode()
    expected_sig = hmac.new(SHARED_SIGNING_KEY, message, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected_sig, meta.signature)

def perturb_embedding_for_privacy(vector: list[float], noise_scale: float = 0.005) -> list[float]:
    """Adds calibrated Gaussian noise to prevent Vec2Text embedding inversion."""
    arr = np.array(vector, dtype=np.float32)
    noise = np.random.normal(loc=0.0, scale=noise_scale, size=arr.shape)
    perturbed = arr + noise
    # Re-normalize vector to unit length for cosine similarity
    norm = np.linalg.norm(perturbed)
    return (perturbed / norm).tolist()

Conceptual Mindmap: Context Security

---

5. Curated Reading & Canonical References

Resource	Canonical Reference & Link	Specific Focus Areas
Research Paper	PoisonedRAG: Knowledge Corruption Attacks (Biese et al., 2024)	Demonstrating corpus poisoning vulnerabilities and adversarial document injection.
Foundational Inversion Paper	Text Embeddings are Invertible (Morris et al., 2023 - Vec2Text)	Reconstructing sensitive cleartext and PII directly from dense float vectors.
OWASP GenAI Standard	OWASP Top 10 for LLMs — LLM03: Training/Context Data Poisoning	Supply chain integrity and document provenance in knowledge bases.

---

6. Active Recall (Module 4.6 Flashcards)

Corpus SecurityClick or press Space to flip ↺

What is Corpus Poisoning (BadRAG / PoisonedRAG), and how does Document Provenance prevent it?

Question / Scenario

Corpus Security • AnswerClick to flip back ↻

Corpus Poisoning occurs when an adversary inserts a malicious document into a shared knowledge base engineered to rank #1 on specific queries, hijacking the LLM's context. Document Provenance attaches cryptographic digital signatures and uploader metadata to every document, immediately rejecting any unverified or tampered documents during retrieval.

💡 Architect Takeaway: Never ingest external documents into enterprise vector stores without signature verification.

Technical Explanation

Vector PrivacyClick or press Space to flip ↺

What is a Vec2Text attack, and how does noise perturbation defend against it?

Question / Scenario

Vector Privacy • AnswerClick to flip back ↻

Vec2Text uses generative neural networks to reconstruct the original cleartext sentences from dense vector embeddings. Adding calibrated micro-scale Gaussian noise to the vector disrupts the reconstruction models while preserving cosine similarity ranking and search accuracy.

💡 Architect Takeaway: Vectors are not irreversible hashes; apply perturbation to protect sensitive document embeddings.

Technical Explanation

---

7. Hands-on Engineering Exercises

Exercise 4.6: Poisoned Document Rejection Drill

• Goal: Build the foundation for your Hardened RAG Pipeline deliverable.
• Task: Simulate an attack by creating a fake compliance document with an altered content hash.
• Verification:
  • Run the document through verify_document_provenance.
  • Prove that the pipeline detects the signature mismatch, logs a security warning, and aborts ingestion before vector embedding.