Agent QA Gates

A field-tested validation system for AI agent output. Born from production failures, not theory.

Quick Start

Before any agent delivers output, run the Pre-Ship Checklist:

1. Accurate? — every number/date/metric has a source. Unsourced → prefix "estimated"
2. Complete? — no missing pieces, no "I'll do that next"
3. Actionable? — ends with clear next step or decision point
4. Fits the channel? — check character limits for your delivery surface
5. No leaks? — no internal context, private data, or secrets
6. Not a duplicate? — verify no recent identical send
7. Would the human be embarrassed? — if yes, don't ship

Gate Tiers

Four ascending tiers by risk level:

See references/gates-detail.md for full gate checklists.

Severity Classification

Not all failures are equal:

• 🔴 BLOCK — cannot ship (secrets, privacy, hallucinated data, wrong recipient)
• 🟡 FIX — fix before shipping, <2 min (formatting, too long, missing citation)
• 🟢 NOTE — log and ship (style preference, minor optimization)

Protocol Gates

Recurring failure modes need dedicated gates. These are the most common:

Heartbeat / Periodic Check Output

• Binary output: alert text ONLY or status-OK ONLY. Never mixed.
• Every data point verified by current-session tool call. No hallucinated metrics.
• No stale data from previous cycles or pre-compaction sessions.

Post-Compaction / Context Reset

• Do not trust facts from the pre-reset session — verify from files and tools.
• Rerun pending checks from scratch.
• Zero carryover for periodic checks.

Scheduled Job / Cron Changes

• Explicit timeout set
• Explicit model set
• Verify schedule after creation
• Output fits destination channel limits

Sub-Agent Output Review

• Does output match the brief's success criteria?
• Any uncertainty flags unresolved?
• Is the reasoning (not just the conclusion) sound?

Isolated Agent / Cron Output (real-world data)

For any cron or sub-agent that reports external data without orchestrator review:

• Did the agent make a verifiable live tool call? Is the raw response traceable?
• Any names, dates, amounts, or IDs that can't be traced to a tool result? → 🔴 BLOCK
• If tool call failed: output must be DATA_UNAVAILABLE — [reason], not fabricated data
• Does the cron prompt include the Real-World Data Verification Rule?

Severity: Fabricated real-world data = 🔴 BLOCK. Same as hallucinated metrics.

Delegated Work Acceptance

For any non-trivial delegated task (especially builds, audits, config changes, or external deliverables):

• Does the handoff include a clear artifact path or proof object?
• Did the worker report exact commands run rather than vague claims?
• Did verification actually happen, with results stated?
• Is the output non-empty and specific, not just "done" or "completed successfully"?
• Are known gaps / next actions named explicitly?
• If the handoff is empty, artifact-free, or self-certifying without proof → 🔴 BLOCK
• Valid dispositions: Done, Revision Needed, Blocked, Failed, Stale

Silent Worker / Stale Task Classification

For delegated work that appears to be running:

• Was the spawn actually accepted? If not, it is not running.
• No start signal within 10 minutes after accepted spawn → Stale
• No materially new output for 30 minutes on active work → Stale unless the task explicitly justifies a longer quiet window
• Stale work must be investigated, respawned, or escalated — never left as indefinite In Progress

Gate Evolution

Gates should evolve based on real failures, not imagination:

1. When a failure occurs → log it with root cause
2. Same failure class occurs 2+ times → add a gate item
3. Monthly: prune gates that haven't caught anything in 60 days

Anti-Patterns

• Gates that sound good but never catch anything → kill them
• Per-agent checklists that duplicate general gates → merge or reference
• "ADHD-friendly" or "high-quality" as gate items → not testable, replace with mechanical checks
• Aspirational gates nobody runs → either automate or cut

Adapting to Your System

This skill provides the pattern. Adapt it:

1. Start with the Pre-Ship Checklist — it works for any agent system
2. Add Protocol Gates for your top 3 recurring failure modes
3. Set channel limits for your delivery surfaces
4. Map real failures to gates — if a failure isn't gated, add the gate
5. Kill gates that never fire — a shorter, sharper checklist wins

For the full reference implementation, see references/gates-detail.md.

For automation scripts, see scripts/qa-check.sh.