RAG Pipelines (Deep Workflow)

RAG quality is dominated by chunking, retrieval, and evaluation—not the LLM alone. Treat the system as data engineering plus generation with explicit failure modes.

When to Offer This Workflow

Trigger conditions:

• Building Q&A over internal docs, support assistants, or copilots
• Hallucinations, wrong citations, or stale answers
• New content types (PDF, HTML, code repositories)

Initial offer:

Use six stages: (1) task & success criteria, (2) ingestion & cleaning, (3) chunking & metadata, (4) retrieval & rerank, (5) generation & grounding, (6) evaluation & monitoring). Confirm embedding model and retrieval stack (vector DB, search engine, hybrid).

Stage 1: Task & Success Criteria

Goal: Define what a “good” answer contains: required citations, length, tone, and when to refuse.

Exit condition: Written rubric with examples of acceptable vs unacceptable answers.

Stage 2: Ingestion & Cleaning

Goal: Deterministic text extraction (strip boilerplate, handle PDF/OCR if needed); deduplicate documents; track source URL and updated_at for staleness.

Practices

• Version pipelines when parsers change (re-embed job)

Stage 3: Chunking & Metadata

Goal: Tune chunk size and overlap to query patterns—not one global token count for all content.

Practices

• Attach metadata for ACL filtering (tenant, product area)
• Prefer structure-aware splits for docs (headings, sections)

Stage 4: Retrieval & Rerank

Goal: Hybrid lexical + dense retrieval often beats vector-only for keyword-heavy queries.

Practices

• Cross-encoder reranking on top-k for quality (watch latency)
• Query rewriting for multi-turn contexts

Stage 5: Generation & Grounding

Goal: System prompts that require using only provided context; explicit “not found” behavior; optional citation format (snippet, doc id, link).

Stage 6: Evaluation & Monitoring

Goal: Offline golden questions with expected supporting docs; online thumbs-down reasons; monitor retrieval hit rate, nDCG@k, and age of sources used.

Final Review Checklist

• [ ] Rubric and refusal behavior defined
• [ ] Ingestion deterministic; dedupe and versioning
• [ ] Chunking and metadata match queries and ACLs
• [ ] Hybrid retrieval and rerank tuned with metrics
• [ ] Grounding and citation behavior enforced in prompts
• [ ] Offline eval plus production monitoring

Tips for Effective Guidance

• Debug retrieval before blaming the LLM.
• Long chunks hurt precision; short chunks hurt context—sweep experiments.
• See also vector-databases and llm-evaluation skills for depth.

Handling Deviations

• Code RAG: symbol- or AST-aware chunking often beats line-based splits.
• High-stakes domains: add human review gates and audit logs for sources cited.