Back to the library
software engineering
Production RAG: embeddings, chunking, retrieval quality, and why it returns the wrong document
A RAG pipeline that returns the wrong document, diagnosed and fixed one failure at a time: chunk boundaries, hybrid retrieval, reranking and the evaluation set that proves it moved.
Starts from the failure everyone actually has: retrieval that looks fine until a query needs the exception clause and gets back half a policy.
Every chapter isolates one cause. Chunking that splits a rule from its qualifier, an embedding neighbourhood that puts the wrong thing nearest, a keyword query a vector search cannot serve, a ranking that never sees the right candidate.
You build the inspection harness first, so every later change is measured rather than argued about, and you finish with a working retrieval-augmented generation system and the numbers that show it improved.
Who it's for
- Engineers whose RAG prototype works in a demo and not on the real corpus
- Anyone who has raised top-k, swapped the embedding model, and seen recall stay flat
- People who need an evaluation set before they can defend a retrieval change
Chapter outline
Retrieval Inspection & Local EmbeddingsAfter this part you will be able to embed a document corpus locally, run your first vector search, print raw retrieved chunks alongside answers, and diagnose why a query returns the wrong document.5 Lessons · ~2.5h
Chunking Diagnostics & Semantic BoundariesDiagnose how fixed-size token splitting destroys context by tracing split boundaries against source policy documents, and re-index your corpus using semantic chunking to preserve exception clauses.5 Lessons · ~2.8h
- Diagnosing Fixed-Size Chunking Token Boundaries36m
- Re-Indexing the Corpus with Semantic Chunking58m
- Concept canvas10m
- Verifying Exception Clause Retrieval Recall1h 1m
- Practice checkpoint5m
Hierarchical Chunking & Context SeparationAfter this part you will be able to implement parent-child chunking, balancing narrow vector retrieval precision against the broad context required by the LLM.5 Lessons · ~3h
- Designing Hierarchical Chunking Boundaries49m
- Routing Parent Context to the Generation LLM56m
- Concept canvas10m
- Diagnosing and Resolving the Context Cliff1h 2m
- Practice checkpoint5m
Hybrid Retrieval & Metadata FilteringAfter this part you will be able to combine keyword search with semantic similarity, attach document metadata, and merge ranked lists using reciprocal rank fusion.4 Lessons · ~2.8h
- Implementing BM25 Keyword Search for Exact-Match Queries34m
- Attaching Document-Level Metadata for Filtered Retrieval1h 1m
- Merging Ranked Lists Using Reciprocal Rank Fusion1h 5m
- Practice checkpoint5m
Cross-Encoder Reranking & Evaluation SetsAfter this part you will be able to add a cross-encoder reranker, build a golden evaluation set, and measure retrieval recall objectively.4 Lessons · ~3.1h
- Implementing a Cross-Encoder Reranker56m
- Building a Golden Evaluation Set1h 1m
- Measuring Retrieval Recall and Top-Rank Performance1h 5m
- Practice checkpoint5m
Query Expansion & Domain Embedding TuningAfter this part you will be able to implement query expansion for vocabulary gaps and evaluate how embedding model selection impacts retrieval in specialized domains.4 Lessons · ~2.4h
- Implementing Query Expansion to Bridge User Vocabulary Gaps35m
- Evaluating General Versus Domain-Specific Embedding Models42m
- Diagnosing and Resolving Embedding Neighborhood Failures1h 1m
- Practice checkpoint5m
Versioned Ingestion & Drift MonitoringAfter this part you will be able to build a document-versioned ingestion pipeline with hash-based deduplication, monitor retrieval provenance, and detect corpus drift.4 Lessons · ~3.2h
- Building Hash-Based Document Ingestion and Versioned Deduplication1h 2m
- Implementing Retrieval Provenance Logging and Drift Detection1h
- Production RAG Pipeline Checkpoint and Drift Inspection1h 6m
- Practice checkpoint5m
Build: Functional Retrieval-Augmented Generation (RAG) systemBuild a functional Retrieval-Augmented Generation (RAG) system that demonstrates effective document chunking, embedding generation, and retrieval, and includes mechanisms to analyze and improve retrieval quality.4 Lessons · ~2.7h
- Phase 1: Foundation — RAG Core Concepts41m
- Phase 2: Scaffold — Project Structure and Base Pipeline41m
- Phase 3: Feature — Advanced Chunking Strategies41m
- Phase 4: Polish + Ship41m