Part IV: Engineering AI Products
Chapter 17

Retrieval and Knowledge Systems

Your AI product passes every eval. It generates fluent, confident responses that look right. Users start relying on it for high-stakes decisions. Then someone notices the AI is citing a medical study that does not exist, referencing a product that was discontinued three years ago, and pulling pricing data from a database that has not been updated in eight months. The model is fine. The retrieval failed silently, returning plausible but wrong context that the AI then used to generate authoritative-sounding nonsense. The quality of an AI product is often determined by the quality of its retrieval. Garbage in, garbage out. Building robust retrieval systems is more important than choosing the right model.
The Tripartite Loop in Retrieval Systems and Knowledge

Building retrieval systems requires all three disciplines working together: AI PM defines what knowledge the product needs access to and how relevance should be measured; Vibe-Coding experiments with different retrieval strategies, chunking approaches, and ranking algorithms to find what works; AI Engineering implements the retrieval pipeline, vector database, and caching that make retrieval fast and reliable.

Chapter 17 opener illustration
Retrieval systems connect AI models to relevant knowledge at inference time.
A diagram showing the RAG pipeline: documents go in, get chunked and embedded, stored in vector DB, then at query time the user question is embedded and matched against stored vectors to retrieve relevant context for the LLM
The RAG pipeline: documents to vectors to retrieval to generation. Each stage is a potential failure point.
Two panels: left shows a messy library with books everywhere and a person looking overwhelmed (search without retrieval), right shows a helpful librarian holding the exact right book (semantic retrieval)
The library vs librarian: having all the data is not the same as retrieving the right data. Retrieval is the librarian, not the library.
Vibe-Coding in Retrieval Behavior Probing

Use vibe coding to probe retrieval behavior before investing in production RAG systems. Quickly test different embedding models, chunking strategies, and query formulations against your actual data. Vibe coding retrieval variants lets you discover failure modes like irrelevant context, missing top results, and chunk boundary issues before they affect users, reducing the time spent debugging retrieval in production.

PM Decision Points in Retrieval Systems

PMs must decide: What is the acceptable staleness of retrieved content? How do retrieval failures map to user-facing failures? When should the system say "I do not know" versus guessing? These decisions directly affect product reliability perceptions. PMs should define explicit data freshness requirements, establish escalation paths when retrieval fails, and determine which high-stakes domains require human verification rather than AI retrieval. Retrieval quality often matters more than model quality for user trust.

Objective: Master retrieval architectures from basic RAG to advanced knowledge systems.

Chapter Overview

This chapter covers the engineering decisions that determine how models are selected, routed, and allocated to tasks. Model selection involves understanding open versus closed models, size versus capability trade-offs, and task-model matching. Model routers direct requests to appropriate models based on task requirements, cost constraints, and quality targets. Ensembles and specialization combine multiple models for better results than any single model. Structured outputs and tool compatibility enable reliable integration with external systems. The chapter concludes with latency, cost, and quality trade-offs that guide optimization priorities.

Four Questions This Chapter Answers

  1. What are we trying to learn? How to build retrieval systems that consistently provide the right context to AI models at the right time.
  2. What is the fastest prototype that could teach it? A simple RAG prototype with your actual data that reveals whether retrieval quality is the bottleneck in your AI pipeline.
  3. What would count as success or failure? Retrieval systems where garbage-in-garbage-out failures are caught and measured, not hidden downstream in model behavior.
  4. What engineering consequence follows from the result? Retrieval quality often matters more than model quality; investment in retrieval infrastructure typically has higher ROI than model upgrades.

Learning Objectives

Sections in This Chapter