Part Overview
Part V focuses on making AI products work reliably. You will master evals, observability, guardrails, cost management, and compliance frameworks.
Interlock with Previous Part
What this part inherits from Part IV:
- Reference architectures (Ch 15) become the eval substrate: each pattern needs specific reliability tests
- Model routing and capability allocation (Ch 16) require eval-driven quality verification
- Retrieval and knowledge systems (Ch 17) need continuous accuracy monitoring
- Security concerns (Ch 20) become guardrail requirements
What this part changes retroactively:
- Engineering decisions get evaluated for reliability impact: some architectures are harder to eval
- Vibe-coding quality bar (Part III) gets replaced with explicit reliability metrics
Artifacts that now need updating:
- Architecture checklists (Appendix E) now include eval requirements
- Eval-first PRDs (Part II) now have concrete measurement frameworks
Chapters in This Part
LLM-as-Judge, eval pipelines, eval-driven development.
Tracing, debugging AI failures, failure mode analysis.
Guardrails, circuit breakers, graceful degradation.
Cost optimization, latency management, unit economics.
NIST AI RMF, ISO 42001, EU AI Act, bias detection.
Bridge Notes
Earlier artifacts updated by this part:
- Part II, Ch 9: Eval-first PRDs now have concrete eval pipelines (Ch 21)
- Part III, Ch 12-13: Prompting and multi-agent patterns get eval coverage
- Part IV, Ch 15-20: Architectures are now evaluated for reliability properties
Later chapters this part prepares for:
- Part VI, Ch 26-30: Launch criteria, SLAs, and compliance are built on eval foundations
- Part VII, Ch 31-33: The capstone uses evals as the quality language throughout