"You cannot build what you cannot name. Capability maps transform abstract AI potential into organizable, improvable, and hirable competencies."
Head of AI Talent who Built the First Taxonomy
Why Capability Maps Matter
Organizations building AI products need a shared language for AI capabilities. Without this language, discussions about skills, hiring, and development become confused. Is "prompt engineering" one skill or three? Where does "eval literacy" fit? How do you assess whether a team can build a RAG system?
Capability maps provide this shared language. They decompose AI product work into discrete capabilities that can be assessed, developed, hired for, and catalogued. This section presents a capability map framework adapted for AI product organizations.
AI Product Capability Taxonomy
The following taxonomy organizes AI product capabilities into four domains, each with specific competencies:
Domain 1: Model and System Design
This domain covers capabilities related to defining what AI system to build and how it should behave:
This domain covers capabilities related to defining what AI system to build and how it should behave. The competencies include ML architecture selection choosing appropriate model architectures for given problems such as transformers, gradient boosting, and hybrid systems, training strategy design designing data requirements, training procedures, and evaluation approaches, prompt engineering developing effective prompts for desired behaviors, system integration design architecting how AI components integrate with broader systems, and output format design structuring AI outputs for downstream consumption.
Domain 2: Evaluation and Quality
This domain covers capabilities related to assessing and ensuring AI quality:
This domain covers capabilities related to assessing and ensuring AI quality. The competencies include eval framework design building comprehensive evaluation systems as covered in Chapter 21, test case development creating representative test inputs and expected outputs, quality metric selection choosing metrics that track real user value, production monitoring observing AI behavior in production environments, and regression identification detecting when AI quality degrades.
Domain 3: AI Product Management
This domain covers capabilities at the intersection of AI and product thinking:
This domain covers capabilities at the intersection of AI and product thinking. The competencies include AI opportunity identification finding product areas where AI adds value, requirement decomposition translating product needs into AI specifications, risk assessment identifying where AI failures would be most impactful, user expectation calibration setting appropriate user expectations for AI capabilities, and launch strategy planning staged rollouts and canary deployments.
Domain 4: Data and Infrastructure
This domain covers capabilities related to data and operational infrastructure:
This domain covers capabilities related to data and operational infrastructure. The competencies include data pipeline architecture building systems to collect, process, and store training data, feature engineering transforming raw data into model inputs, model deployment getting models from training to production, inference optimization balancing latency, cost, and quality, and observability implementation building systems to monitor AI health.
Skill Level Definitions
Within each capability, define skill levels that enable assessment and development planning:
Skill Level Framework
Level 1 (Foundation): Understands the concept. Can explain what the capability is and why it matters. Can follow established practices with guidance.
Level 2 (Practitioner): Applies the capability independently. Can execute standard approaches, recognize when something is not working, and know when to escalate.
Level 3 (Expert): Adapts the capability to new situations. Can design solutions for novel problems, coach others, and improve organizational practices.
Level 4 (Authority): Defines the capability for others. Can build frameworks, set organizational standards, and advance the state of practice.
Capability Gap Analysis
Once you have a capability taxonomy, use it to analyze current state and identify gaps:
Team Capability Mapping
Once you have a capability taxonomy, use it to analyze current state and identify gaps. Map current team capabilities to the taxonomy to understand coverage by asking which capabilities the team has at each level, which capabilities are concentrated in single individuals creating bus factor risk, which capabilities have no one at Level 2 or above, and which critical capabilities rely on external parties.
Project Capability Requirements
For planned projects, identify required capabilities and compare to available capabilities. The process begins by decomposing the project into required capabilities, then identifying the minimum skill level needed for each capability, comparing to available capabilities in the team, and finally identifying gaps that require hiring, upskilling, or contractor support.
Practical Example: HealthMetrics AI Team Capability Assessment
Who: HealthMetrics engineering leadership assessing their AI team
Situation: Team had grown from 3 to 12 people in 18 months. No systematic understanding of capability coverage.
Process: Mapped all team members to capability taxonomy using self-assessment and manager validation. Aggregated results to see team-level coverage.
Findings: Strong coverage in model deployment (Level 2+ across team). Gaps in eval framework design (only 1 person at Level 3). No coverage in ML architecture selection (all Level 1).
Actions: Hired senior ML engineer for architecture expertise. Created internal training program for eval framework design with the Level 3 expert as lead. Documented architecture decision patterns for organizational learning.
Lesson: Without capability mapping, gaps remain invisible until they cause problems. Systematic mapping reveals where to invest in hiring and development.
Capability Roadmap Development
Build a roadmap that develops capabilities in alignment with product roadmap:
Forward-Looking Gap Analysis
Review the product roadmap for upcoming capabilities needed:
Build a roadmap that develops capabilities in alignment with product roadmap. Review the product roadmap for upcoming capabilities needed by asking what capabilities next year's products require, what the lead time to develop those capabilities is, which gaps require external hiring versus internal development, and what training or learning resources are needed.
Succession and Redundancy Planning
For each critical capability, ensure no single point of failure by identifying capabilities where loss of one person would create crisis, prioritizing developing second owners for those capabilities, and considering contractor or vendor relationships for critical but uncommon capabilities.