Maturity Assessment Model
This section provides the detailed maturity assessment model that enables organizations to objectively evaluate their AI-assisted development capabilities, identify gaps, prioritize investments, and track progress over time. The model spans all five AEEF pillars and provides a structured path from initial awareness to optimized mastery.
Maturity Levels
The AEEF maturity model defines five levels, each representing a distinct stage of organizational capability:
| Level | Name | Description |
|---|---|---|
| 1 | Initial | AI tools are used ad-hoc by individuals. No organizational standards, training, or governance. Results are inconsistent and unmeasured. |
| 2 | Developing | The organization has acknowledged AI-assisted development and begun establishing basic standards. Some teams are adopting structured approaches, but consistency is low. |
| 3 | Defined | Standards are documented and widely communicated. Training programs are in place. Most teams follow defined workflows. Quality gates exist but may not be consistently enforced. |
| 4 | Managed | AI-assisted development is measured, managed, and continuously improved. Metrics drive decisions. Quality is consistent. Teams are proficient and share knowledge actively. |
| 5 | Optimized | AI-assisted development is a core organizational capability. Innovation is continuous. The organization contributes to industry best practices. AI is seamlessly integrated into all engineering processes. |
info
Most organizations beginning their AEEF journey will be at Level 1 or 2. Reaching Level 3 is a realistic 6-12 month goal. Level 4 typically requires 12-24 months. Level 5 represents best-in-class maturity that few organizations will achieve across all dimensions simultaneously.
Evaluation Criteria by Pillar
The maturity assessment evaluates capabilities across all five AEEF pillars. Each pillar is scored independently, and the overall maturity level is the minimum of all pillar scores (the weakest link determines overall maturity).
Pillar 1: AI Governance & Risk Management
| Level | Criteria |
|---|---|
| 1 | No formal AI governance. Individual tool selection. No risk assessment. |
| 2 | Basic approved tool list exists. Some data handling guidelines in place. |
| 3 | Comprehensive governance framework documented. Risk assessment process defined. Policies cover data classification, IP, and compliance. |
| 4 | Governance is actively enforced and measured. Regular audits occur. Policy exceptions are tracked and reviewed. |
| 5 | Governance is automated where possible. Predictive risk management. Policies evolve proactively based on emerging threats and regulations. |
Pillar 2: Quality & Security Controls
| Level | Criteria |
|---|---|
| 1 | No AI-specific quality controls. Standard code review applies to all code equally. |
| 2 | AI-generated code is identified and receives additional review attention. Basic security scanning in place. |
| 3 | Formal AI-specific quality gates in CI/CD. Enhanced static analysis for AI code. Test coverage requirements defined. AI code provenance tracking. |
| 4 | Quality metrics show AI-generated code defect rates at or below baseline. Security vulnerability rates are actively managed. Quality gates are continuously refined based on data. |
| 5 | AI-generated code quality consistently exceeds human-only baseline. Advanced security analysis including AI-specific threat modeling. Quality gates are industry-leading. |
Pillar 3: Productivity Architecture
| Level | Criteria |
|---|---|
| 1 | No shared productivity assets. Each developer works independently with AI tools. |
| 2 | Basic prompt sharing occurs informally. Some teams have documented workflows. |
| 3 | Centralized prompt repository in active use. Workflow patterns documented. Toolchain standardized. Basic metrics collection. |
| 4 | Prompt repository actively curated with effectiveness ratings. Automation libraries in production use. Comprehensive metrics drive decisions. Feedback loops operational. |
| 5 | Productivity assets are continuously optimized. Reusable components library is extensive. Metrics show sustained, measurable productivity gains. Feedback loops drive innovation. |
Pillar 4: Operating Model Integration
| Level | Criteria |
|---|---|
| 1 | No operating model adaptation. AI-assisted work fits awkwardly into existing processes. |
| 2 | Teams are beginning to adapt estimation and sprint practices. Changes are informal and team-specific. |
| 3 | Sprint ceremonies formally adapted. Estimation methodology documented. Role definitions updated. Change management plan in execution. |
| 4 | Estimation accuracy matches pre-AI levels. Sprint processes are stable and effective. Team structures optimized. Change management is mature. |
| 5 | Operating model seamlessly incorporates AI capabilities. Predictive planning models in use. Team structures dynamically adapt. The operating model itself is AI-augmented. |
Pillar 5: Organizational Enablement & Maturity
| Level | Criteria |
|---|---|
| 1 | No formal training or cultural initiatives. AI adoption is organic and unguided. |
| 2 | Basic training available. Some awareness of cultural needs. |
| 3 | Structured training programs deployed. AI Champions active. CoE established. Cultural norms documented. |
| 4 | Training is tiered and role-specific. Champions network is effective. CoE is a recognized strategic asset. Culture actively supports experimentation and quality. |
| 5 | Continuous learning is embedded in the culture. The organization is recognized externally for AI-assisted development excellence. The CoE drives industry-level innovation. |
Scoring Methodology
Assessment Process
- Assemble Assessment Team: Include representatives from each pillar area: governance, quality/security, productivity, operations, and organizational enablement
- Gather Evidence: Collect artifacts, metrics, and interview data that demonstrate capability levels
- Score Each Pillar: Rate each pillar on the 1-5 scale using the criteria above
- Calculate Overall Score: The overall maturity level equals the minimum pillar score
- Identify Gaps: Compare pillar scores to identify areas most in need of investment
- Set Targets: Define target scores for each pillar for the next assessment period
Evidence Requirements
Each claimed maturity level MUST be supported by evidence:
| Evidence Type | Examples |
|---|---|
| Artifacts | Policies, standards documents, prompt repositories, training materials, configuration files |
| Metrics | Dashboard screenshots, trend reports, survey results, ROI calculations |
| Interviews | Structured conversations with developers, managers, champions, and CoE staff |
| Observations | Attending sprint ceremonies, reviewing code review processes, observing training sessions |
| Audit Results | Security audit findings, compliance review results, quality gate effectiveness reports |
Scoring Rules
- A pillar MUST demonstrate all criteria for a level to be scored at that level
- Partial achievement of a level's criteria scores at the level below
- Self-assessment scores SHOULD be validated by an independent reviewer (another team, the CoE, or an external assessor)
- When evidence is ambiguous, score conservatively -- overestimating maturity prevents necessary investment
warning
Maturity assessment is a diagnostic tool, not a grading system. The purpose is to identify where to invest, not to rank or judge. Organizations that use maturity scores punitively will receive inflated self-assessments that hide real gaps.
Benchmarking
Internal Benchmarking
Organizations with multiple teams or business units SHOULD benchmark maturity across internal groups:
- Compare pillar scores across teams to identify best practices and struggling areas
- Investigate why some teams score higher -- is it tools, training, leadership, or culture?
- Use internal benchmarking to set realistic targets based on what peer teams have achieved
- Share specific practices from high-maturity teams with lower-maturity teams
Industry Benchmarking
Where available, organizations SHOULD compare their maturity scores to industry data:
| Maturity Level | Typical Organization Profile |
|---|---|
| Level 1 | Organizations that have not yet formally addressed AI-assisted development |
| Level 2 | Organizations in early adoption, typically within first 2 months |
| Level 3 | Organizations with 2-6 months of structured AI-assisted development |
| Level 4 | Organizations with 6+ months and strong CoE support |
| Level 5 | Industry leaders with 12+ months of mature, measured AI-assisted development |
tip
Do not compare your maturity score to vendor marketing claims or conference keynote aspirations. Compare to peer organizations with similar industry, regulatory, and technical complexity. The Center of Excellence SHOULD maintain benchmarking relationships with peer organizations.
Self-Assessment Checklist
Use this checklist for a rapid initial assessment. For each item, indicate whether it is Not Started, In Progress, or Complete.
Governance & Risk (Pillar 1)
- Approved AI tool list exists and is enforced
- Data classification policy addresses AI tool usage
- IP and licensing policies cover AI-generated code
- Compliance requirements for AI-assisted development are documented
- Regular governance audits are conducted
Quality & Security (Pillar 2)
- AI-generated code is identifiable in the codebase
- Enhanced code review practices exist for AI-generated code
- CI/CD pipelines include AI-specific quality gates
- Security scanning includes AI-specific vulnerability patterns
- Test coverage requirements are defined and enforced for AI-generated code
Productivity Architecture (Pillar 3)
- Centralized prompt repository exists and is actively used
- Developer workflow patterns are documented
- Toolchain is standardized across teams
- Productivity metrics are collected and reported
- Feedback loops are operational and driving improvements
Operating Model (Pillar 4)
- Sprint ceremonies are adapted for AI-assisted development
- Estimation methodology accounts for AI acceleration variability
- Role definitions reflect AI-augmented expectations
- Change management plan exists and is in execution
- AI Champions are designated and active in teams
Organizational Enablement (Pillar 5)
- Training programs are available for all engineering roles
- AI literacy is widespread across the engineering organization
- Psychological safety for AI experimentation is cultivated
- Center of Excellence is established and operational
- Maturity assessment is conducted regularly
Assessment Cadence
| Assessment Type | Frequency | Conducted By | Purpose |
|---|---|---|---|
| Quick Self-Check | Monthly | Team leads using checklist | Track momentum, identify emerging issues |
| Formal Self-Assessment | Quarterly | CoE + pillar representatives | Comprehensive scoring, gap identification, target setting |
| Independent Assessment | Annually | External assessor or cross-organizational peer | Validation, benchmarking, strategic planning |
Using Assessment Results
Assessment results MUST drive action. The following framework connects scores to priorities:
Gap Prioritization
When multiple pillars have gaps, prioritize based on:
- Foundation First: Governance (Pillar 1) and Quality (Pillar 2) gaps MUST be addressed before optimizing Productivity (Pillar 3) or Operations (Pillar 4)
- Enablement Parallel: Organizational Enablement (Pillar 5) SHOULD be advanced in parallel with all other pillars
- Biggest Gap: Among equally foundational pillars, address the largest gap first
- Quick Wins: If a pillar is close to the next level, a small investment may yield a level advancement
Target Setting
- Set pillar-level targets for the next assessment period (typically one quarter)
- Advancing one level per pillar per year is an ambitious but achievable pace
- Not all pillars need to advance simultaneously -- prioritize based on gap analysis
- Targets MUST be accompanied by specific investment plans (training, tools, process changes, staffing)
danger
Do not set a target of Level 5 across all pillars within the first year. This is unrealistic and will demoralize teams. Set achievable targets, celebrate progress, and build momentum incrementally. Sustainable improvement beats aspirational failure.