Measuring success
If you can't measure it, you can't lead it — or defend the role.
Track both org/value metrics and your-own-growth metrics. Be conservative and honest — execs trust honest numbers, and vanity metrics destroy the trust your role depends on.
Key ideas
- 1
Org/value metrics: adoption (active usage, not seats), DORA metrics before/after, eval scores & AI-incident rate, value (time saved, cost per unit), and % of use cases through risk triage.
- 2
Your-own-growth metrics: # champions trained & active, # teams shipping AI features WITHOUT you in the room, reusable artifacts shipped, cross-BU reach.
- 3
Measure outcomes (cycle time, defect rate, satisfaction), not vanity (lines of AI-generated code).
- 4
Conservative, honest numbers build the trust that vanity metrics destroy.
Org / value metrics
- Adoption: teams/engineers actively on the paved road.
- Throughput: DORA (lead time, deploy frequency, change-fail rate, MTTR) before vs after; ticket cycle time.
- Quality: eval scores over time; AI-feature incident rate; % AI features with evals in CI.
- Value: time saved, cost per unit (per PR/ticket/case), business outcomes per use case.
- Risk: % use cases through triage; # red-team findings closed.
Your own growth
The clearest signal you're succeeding: teams ship AI features without you in the room, and the number of active champions grows.
Watch
Do the work
0/4 · 0%Test yourself
Which is a vanity metric to avoid leaning on?
27 chapters · progress saves automatically