The easiest AI numbers to collect are the least useful. Lines suggested, suggestions accepted, pull requests opened: they all go up the week you roll out a coding assistant, and none of them tells you whether your team ships better software.
Why output numbers mislead
AI makes writing code cheaper. It does not make reviewing, testing or deploying it any cheaper. So the first thing many teams see is more pull requests waiting longer for review, and more changes that need a second fix after they merge.
Google's DORA research on AI-assisted development keeps returning to the same idea: AI amplifies the system it lands in. A team with fast review, good tests and small changes gets faster. A team without them gets a bigger queue.
Four numbers you already have
You do not need a new dashboard. Your Git host and incident log already hold what matters:
- Time to first review and time to merge. If AI is working, changes should flow through faster, not pile up in review.
- Rework. Changes reverted or fixed again within two weeks of merging. This is where "almost right" code shows up.
- Change failure rate. Deploys that cause an incident or need a rollback.
- A two-question pulse, once a month. "Do AI tools save you time?" and "Do you trust what you merge?" The second answer often moves before the other numbers do.
Take a baseline first
Measure four to six weeks before the rollout, or before a policy change, and compare the same team afterwards. Without a baseline every number is a story. Comparing your team with another company's published figures tells you even less.
Give it a quarter. The first weeks include learning time, and the numbers that matter, rework and failures, show up late.
What not to measure
Do not rank people by AI usage, and keep these numbers out of individual performance reviews. The moment people are measured on how much they use a tool, they will use it more, and you will learn nothing. Measure the team's delivery and let people choose how they get there.
Fix what it exposes
If review time climbs, the answer is smaller pull requests and review rules by risk, not less AI. If rework climbs, the answer is better checking: tests written from the requirement, not from the code. Most of the gains come from fixing the bottleneck the tool made visible.
For the full method, including commands to pull these numbers, how DORA metrics shift with AI and a 90-day rollout plan, see the free guide to measuring AI impact. The one-hour team session gets your team checking AI-written code and agreeing its first rules before you start counting.