AI News · Adoption

What does an AI coding assistant deliver?
A business case without wishful numbers.

August 31, 2026 · approx. 12 min read

Anyone who pegs the value of a coding assistant at "30 percent faster" doesn't have a business case yet. They've copied someone else's number into a spreadsheet. A usable pilot measures time to a finished change, rework, and delivery capacity against your own backlog.

The question from leadership sounds reasonable: what do we get back for licenses, rollout, and training? Answering it from published studies is tempting. Unfortunately, the studies measure very different things. A short lab task is not the same as a change in a codebase that has grown for ten years. More pull requests is not the same as shorter lead time. Accepted lines of code say nothing yet about later rework.

Three studies, three answers

In the experiment by Peng and colleagues, the Copilot group completed a scoped JavaScript task 55.8 percent faster. The task was short, done solo, and automatically checkable. That's a clean test for that case, but not a picture of a normal sprint.

Cui and colleagues pooled three randomized field experiments at Microsoft, Accenture, and a Fortune 100 company. A total of 4,867 developers took part. With access to the assistant, 26.08 percent more tasks were completed. That measure is closer to everyday company work, but it leaves open how large the tasks were and how long-term quality and maintenance developed.

METR chose a different setup: 16 experienced open-source developers worked on 246 real issues in repositories they had known for years. With AI use permitted, they took 19 percent longer on average. Beforehand they had expected a 24 percent time saving; afterward they still believed they had saved 20 percent. The sample is small and specific. That is exactly what makes the finding useful: gut feeling does not work as a measurement.

None of the three numbers is the right number for your team. Together they say something more concrete: task, experience, repository, and measurement method all strongly change the outcome.

The math starts with fully loaded costs

The license price is usually the smallest and most visible line item. For a pilot, at least these costs belong in the calculation:

• Licenses and usage-based fees for models, agent runs, and additional infrastructure.
• Setup: policies, permissions, repository instructions, hooks, and approved integrations.
• Onboarding: training plus work time for exercises and questions.
• Usage: time for prompting, waiting, reviewing, and correcting.
• Rework: extra reviews, rejected changes, bug fixing, and later maintenance.

A worked example for eight developers

Assume a four-week pilot costs €40 per license per month. Add a training day for €1,900 and eight internal hours for setup and coordination. We set the notional developer hour at €90.

The pilot therefore needs to save roughly 33 hours of usable work time, about 4.1 hours per developer. This calculation does not yet include extra review effort or later bug-fixing costs. Both need to be added from the pilot data. For a continuation into a second month, the one-time rollout drops out of the calculation, so the break-even is markedly lower.

What the pilot should measure

A single metric does not do justice to the workflow. The SPACE framework sorts developer productivity into five areas: satisfaction, performance, activity, communication and collaboration, and efficiency and flow. For a small pilot, a selection across three levels is enough.

1. Time to a finished change

• active work time from start to a reviewable pull request
• wait time for review and time spent on correction loops
• total lead time to merge or deployment

The DORA metrics extend the single task with delivery capacity: change lead time, deployment frequency, change failure rate, rework rate, and time to recover from failed deployments. DORA itself warns against turning a single metric into a target or comparing teams by raw values.

2. Quality and rework

• share of changes that pass CI and functional acceptance on the first try
• review comments and correction loops, weighted by severity rather than just count
• rolled-back changes, production bugs, and rework within 14 or 30 days
• security findings and violations of repository conventions

The joint guidance paper from BSI and ANSSI names insecure or non-functional code, dependencies, and handling of confidential data as risks, among others. The pilot therefore needs the same checks as the rest of the code. A high suggestion-acceptance rate does not replace them.

3. Working style and workload

• For which task types was the assistant used, or deliberately switched off?
• How much time went into prompting, waiting, reading, and correcting?
• Has cognitive load changed?
• Were context switches and blocked work less frequent?

These figures are subjective but not worthless. They explain the measurements. If a task gets done faster but developers can barely keep track of the generated code, the time measurement alone has missed a problem.

A four-week pilot

1. Week 0: capture the baseline. Choose two to four recurring task types. For comparable completed tasks, record time, review, rework, and result quality. Define allowed data, tools, and repositories.
2. Week 1: get the team ready. Train on usage, permissions, and review. Write down shared repository instructions. The team logs per task whether and for what the assistant was used.
3. Weeks 2 and 3: measure comparable work. Split similar tasks, where possible, between work with and without the assistant. Rotate the assignment so that not just certain people or especially easy tickets end up in one group.
4. Week 4: calculate and decide. Compare task types separately. Factor in extra review and rework time. Also document cases where the assistant was switched off.

A perfect randomized experiment is rarely possible day to day. A before-and-after comparison with no controls at all is still too weak: sprint content, experience, and time pressure change the outcome. Even a rotating assignment of similar tasks beats an end-of-month survey.

Set the expansion path before you start

A pilot without agreed thresholds set in advance often ends in a debate about individual success stories. So write down beforehand which next expansion step belongs to which result.

The thresholds have to fit the company. A team under heavy regulatory load weighs an extra security finding differently than an internal prototyping team. The task types stay separate too: test cases often need a reusable skill, changes to an old core component more often need good repository rules and access to existing documentation.

What number I plan with before a pilot

For planning, I budget 8 to 15 percent less time on suitable tasks for a development team. That is not a study figure and not a promise. It is my working range for a properly set-up agent operation, where training, repository rules, hooks, skills, and needed MCP connections come together.

The range applies only to work time where the assistant can be used meaningfully. Coordination, domain decisions, architecture work, and many reviews don't disappear because of it. For clearly scoped changes, tests, documentation, migrations, and recurring debugging, the share is usually higher than for new or poorly understood components.

What this means for eight developers

Assume each of the eight developers spends 20 hours a week on tasks for which the assistant is a candidate. Over four weeks that's 640 hours. My planning range yields the following expectation:

In the middle scenario, the team earns back the cost of training, setup, and the first license month within the first month. From the second month on, this example is left mainly with licenses, upkeep, and time for new rules. At the same work volume, the monthly gross value would stay at roughly €6,900.

What ten days together change

A training day brings the team to a shared baseline and delivers the first rules. That leaves little implementation time for hooks, skills, and MCP connections. A program over ten days can carry that work through into operations.

Ten days of instruction with the whole team would be expensive. At €90 an hour, eight developers would tie up 640 hours in total, and that time alone would carry a notional value of €57,600. A better approach is a program where the whole team works together for two days. On the remaining days, the building blocks for the repository take shape with individual developers or independently.

The math for the ten-day program

For a program like this, I use a project price of €16,900 in the example. Two shared team days tie up another 128 hours for eight developers, or €11,520 in internal work time. Together with €320 for the first license month, the initial investment comes to €28,740. The company would still need to add the occasional involvement of individual developers on the implementation days.

A single training day aims at getting started. The ten-day program aims at a way of working that stays in the repository: shared rules, automatic checks, reusable skills, and access to the systems the team needs every day. The expected time gain comes from this combination.

Hooks, skills, MCP, and repository rules don't simply add up to four separate percentages. They reinforce each other. The hook catches an error early, the repository rule prevents it on the next task, and a skill makes the same workflow repeatable. What gets measured, then, is the combined effect on finished, reviewed work.