Say a director of engineering sits down for her quarterly review with the tool vendor's dashboard open. Suggestions accepted are climbing. Her latest pulse survey says nearly everyone feels faster. Nothing on the screen tells her what her engineers did with the hours they stopped spending on typing, and she has not thought to ask, because every number she can see looks like good news.

Two researchers at the University of Auckland, Annie Vella and Kelly Blincoe, did ask. They surveyed professional software engineers who were already using AI assistants, once in October 2024 and again in April 2025, with 158 eligible respondents the first time, 101 the second, and 95 who answered both rounds.1 It is a preprint that has not yet been peer reviewed, the answers are what participants reported about themselves, and the participants came through convenience and referral chains, with men making up 85% of the sample.1 Because only current users took part, the authors caution that their productivity figure describes continuing users, and the data predates the newer agentic tools.1 That makes it a careful early look at how the work feels from inside, and I would not stretch it into a measurement of output. Read that way, it is still a useful early look at what happens after the tools arrive.

Where the effort went

By the second survey, 82% reported spending less time writing code.1 Across the matched group, the balance of their reported effort tilted toward the tasks the authors group as verification, which are reviewing code, testing and debugging, a tilt the authors call modest. Neither creating tasks nor verifying tasks changed significantly when examined alone; the shift showed in the balance between them.1 The authors propose a name for the work their standard categories miss, supervisory engineering work, which they describe as "the effort required to direct AI, evaluate its output, and correct its errors," and they say further research is needed to confirm it is truly distinct.1

My read, which goes beyond what the study measured, is that in most organizations this work has no owner. I suspect few have staffed for it or written it into how people are promoted, and that it simply arrived, one reviewed suggestion at a time, inside roles whose descriptions still say build.

What the productivity question misses

In each round, 84% of respondents said their productivity had improved, and among those who answered both, 77% gave exactly the same rating six months later.1 Over those six months, though, the share of matched participants who rated at least one of three dimensions of developer experience as worse nearly doubled, from 14% to 27%. Feedback loops improved significantly, while flow state and cognitive load declined, though neither decline was statistically significant on its own.1 Maintainability as a primary concern went from 3% to 19%, the only concern whose ranking changed significantly.1

The time saved on writing appears to come back as checking, and a question about speed only captures the first half.

A productivity question that asks whether people feel faster may keep returning yes, since the part it asks about is the part that got easier. It is not built to report that the remaining work is harder to stay absorbed in, which is where this study saw the clearest strain: among those reporting a worse experience, flow state was the main issue for 54% in the first round and 76% in the second.1 Maintainability climbing the list of worries is the one I would watch, since it points at a cost that shows up later and somewhere else, in a quarter when nobody connects it to the tool.

None of this shows that output fell, and the study does not claim it. Engineers may be shipping more and feeling worse about the way they do it, which is a plausible combination and a hard one to catch, because the only people positioned to report it are the ones living inside it. Perception data from a small group deserves a larger study before anyone builds a staffing model on it. What it deserves now is a question at the next review.

Staffing the reviewer's seat

There is also an ordinary reason to expect the checking to be demanding. Reading a change you did not write, and deciding whether it will hold up in a system you partly remember, takes attention of a different kind from producing the change yourself. A person typing a function holds its intent in mind the whole time. A person reviewing it has to rebuild that intent from the page. If the volume of changes goes up while the reviewers stay the same people, the arithmetic of a working day gets tight in a place most tool reports do not look. That is my inference from the pattern the authors describe, and the study does not test it directly.

If checking is now a larger share of the job, then who does it and whether anyone gets credit for it are management questions. They do not belong to the individual engineer's good judgment. I would ask for the review queue to be visible, with its length and age, and I would look at whether the people clearing it are the same people the performance system rewards for shipping. An earlier piece on a bank's mobile app described what happens when freed capacity gets no instructions; the extra capacity went into the same routine work until roles and criteria were redefined. A related piece on a field experiment that gave patent lawyers an AI drafting tool and then took it away found the lasting benefit went to those with deeper expertise, and good checking depends on that same expertise.

Those who write the performance criteria are choosing, by default or on purpose, whether supervisory work gets done well. If the check is the larger share of the job now, who in your organization is being evaluated and promoted for doing it?