A convincing demonstration is a hypothesis
A draft that once took an hour appears in seconds. It is tempting to treat the difference as a productivity gain. Sometimes it is. But the work may still require checking, revision, approval, and coordination before the result becomes useful. A demonstration proves that output can be generated quickly; it does not settle the value of the complete workflow.
Our argument is that an AI productivity claim should begin with an outcome and a boundary. What counts as completed work? Which activities are included? Whose time is being saved, and whose time is being added? Those questions are more informative than counting prompts or drafts.
This is especially relevant when the person generating the output is different from the person reviewing it. A tool may save the author time while increasing the burden on a specialist downstream. The organization needs to see both effects.
What one influential study showed
The 2023 working-paper version of “Generative AI at Work” studied an AI assistant in customer support and reported an average productivity increase of about 14 percent, measured in issues resolved per hour. The gains were larger for less experienced and lower-skilled workers in that setting. [1]
That is evidence about a particular intervention, workforce, and measurement approach. It is not a general forecast for every occupation or every AI product. We refer specifically to the working-paper result, rather than blending estimates from different versions of the research.
The lesson we draw is to look for variation. An average improvement can coexist with smaller gains for experienced workers or certain task types. A useful adoption decision asks where the tool helps and why, rather than applying one percentage to an entire organization.
Count the whole task
Consider a hypothetical team preparing a monthly market briefing. A model speeds up drafting, but the team must still verify facts, resolve conflicting sources, and decide which findings matter. Measuring only typing time misses those activities. Measuring only the number of pages rewards volume even when readers would benefit from a shorter report.
Define an acceptable briefing before running the experiment: relevant questions answered, major claims supported, important uncertainty disclosed, and delivery on time. Then record total effort across the people involved. Include the time spent learning the tool, correcting output, and checking whether the result meets that standard.
It may also be useful to record the spread of outcomes. A tool that is usually fast but occasionally creates a large correction task can have a different operational impact from one that produces smaller, steadier gains. The appropriate choice depends on the workflow’s tolerance for that variation.
Choose a credible comparison
Our suggested pilot compares similar work with and without the tool while keeping the quality standard consistent. When practical, assign tasks in a way that reduces the chance that the easiest work all goes to one condition. Document differences in task difficulty and experience that could affect the result.
Avoid judging the pilot only through enthusiastic volunteers or the best examples. Capture ordinary attempts and failures. Ask reviewers to evaluate the outputs without being told which method produced them when that is feasible. These steps improve the comparison without pretending that a small internal pilot is a definitive scientific study.
Measure adoption friction too. A tool that requires constant copying between systems may create work that disappears from a narrow “time to first answer” metric. Conversely, a modest drafting improvement can be valuable if it fits cleanly into the process people already use.
Follow the saved time
Time saved does not automatically become additional useful output. The team may face another bottleneck, such as approvals, missing data, or limited demand. Decide what the organization hopes to do with the additional capacity: improve quality, respond faster, reduce a backlog, or give people more time for judgment.
Watch for new habits as well. If generating another draft is cheap, teams may produce more drafts than anyone needs. If summaries are easy, people may stop checking the underlying material even when a consequential decision requires it. These are design and management questions, not inevitable outcomes.
A credible productivity report should say what improved, who benefited, what remained difficult, and how the result was measured. The best evidence of value is more acceptable work with an honest accounting of total effort. The demo is the start of that investigation.
Sources & further reading
- Brynjolfsson, Li & Raymond — Generative AI at Work
2023 NBER working-paper result; approximately 14% improvement in issues resolved per hour in the studied support setting. - NBER — Measuring the Productivity Impact of Generative AI
June 2023 research digest explaining the working-paper findings and variation across workers.
Sources checked 5 October 2026. Links support the specific research claims identified in the article; recommendations and examples are ReasoningReport analysis.
