How to Measure AI Adoption Beyond Licence Counts

Licence counts measure procurement. Adoption is whether people's work changed, and measuring it needs different numbers. This playbook covers what to measure, drawing on FISTA Solutions' AI enablement work.

When is this worth doing?

After any enablement programme or tool rollout, and periodically thereafter. The second measurement matters more than the first, because decay is invisible in a single snapshot.

It is also worth doing before a renewal, when licence spend is being justified and the honest number is more useful than the procurement one.

What does the sequence look like?

Step	Purpose
1. Define what use means	Repeated, not first login
2. Measure active use	Weekly, sustained
3. Measure depth	Substantial work, not trivia
4. Segment by task and team	Averages hide everything
5. Measure the work outcome	The point of the exercise
6. Watch for decay	It is the usual pattern

Step 1 — Define what use actually means

Repeated use over a period rather than a first login, and use for work rather than for experimentation.

A single session is curiosity. Use in three consecutive weeks is a changed working practice, and that distinction is the difference between an adoption number that means something and one that flatters.

Write the definition down before pulling the data. Definitions chosen after seeing the numbers select for the flattering one.

Step 2 — Measure active use, not cumulative users

Weekly active users as a proportion of the eligible population, tracked over months.

Cumulative user counts only rise, which makes them useless for detecting decline. Weekly active use rises, peaks, and frequently falls, and that shape is the actual story.

Compare against the eligible population rather than against licences issued. Those diverge and the second is the more flattering denominator.

Step 3 — Measure depth of use

Whether people use the tool for substantial parts of their work or only for peripheral tasks.

Broad shallow adoption — everyone uses it occasionally for something minor — produces very little and looks good on a usage chart. Narrow deep adoption in a team that has restructured its work around it produces most of the value.

Proxies for depth include session length, tasks per user per week, and whether use concentrates in the tasks the enablement targeted.

Step 4 — Segment by task and team

Aggregate adoption numbers conceal that one team has changed how it works and four have not.

Segmenting shows where it is working and where it is not, which is the information that tells you what to do next. It also surfaces the teams worth learning from, who are usually not the ones reporting enthusiasm.

Segment by task too. Adoption for document drafting and adoption for data analysis have different determinants and different interventions.

Step 5 — Measure the work outcome

Time per task, volume handled, quality, or cycle time for the specific tasks the adoption was meant to change.

This is the number that justifies the investment, and it is the one most adoption reporting omits because it is harder to obtain. Tool usage is an input; the work changing is the result.

Use the baseline from before the rollout if one exists. If it does not, that is a lesson for the next programme and a reason to establish one now.

Step 6 — Watch for decay

Adoption typically peaks a few weeks after enablement and declines, and the decline is the most actionable signal available.

Organisations measuring cumulative users never see it. Those measuring weekly active use see it clearly and can ask why: the tool was worse than expected, the sanctioned path is slow, the champion left, or the work it helped with was not the work people actually do.

Each of those has a different response, which is why detecting the decline matters more than reporting the peak.

What about shadow usage?

Measure it too, as far as you can. Network logs and expense records show people using unsanctioned tools, and that is adoption of a kind.

High shadow usage alongside low sanctioned usage is the clearest possible signal that the sanctioned path is worse. Treating it as a policy violation rather than a product finding misses the information. See what is shadow ai.

How do you measure adoption of an embedded feature?

Differently, because there is no tool to log into. Measure the proportion of relevant interactions where the AI feature was used, accepted, edited, or dismissed.

Acceptance and editing rates are the depth measure here: a feature whose suggestions are always heavily edited is being tolerated rather than adopted, and the usage count will not show that.

Who needs to be involved?

Someone who can pull the usage data, someone who owns the enablement programme, and the team managers whose people are the population.

Managers are needed for the outcome measurement, because the work data usually lives with them rather than in the tool.

How long does it take?

A week to establish the measures, then continuous. The first meaningful reading is six to eight weeks after enablement, once the novelty has passed.

What are the common failure modes?

Counting licences. Cumulative user totals. Ignoring depth. Aggregate numbers only. No outcome measure. And reporting the peak without watching for decay.

How do you know it worked?

Weekly active use stable or rising months after enablement, depth increasing, the targeted work measurably changed, and shadow usage falling.

What does it cost?

Mostly people's time rather than tooling. The expensive version is the one that stalls halfway and leaves the organisation with neither the old state nor the new one, which is why a narrow first pass beats a comprehensive plan nobody finishes.

Budget the work as an operated change rather than a project with an end date, because most of these need a maintenance tail. See AI total cost of ownership.

What should you do first?

Pull weekly active use as a proportion of the eligible population for the last three months. The shape of that line is usually more informative than any survey.

What should you do with the numbers?

Use them to decide where to intervene rather than to report status. Low adoption in a team with a champion and good tooling is a different problem from low adoption where the sanctioned path is slow, and the numbers alone do not distinguish them — the conversation with that team does.

Report the numbers honestly upward, including the decline. Adoption reporting that only shows growth trains everyone to stop believing it, and the first honest number after that is read as a crisis rather than as information.

How FISTA Solutions helps

FISTA Solutions runs this work alongside client teams rather than around them: adoption measured as sustained repeated use and depth rather than licence coverage, with the work outcome tracked alongside the tool usage, evidence produced as the work proceeds, and handover that leaves your people able to continue without us. Delivery runs through AI agents, AI enablement, and forward deployed engineers. The record is 150+ projects for 50+ companies across 12+ countries, with 47% average efficiency gains where measured.

To run this with support, message FISTA on WhatsApp, or read how to run an AI training program.