ITSM AI Features: Deflection, Triage, and What to Measure

Service management AI features promise deflection and triage, both of which are easy to measure dishonestly. This guide covers measuring them properly, drawing on FISTA Solutions' AI enablement delivery work.

What should be measured?

Six measures, several of which vendors do not report.

Measure	What it tells you	Vendor default
Resolution rate	Genuine resolution	Rarely reported
Reopen rate	Whether it stuck	Rarely reported
Subsequent contact	Deferral versus resolution	Rarely reported
Triage accuracy	Routing correctness	Sometimes
Deflection rate	Conversations avoided	Always reported
Agent time saved	Assistance value	Sometimes

What is wrong with deflection?

It counts users giving up as a success.

A conversation that did not become a ticket may have been resolved, or the user may have abandoned the channel, found a workaround, or asked a colleague. Deflection does not distinguish these.

It is also the metric most prominently reported, which makes it the one that shapes decisions. Insist on resolution and reopen figures alongside it. See how to calculate AI ROI.

How should triage be tested?

Against your own routing history, including reassignments.

Run the feature on past tickets and compare its routing against where they finally landed rather than where they were first sent. Initial routing is frequently wrong, and matching it is not a success.

Measure per category, since accuracy varies substantially between common and unusual ticket types. See AI eval report template.

Why does knowledge quality decide it?

Because deflection means retrieving from your knowledge base.

Stale articles, contradictory instructions, and undocumented procedures all produce answers that are confidently wrong. A user following one creates a ticket with a worse starting position.

Audit the knowledge base before enabling deflection. This is the most reliable predictor of whether the feature helps or harms. See AI knowledge base quality checklist.

What about automated resolution?

It needs reversibility limits and a clear escalation path.

Features that act — resetting a password, provisioning access, restarting a service — are agents, with the permission and audit questions that implies. A wrong action on an access request is a security event.

Scope narrowly, enforce limits in code, and log every action. See agent permission review checklist.

Where is the underrated value?

Agent-facing assistance.

Drafting a response, summarising a long thread, and surfacing similar resolved tickets all save experienced agents real time with a human reviewing every output. The risk is low and the benefit is measurable.

That is frequently a better first deployment than user-facing deflection, and it is less often the headline feature. See the end of generic chatbots.

What governance applies?

The same as any AI system, plus access implications.

Service management tools hold sensitive operational and sometimes personal information, and AI features process it. Data handling terms, retention, and audit trail all need the usual assessment.

Automated access provisioning in particular needs security review before enabling. See AI third party risk checklist.

How do you run your own comparison?

Run triage against several hundred historical tickets and measure accuracy against final routing. Run deflection against real user questions and have agents judge whether the answers were correct.

Then measure reopen rate on anything the feature resolves. That number tells you whether deflection was resolution.

What does switching cost later?

Low technically. The cost is process adaptation if teams have reorganised around the features.

Measure benefit before allowing staffing decisions to depend on a deflection figure.

What do people get wrong here?

Accepting deflection as the primary metric. Triage tested against initial rather than final routing. Deflection enabled over a stale knowledge base. Automated actions without limits. And ignoring agent-facing assistance.

Should you reduce staffing on these figures?

Not on deflection alone. If the deflected contacts return as tickets later, or as escalations elsewhere, the reduction produces a backlog.

Measure total contact volume across all channels over several months before changing staffing. See how to staff an AI support rotation.

Which should you choose?

Measure resolution and reopen rates rather than deflection, test triage against final routing, and audit the knowledge base before enabling user-facing features. Agent-facing assistance is frequently the better first deployment.

What should you do first?

Measure the reopen rate on tickets your AI features resolve. If it is high, deflection was deferral.

How FISTA Solutions helps

FISTA Solutions builds and operates production AI systems through AI agents, AI enablement, and forward deployed engineering: deflection claims tested against reopen and subsequent contact rates, with the knowledge base audited before user-facing features are enabled, decisions documented with their reasoning, and handover that leaves your team able to maintain what was delivered. The record is 150+ projects for 50+ companies across 12+ countries.

To run this comparison against your own workload, message FISTA on WhatsApp, or read AI knowledge base quality checklist.