AI Bias Testing Checklist: Finding Disparate Outcomes

Bias testing works when it measures outcomes rather than inspecting intentions. This checklist covers doing it concretely, drawn from FISTA Solutions' AI enablement governance work. This is general guidance, not legal advice.

What is being measured?

Six measurements, each answering a different question.

Measurement	Question it answers
Outcome rate by group	Do groups get different results?
Error rate by group	Is it wrong more often for some?
False positive and negative split	Which kind of error, for whom?
Proxy correlation	Do inputs stand in for attributes?
Subgroup intersections	Are combined groups worse off?
Drift over time	Is disparity emerging?

Scope and definitions

Decide what you are measuring before you measure it.

• Groups relevant to this use case identified
• Legal protected characteristics for your jurisdictions listed
• Fairness definition chosen and justified in writing
• Acceptable disparity threshold agreed with the business owner
• Legitimate basis for any expected difference documented
• Intersectional subgroups identified
• Legal input obtained on the approach

Data for testing

You cannot measure disparity without group data, which is itself a sensitive question.

• Test data representative of the actual population
• Group membership available for testing, lawfully obtained
• Sample sizes sufficient per group for meaningful measurement
• Small groups handled without drawing invalid conclusions
• Historical data checked for encoded past bias
• Data collection for fairness testing documented and lawful
• Access to group data restricted appropriately

Outcome measurement

The core of the test.

• Outcome rates computed per group
• Disparities compared against the agreed threshold
• Statistical significance assessed, not just raw difference
• Intersectional subgroups measured separately
• Results reviewed by someone outside the build team
• Differences that appear justified documented with the justification
• Findings recorded whether or not they are actionable

Error analysis

Equal outcomes with unequal accuracy is still a problem.

• Error rates computed per group
• False positives and false negatives separated
• The consequence of each error type per group assessed
• Confidence calibration checked per group
• Cases the system refuses analysed per group
• Escalation rates compared across groups
• Human override patterns analysed for their own disparity

Proxies and inputs

Removing the attribute does not remove the correlation.

• Input features tested for correlation with protected attributes
• Geographic and postal features examined specifically
• Name, language, and education features examined
• Retrieved content checked for skewed representation
• Training or example data checked for skew
• Proxy effects measured rather than assumed absent
• Feature removal tested for whether it actually reduces disparity

Production monitoring

Disparity can emerge after launch.

• Outcome rates by group monitored in production
• Alert thresholds set on disparity, not only on volume
• Input distribution shift monitored
• Complaints analysed for patterns by group
• Periodic re-testing scheduled
• Findings routed to someone who can act
• Re-test triggered by any model or prompt change

What are the most common failures?

Testing intentions rather than outcomes. Assuming attribute removal is sufficient. Measuring outcome rates without error rates. Testing once before launch. And no threshold agreed, so findings have no decision attached.

Who should own this?

The business owner of the system is accountable for the outcomes; a function independent of the build team should run or review the testing. Self-assessment by the builders is weaker evidence.

How often should it run?

Before launch, on every material model or logic change, and continuously in production for systems making decisions about individuals. Quarterly formal review at minimum.

What evidence should it produce?

Dated test results per group, the fairness definition and threshold with their justification, findings and actions taken, and production monitoring records.

What if you find disparity you cannot remove?

Document it, assess whether the system should still operate, and consider compensating controls — human review for affected groups, a higher escalation rate, or narrowing the system's scope.

Suppressing an inconvenient finding is the worst available response. Recorded findings with a reasoned decision are defensible; undocumented knowledge of a problem is not. This is general guidance, not legal advice.

What should you do first?

Measure outcome rates by group for one deployed decision system. If you cannot, obtaining the data to do so is the first task.

How FISTA Solutions helps

FISTA Solutions builds and operates production AI systems through AI agents, AI enablement, and forward deployed engineering: disparity measured on outcomes and error rates across groups with an agreed threshold, monitored in production rather than tested once before launch, decisions documented with their reasoning, and handover that leaves your team able to maintain what was delivered. The record is 150+ projects for 50+ companies across 12+ countries.

To adapt this checklist to your environment, message FISTA on WhatsApp, or read AI explainability checklist.