AI Evaluation Checklist

Evaluation is how a probabilistic system is proven to meet its specification, and it is the discipline most AI teams under-invest in until production teaches them otherwise. This checklist covers what an adequate evaluation program contains. It is the operational form of the AI evaluation and testing whitepaper and complements llm evaluation explained and how to build an ai quality gate.

Who should use this checklist?

Engineering owners building or reviewing AI systems, domain owners responsible for the definition of correct, and governance functions requiring evidence before launch.

Does the specification support evaluation?

1. The spec defines correct behavior per task, including rules, prohibited actions, and escalation.
2. Every spec element generates test categories.
3. Quality thresholds are stated with measurement method and consequence rationale.
4. The domain owner has signed the spec.

Reference: the spec-driven development for AI whitepaper.

Is the golden dataset adequate?

1. Every category in the spec is represented with several examples.
2. Edge cases, boundary cases, and past failures are included.
3. Inputs are real where possible, anonymized as required; synthetic cases are labeled as such.
4. Labels were produced by domain experts with written guidelines and measured agreement.
5. Expected outputs or rubrics are attached per case.
6. The dataset is versioned with provenance, and a held-out set is reserved.
7. It is stratified so results report per category.

Reference: what is a golden dataset and ai training data checklist.

Are metrics matched to task types?

Task type	Metrics present?
Classification and routing	Accuracy, precision, recall, F1 per class
Structured extraction	Field-level exact match, schema validity, completeness
Retrieval	Recall at k, precision at k, MRR
Grounded generation	Groundedness, citation validity, answer correctness, refusal correctness
Free-text generation	Rubric scores with calibrated graders
Agent tasks	Task success, action correctness, unsafe-action rate, steps and cost
Safety	Injection success rate, leakage rate, policy-violation rate

Reference: how to build an agent evaluation harness.

Are graders trustworthy?

1. Deterministic checks are used wherever the task allows.
2. Model graders have explicit rubrics with score anchors.
3. Graders are calibrated against human labels with reported agreement.
4. Bias checks (position, verbosity, self-preference) have been run.
5. Graders are re-calibrated on grader model changes.

Reference: what is llm-as-a-judge.

Is human review structured?

1. Humans label the golden set and calibrate graders.
2. Stratified production samples are reviewed on a schedule weighted by consequence.
3. Review uses the same guidelines as labeling.
4. Disagreements feed rubric and dataset improvements.

Reference: how to build a human review queue.

Are safety tests included?

1. Prompt injection cases across every content channel.
2. Data leakage attempts.
3. Policy violation and prohibited-action cases.
4. Robustness cases: malformed inputs, adversarial formatting.
5. Red-team findings added to the suite.

Reference: the prompt injection defense checklist.

Does the suite gate releases?

1. The suite runs in CI on every change to prompts, models, retrieval, tools, or data.
2. Thresholds per category are set by consequence; breaches block release.
3. No-regression rules compare against production.
4. Provider model updates run the suite in staging before adoption.
5. Non-determinism is handled with pinned versions and pass rates.
6. Results and decisions are recorded in the registry.

Reference: how to build a ci-cd pipeline for machine learning.

Is production evaluated continuously?

1. Sampled automated scoring of live traffic with the same graders.
2. Scheduled human review of stratified samples.
3. Drift indicators on inputs, confidence, outputs, and quality.
4. User feedback captured and analyzed.
5. Production failures triaged and added to the golden set.

Reference: the AI observability whitepaper.

Is reporting useful?

1. Results are reported per category and per version, with confidence intervals where samples allow.
2. Safety results and trends are visible to governance.
3. Failing cases are linked from reports.
4. Business owners read category views; engineers read version diffs.

Reference: how to measure ai success.

Have the common mistakes been avoided?

1. Prompts were not tuned on the test set; a held-out set exists.
2. The dataset is not twenty hand-picked examples.
3. Overall averages are not the only reported figure.
4. Evaluation is not launch-only.
5. A feedback loop from production exists.

How should the checklist be applied?

Use it at three moments: when planning a new system, to size the evaluation work honestly; before launch, as the evidence review; and quarterly in operation, to confirm the suite still reflects the specification and production reality. Gaps in the golden dataset and the CI gate are the ones to close first.

What comes first in practice?

The golden dataset and the CI gate are the foundation; without them every other item is advisory. Then calibrate graders, add safety cases, and establish production sampling and per-category reporting.

How FISTA Solutions runs evaluation

FISTA Solutions builds the evaluation program before tuning a single prompt: spec-derived golden datasets labeled with domain experts, task-matched metrics, calibrated graders, safety cases in the same suite, CI gates with consequence-based thresholds, production sampling, and per-category reporting. The AI enablement practice delivers the shared evaluation service, AI agents are evaluated at trajectory level, and forward deployed engineers work with your domain experts on labels and thresholds. The record behind the approach is 150+ projects with 99.9% uptime.

To assess an evaluation program against this checklist, message FISTA on WhatsApp, or read how to evaluate an llm for the practical starting point.