AI Vendor Due Diligence: A Whitepaper for Buyers

Buying AI differs from buying software. The system is probabilistic, its quality depends on the vendor's method as much as its technology, and the data it touches is often sensitive. Yet most AI procurement still runs on demos and decks. This whitepaper gives buyers a due diligence framework for AI development partners and AI product vendors, organized around the evidence that actually predicts outcomes.

Why does AI vendor diligence differ from software diligence?

Three reasons. Quality is not inspectable: you cannot read a model's logic, so you must evaluate how the vendor defines and measures correctness. Method determines outcomes: two vendors with the same tools produce wildly different results depending on whether they specify, evaluate, and verify. Data exposure is inherent: AI systems consume enterprise data, so data handling is central. The general vendor landscape is in how to evaluate AI vendors and the AI vendor comparison framework.

What are the seven diligence areas?

Area	Core question	Evidence to request
Capability	Have they done comparable work in production?	Case descriptions with measured outcomes; engineer profiles
Method	How do they define and prove correctness?	A specification and evaluation report from a past project
Security and data	How is our data protected, technically?	Architecture, access policies, data-flow diagrams, audit reports
IP and commercial	Who owns what, and how do we exit?	Contract terms on IP, data use, exit, and transition
Operations	Can they run and support it?	Monitoring approach, support model, incident process
Stability	Will they exist and staff us through the engagement?	Company facts, tenure, team depth
References	Did outcomes materialize for others?	Reference calls focused on results and problem handling

How do you evaluate capability?

Capability is evidenced by production outcomes, not demos. Request:

• Descriptions of comparable projects: problem, approach, what shipped, measured results, and what went wrong.
• The specific engineers who would work on your engagement, with their production history. A brand's reputation does not ship code; people do.
• Technical depth conversations with those engineers, not sales engineers.
• For AI products: evaluation results on tasks like yours, and the opportunity to test on your data.

Be cautious of vendors who cannot name engineers, who present only logos, or who describe outcomes without measures. See what to look for in an AI partner.

How do you evaluate method?

Method is the strongest predictor of AI outcomes. The diagnostic question is: how do you define and measure correctness for a system like ours? A strong answer describes specifications with acceptance criteria, golden datasets built with domain experts, automated and human evaluation, regression gates, and production monitoring. A weak answer describes prompt tuning and user feedback. Ask to see:

• A specification from a comparable project, redacted as needed.
• An evaluation report showing metrics by category and how thresholds were set.
• Their approach to shadow deployment and autonomy graduation for systems that act.
• How they handle model and provider changes after launch.

The method FISTA applies is described in the spec-driven development for AI whitepaper and the AI evaluation and testing whitepaper.

How do you evaluate security and data handling?

Certificates are a starting point, not a conclusion. Evaluate technically:

1. Data flows: where does our data go, including to model providers and subprocessors? Under what terms?
2. Access: how do engineers access our systems and data? SSO, MFA, least privilege, time-bound grants, device policy?
3. Asset ownership: will code, infrastructure, and data live in our accounts?
4. Secrets and logging: how are credentials managed and what is logged, with what redaction?
5. Model-provider terms: training-data use, retention, residency.
6. AI-specific threats: how do they defend against prompt injection and data leakage? Ask for their test approach.
7. Incident history and process.

Ask for architecture and data-flow diagrams and, where available, third-party audit reports. Guidance is in SOC 2 for AI vendors, AI vendor security questionnaire, and data security offshore AI.

What IP and commercial terms matter?

Term	What to require
IP assignment	All custom work product assigned to the buyer; pre-existing vendor IP identified and licensed clearly
Data use	Vendor and its providers may not train on buyer data; retention and deletion defined
Asset location	Buyer-owned repositories and infrastructure from day one
Exit and transition	Documentation, runbooks, and transition assistance on termination
Change notification	Advance notice of material changes to models, subprocessors, or terms
Pricing structure	Drivers explained; no guarantees of outcomes or timelines before scoping
Liability and indemnity	Proportionate to the engagement and data sensitivity
Governing law	Buyer's jurisdiction where feasible

For AI products, add lock-in analysis: can you export data, prompts, configurations, and evaluation sets? Guidance is in how to negotiate an AI development contract and the outsourcing contract checklist.

How do you evaluate operational maturity?

A vendor that can build but not operate leaves you with a fragile system. Evaluate:

• Their monitoring and observability approach for AI systems: quality, cost, drift, not just uptime.
• Support model: response times, escalation, who answers.
• Incident process: how AI-specific incidents are contained and root-caused.
• Change management: how prompt, model, and retrieval changes are evaluated and released.
• Handoff practice: documentation, runbooks, training, and transition periods.

Reference designs are in the AI observability whitepaper and the AI project handoff checklist.

How do you assess stability?

Confirm the company's legal identity and jurisdiction, founding date, size, leadership tenure, and team depth in the skills you need. Ask how continuity is handled if an engineer leaves. For offshore or cross-border vendors, confirm the contracting entity and governing law; see the cross-border engineering delivery model whitepaper. Treat claims you cannot verify, such as unnamed marquee clients, as unverified.

How do you run references well?

References are most useful when focused on outcomes and problem handling rather than satisfaction:

• What was the measured result, and how was it measured?
• What went wrong, and how did the vendor respond?
• Were the engineers who started the engagement the ones who finished it?
• How was quality verified before launch?
• What did handoff look like, and can you operate the system without them?
• Would you engage them again for a harder problem?

Ask for references from comparable projects and, where possible, from engagements that had difficulties.

What are the red flags?

• Demos without production references.
• No clear answer to how correctness is defined and measured.
• Guarantees of outcomes, timelines, or savings before scoping.
• Reluctance to name engineers or let you speak with them.
• Vendor-owned repositories or infrastructure.
• Vague data-use terms or providers without appropriate agreements.
• Pressure to skip discovery or specification.
• Unverifiable client claims.

Expanded in red flags outsourcing AI.

How should a paid discovery or pilot be structured?

A scoped, paid discovery or pilot is often the most informative diligence step. Structure it with a defined problem, acceptance criteria, an evaluation approach, IP assignment for all outputs, buyer-owned assets, a fixed duration, and explicit exit criteria. The deliverable should include a specification and an evaluation report, so the buyer gains value regardless of whether the relationship continues. Guidance is in how to structure an AI pilot agreement and how to run an AI pilot.

How should the evaluation be scored?

Score each area on evidence quality, weight by your risk profile, and require minimum thresholds on method, security, and IP regardless of price. A vendor strong on price and weak on method is the most expensive option once rework and incidents are counted. A structured approach is in ai vendor evaluation checklist and how to choose an outsourcing partner.

What does a diligence timeline look like?

A proportionate process for a significant AI engagement runs in four steps. First, a written request covering the seven areas, answered with artifacts rather than prose: case descriptions with measures, a redacted specification and evaluation report, architecture and data-flow diagrams, security policies, standard contract terms, and company facts. Second, technical sessions with the named engineers or product team, including a walkthrough of how they would approach your problem and how they would define and measure correctness for it. Third, reference calls focused on outcomes and problem handling, and verification of company facts and jurisdiction. Fourth, where the engagement warrants it, a scoped paid discovery or pilot with acceptance criteria and IP assignment. Scoring across the areas, with minimum thresholds on method, security, and IP, produces a decision that can be defended to leadership and revisited if circumstances change.

How should due diligence findings be used after signing?

Findings become the monitoring plan: the gaps a vendor promised to close get dates and evidence requirements, the risks accepted get owners and review cadences, and the commitments captured in the contract get verification points at renewal. Diligence that ends at signature has done half its job. Teams that carry findings into vendor management catch drift in security posture, model behavior, and support quality before it becomes an incident, and they enter renewal negotiations with a documented record rather than a memory.

How FISTA Solutions approaches buyer diligence

FISTA Solutions expects diligence and is built to pass it: a US-registered entity in Delaware with engineering in Faisalabad, named engineers on every engagement, specification and evaluation artifacts from comparable work, buyer-owned repositories and infrastructure, explicit IP assignment and data terms, and references on measured outcomes. Our verified record is 150+ projects for 50+ companies across 12+ countries with 99.9% uptime and 47% average efficiency gains, and we do not claim what we cannot verify. Engagements run through forward deployed engineers, AI enablement, AI agents, and staff augmentation.

To put FISTA through your diligence process, message us on WhatsApp, or start with questions to ask an AI development company.