Playbook · 6 minute read
How to Transition an AI System From Vendor to In-House
Transitioning an AI system in-house works when you acquire the evaluation set and decision records alongside the code, build the receiving capability before the handover, run a parallel period, and accept that some components will need rebuilding rather than transferring.
Bringing an AI system in-house fails most often because the transfer covered code and not understanding. This playbook covers acquiring what matters and building the capability to receive it, drawing on FISTA Solutions' forward deployed engineers work.
When is this worth doing?
When a vendor relationship is ending, when the capability has become core enough to own, or when the cost or responsiveness of the arrangement no longer works.
It is not worth doing for a system that is stable, cheap to run, and not strategically important. Insourcing a working commodity system converts a supplier relationship into a maintenance obligation.
What does the sequence look like?
| Step | Purpose |
|---|---|
| 1. Inventory what exists | Code, data, artefacts, knowledge |
| 2. Negotiate the transfer scope | Before the relationship ends |
| 3. Build the receiving team | Before the handover, not after |
| 4. Transfer knowledge actively | Working sessions, not documents |
| 5. Run in parallel | With the vendor still available |
| 6. Rebuild what did not transfer | Expect some of it |
Step 1 — Inventory what exists
Code, prompts, evaluation sets, training and fine-tuning data, decision records, runbooks, monitoring configuration, incident history, and the knowledge in people's heads.
The last is the largest and the most commonly overlooked. A vendor's engineers know why a threshold is set where it is and which inputs cause trouble, and none of that is written down.
Build the inventory before negotiating, so the transfer scope covers what you actually need rather than what the contract happened to mention.
Step 2 — Negotiate the transfer scope early
Agree what is being handed over while the relationship is still good, ideally before the notice period.
Evaluation sets, decision records, and data provenance are frequently not covered by a standard exit clause, and a vendor with no ongoing interest is considerably less helpful than one still under contract.
Include a defined period of vendor availability after the transfer, with a response expectation. That is the cheapest insurance available in this process. See how to review an ai contract.
Step 3 — Build the receiving team first
Have the people in place before the handover, with time to work alongside the vendor.
Teams assembled after the transfer learn the system from code with nobody to ask. That is slower, produces a worse understanding, and frequently ends with the team afraid to change anything.
The receiving team should include someone who will own it in production, not only people who will maintain it. Ownership is what determines whether the system stays healthy.
Step 4 — Transfer knowledge actively
Working sessions where the receiving team changes something, debugs something, and handles an incident with the vendor present.
Document-based handover transfers what the vendor thought to write down, which is a fraction of what matters. Doing the work while help is available transfers the rest.
Structure it: each session covers a real task, and the receiving team drives while the vendor advises. Sessions where the vendor demonstrates and the team watches transfer very little.
Step 5 — Run a parallel period
The receiving team operates the system while the vendor remains available, for weeks rather than days.
This is where the gaps surface: the monitoring nobody explained, the deployment step that only works from the vendor's environment, the incident type nobody mentioned.
A clean cutover moves all of that discovery into production with no help available, which is how transitions produce outages that get attributed to the decision rather than to the process.
Step 6 — Rebuild what did not transfer
Expect some components to need rebuilding: vendor-specific infrastructure, monitoring tied to their platform, deployment tooling, and anything depending on their internal services.
Budget for it explicitly rather than discovering it. Rebuilding is frequently cheaper than adapting, and knowing which is which requires looking before the transfer rather than after.
Prioritise monitoring and deployment. A system you cannot observe or deploy is not one you have taken ownership of, whatever the contract says.
What if the vendor is uncooperative?
Then the transfer is harder and still possible, and the priority changes to extracting artefacts over knowledge.
Get the code, data, evaluation sets, and documentation as early and as completely as possible, because access typically ends abruptly. Then plan for a longer period of rebuilding understanding from the artefacts.
This is the scenario that exit clauses exist for, which is an argument for negotiating them at signing rather than at termination.
What should stay with a vendor?
Commodity capability that is not differentiating and is cheaper to buy than to operate.
Insourcing everything is as much a mistake as outsourcing everything. The question is whether the capability is strategic and whether you will invest in maintaining it, and honest answers frequently favour continuing to buy.
What should always come in-house is the understanding: evaluation sets, decision records, and enough knowledge to change suppliers.
Who needs to be involved?
The receiving team including a production owner, someone managing the vendor relationship, and a technical lead who will be accountable afterwards.
Transitions managed by procurement without technical ownership acquire assets and not capability.
How long does it take?
Two to four months for a system of moderate complexity, including team building, active transfer, and a parallel period. Vendor estimates for handover are usually optimistic.
What are the common failure modes?
Acquiring code without evaluation sets. Building the team after the handover. Document-based transfer. Clean cutover. Discovering vendor-specific components late. And no post-transfer availability.
How do you know it worked?
The receiving team changing the system confidently, an evaluation suite they can run, incidents handled without the vendor, and no capability gap during the transition.
What does it cost?
Mostly people's time rather than tooling. The expensive version is the one that stalls halfway and leaves the organisation with neither the old state nor the new one, which is why a narrow first pass beats a comprehensive plan nobody finishes.
Budget the work as an operated change rather than a project with an end date, because most of these need a maintenance tail. See AI total cost of ownership.
What should you do first?
Ask the vendor for the evaluation set and the decision records. Whether those exist tells you how difficult this transition will be.
How FISTA Solutions helps
FISTA Solutions runs this work alongside client teams rather than around them: evaluation sets and decision records transferred alongside code, knowledge moved through working sessions and a parallel period rather than documents, evidence produced as the work proceeds, and handover that leaves your people able to continue without us. Delivery runs through AI agents, AI enablement, and forward deployed engineers. The record is 150+ projects for 50+ companies across 12+ countries, with 47% average efficiency gains where measured.
To run this with support, message FISTA on WhatsApp, or read how to audit an AI vendor.
Share-ready article cover
Download the generated social format.
Clear answers
Questions raised by this field note.
Straightforward guidance for evaluating scope, fit, and the next step.
01What should you acquire besides the code?
The evaluation set, the decision records, the prompt history, the data provenance, the runbooks, and the incident history. Those explain why the system is as it is, and they are the parts that cannot be reconstructed.
02Why does the evaluation set matter most?
Because it encodes what the task actually requires, accumulated over the system's life. Without it, the receiving team cannot change anything safely and will be afraid to touch the system, which is the common post-transition state.
03When should the receiving team be built?
Before the transfer, with a period working alongside the vendor. Teams hired after the handover learn the system from code with nobody to ask, which is considerably slower and produces a worse outcome.
04Why run in parallel?
Because a clean cutover discovers the gaps in production. A period where the receiving team operates the system with the vendor available surfaces what was not transferred while help is still contractually available.
05What usually needs rebuilding?
Anything built on vendor-specific infrastructure, monitoring tied to their platform, and deployment tooling. Those transfer poorly and are frequently cheaper to rebuild than to adapt.
Continue exploring
Related capabilities
Start with the hard problem
Need the outcome owned, not merely analyzed?
Tell us where delivery is constrained. We’ll map the fastest credible path from intent to verified production.