AI Data Migration Checklist: Moving a Corpus Without Loss

Corpus migrations lose content quietly, and the loss is usually discovered by a user rather than by the team. This checklist covers doing it verifiably, drawn from FISTA Solutions' AI enablement delivery work.

What does a migration involve?

Six phases, each with a verification step.

Phase	Verification
Inventory	Counts and checksums recorded
Permission mapping	Access tested per role
Transfer	Every item accounted for
Re-embedding	Coverage confirmed, cost tracked
Validation	Retrieval compared side by side
Cutover	Rollback available and tested

Inventory and baseline

You cannot verify a migration you did not measure first.

• Document count, chunk count, and total size recorded
• Source systems and their owners listed
• Metadata schema documented for every field in use
• A checksum or identifier recorded per document
• Known-broken or orphaned content identified before the move
• A query set assembled for before-and-after comparison
• Baseline retrieval results captured for that query set

Permissions and access

This is where migrations cause security incidents. Map explicitly rather than assuming equivalence.

• Permission model of both systems documented
• Mapping written for every role and access level
• Documents with restricted access identified explicitly
• Access tested per role after transfer, not assumed
• Over-granting checked as carefully as under-granting
• Service account permissions scoped to the migration only
• Permission changes logged for audit

Transfer and integrity

Every item must be accounted for, including the ones that failed.

• Transfer is resumable from any interruption point
• Every document accounted for: transferred, skipped, or failed
• Failures logged with reasons, not silently dropped
• Metadata preserved including dates, sources, and ownership
• Character encoding verified on a sample of non-English content
• Large or unusual documents tested explicitly
• Counts reconciled against the inventory baseline

Re-embedding

A full pass over the corpus with real cost and real elapsed time.

• Whether re-embedding is required determined and stated
• Compute cost estimated before starting
• Elapsed time estimated and scheduled
• Progress tracked so partial completion is visible
• Embedding model version recorded with the index
• Dimension and index configuration verified against the new model
• A sample of embeddings spot-checked for sanity

Validation

Compare behaviour, not counts. See RAG quality checklist.

• The baseline query set run against the new system
• Retrieved results compared item by item against the baseline
• Differences investigated rather than accepted
• Retrieval latency measured and compared
• Evaluation suite re-run end to end
• A domain expert reviews a sample of answers from both systems
• Permission-filtered retrieval tested for each role

Cutover and rollback

Keep the reverse path available until the forward path is proven.

• Cutover plan written with a rollback trigger
• Old system kept available and readable after cutover
• Traffic moved gradually where possible rather than all at once
• Monitoring in place for retrieval failures and latency
• A defined watch period before decommissioning anything
• Users told what is changing and how to report problems
• Decommissioning scheduled explicitly rather than forgotten

What are the most common failures?

Verifying by document count. Assuming permissions map cleanly. Discovering re-embedding cost mid-migration. Decommissioning the old system immediately. And no baseline query set, which makes validation impossible.

Who should own this?

Engineering runs the migration; the content owner validates that the corpus still answers what it should. Without the second, the migration is verified technically and not functionally.

How often should it run?

Per migration. The baseline query set and validation approach should be reusable, because migrations recur — provider changes, embedding model upgrades, and consolidation all trigger one.

What evidence should it produce?

Inventory counts before and after, the permission mapping, transfer logs including failures, and the side-by-side retrieval comparison. That package demonstrates nothing was lost.

What if you find loss after cutover?

Roll back if the old system is still available, which is the argument for keeping it. If it is not, re-migrate the missing content and treat the gap as an incident.

The more common situation is discovering loss weeks later through user reports. Continuous comparison of retrieval results for a period after cutover is what catches it earlier. See observability for web apps.

What should you do first?

Capture baseline retrieval results for fifty representative queries before you move anything. That single artefact is what makes validation possible.

How FISTA Solutions helps

FISTA Solutions builds and operates production AI systems through AI agents, AI enablement, and forward deployed engineering: migrations validated by comparing retrieval results against a captured baseline rather than by counting documents, with the old system kept readable through a watch period, decisions documented with their reasoning, and handover that leaves your team able to maintain what was delivered. The record is 150+ projects for 50+ companies across 12+ countries.

To adapt this checklist to your environment, message FISTA on WhatsApp, or read RAG quality checklist.