Checklist ┬╖ 5 minute read
AI Onboarding Checklist for Engineers Joining the Team
Engineers joining an AI team need the architecture, the evaluation suite, the cost model, and the incident history, not just repository access. The fastest way to make someone productive is showing them what has already failed and how quality is measured, because both shape every decision they will make.
Engineers joining an AI team need more than repository access, and the usual onboarding order is backwards. This checklist covers what to show and when, drawn from FISTA Solutions' staff augmentation practice.
What should the first week cover?
Six areas, in this order.
| Area | Why it comes here |
|---|---|
| Evaluation suite | Defines what correct means |
| Architecture walkthrough | Shows where things happen |
| Cost model | Shapes every design choice |
| Incident history | Explains the odd parts |
| Conventions | Mostly undocumented |
| First change, paired | Teaches the workflow |
Access and environment
Necessary and not sufficient; get it done quickly and move on.
- Repository access and local environment running
- Provider API access with appropriate limits
- Access to observability and cost dashboards
- Access to the evaluation platform
- Access to staging and preview environments
- Read access to production logs with appropriate restrictions
- Added to the relevant alert and incident channels
Evaluation first
The frame everything else fits into. See how to build an agent evaluation harness.
- They run the evaluation suite themselves on day one
- Walk through what the cases represent and where they came from
- Explain the scoring criteria and who set them
- Show a past regression the suite caught
- Show where results are stored and how they are compared
- Explain how a new case gets added
- Point at the cases that are weakest or missing
Architecture
Where things happen and why they are arranged that way.
- End-to-end walkthrough of one real request
- Retrieval pipeline explained including chunking decisions
- Where prompts live and how they are deployed
- Validation layers and what they catch
- Failure handling and fallback paths
- Tool and permission model if agents are involved
- Known weak points named explicitly
Cost
Directly shapes design decisions in this domain.
- Cost per task for the main operations
- Which operation dominates the bill
- How routing decisions were made
- Where cost alerts go and what the thresholds are
- The monthly budget and current position
- Efficiency work already identified
- How to check the cost impact of their own changes
Incident history
The fastest transfer of hard-won knowledge available.
- Last year of incidents read
- Two or three walked through with someone who was there
- Which code exists because of an incident, and which one
- Rollback procedure explained and observed
- On-call expectations explained
- Runbooks read and any gaps noted
- Near misses and known fragilities discussed
Conventions and first change
Most of this is not written down, which is why it needs a conversation.
- Prompt conventions explained with examples
- Why specific instructions exist in the main prompts
- What has been tried and abandoned, and why
- Review expectations and who reviews what
- A small real change assigned and paired on
- They take it through evaluation, review, and staged rollout
- Onboarding gaps they noticed captured for the next person
What are the most common failures?
Starting with the codebase. Skipping incident history. Leaving cost until it becomes a problem. Assuming prompt conventions are self-evident. And assigning solo work before they have seen the full workflow once.
Who should own this?
A named buddy on the team, with the team lead accountable for completion. Onboarding owned by nobody in particular takes three times as long and leaves gaps.
How often should it run?
Per person, with the checklist updated after each one based on what the new joiner found missing. That feedback loop is what keeps it current.
What evidence should it produce?
A completed checklist per joiner and time to first shipped change. If that time is not falling as the checklist improves, the onboarding is not working.
What about non-engineers joining the team?
Domain experts and reviewers need a different version: what the system does, what it cannot do, how to report a problem, and how their corrections feed back into evaluation.
That last point matters and is usually omitted. Reviewers who understand that their corrections become test cases engage differently with the work. See AI user training checklist.
What should you do first?
Ask your most recent joiner what they wish they had been shown first. The answer is usually the evaluation suite or the incident history.
How FISTA Solutions helps
FISTA Solutions builds and operates production AI systems through AI agents, AI enablement, and forward deployed engineering: onboarding that starts with the evaluation suite and the incident history, and a first paired change taken through the full delivery process, decisions documented with their reasoning, and handover that leaves your team able to maintain what was delivered. The record is 150+ projects for 50+ companies across 12+ countries.
To adapt this checklist to your environment, message FISTA on WhatsApp, or read AI handover checklist.
Share-ready article cover
Download the generated social format.
Clear answers
Questions raised by this field note.
Straightforward guidance for evaluating scope, fit, and the next step.
01What should come first?
The evaluation suite. It shows what the system is supposed to do, what correct means, and how changes are judged, which is the frame everything else fits into.
02Why does incident history matter?
Because it encodes what has already gone wrong. A new engineer who has read the last year of incidents avoids repeating them and understands why odd-looking code exists.
03Why show the cost model early?
Because AI architectural decisions have direct per-call cost consequences. An engineer who does not know the cost per task will make choices that are correct technically and expensive operationally.
04What is usually undocumented?
Prompt conventions, why particular instructions exist, which parts of the system are fragile, and what was tried and abandoned. All of it lives with people rather than in the repository.
05How soon should they ship?
Within the first week, on something small, paired. Shipping through the full process тАФ evaluation, review, staged rollout тАФ teaches the workflow faster than reading about it.
Continue exploring
Related capabilities
Start with the hard problem
Need the outcome owned, not merely analyzed?
Tell us where delivery is constrained. WeтАЩll map the fastest credible path from intent to verified production.