How to Build a Multi-Agent System (Playbook)

The temptation with multi-agent systems is to start with the org chart: a planner agent, a researcher agent, a writer agent, a critic agent, all chatting. The result is usually slow, expensive, and impossible to debug. A production multi-agent system is built the other way around: from a measured single-agent failure, along real boundaries, with typed contracts, deterministic orchestration, budgets, and trajectory-level evaluation. This playbook covers the build, following the patterns in the multi-agent orchestration patterns whitepaper and FISTA's AI agents practice.

What are you building?

Element	Description
Orchestration	Code that sequences, parallelizes, and routes work among agents
Agents	Specialized workers with narrow tools, instructions, and contracts
Shared state	Task context, intermediate results, decisions, citations
Contracts	Input and output schemas and success criteria per agent
Controls	Budgets, loop detection, handoff validation, gates
Evaluation	Agent, handoff, and trajectory levels
Observability	Trajectory tracing with cost and decisions

Step 1: Prove the single agent falls short

Build the single-agent version first with good tools and a clear specification, and evaluate it. Identify the specific failure: quality drop as tools grow, context exhaustion on long tasks, latency from sequential independent work, or unverified errors. If there is no measured failure, stop here. Decision guidance is in when to use multi-agent systems.

Step 2: Decompose along real boundaries

Split the task where tools, expertise, or independence naturally differ. Each resulting agent should have a small tool set justified by its spec, instructions a domain expert could review, and a golden set that evaluates it alone. If two candidate agents share most tools and instructions, merge them. If an agent only passes messages, it is orchestration code. Choose the pattern (orchestrator-worker, pipeline, parallel, hierarchical, critic) from the failure you measured.

Step 3: Define typed contracts

For each agent, write the contract: input schema, output schema, success criteria, allowed tools, budget, and escalation behavior. Contracts are versioned and validated at runtime. Structured outputs make handoffs testable. See what is structured output and what is agent handoff.

Step 4: Implement orchestration in code

Write the orchestrator as deterministic code: task decomposition where it can be specified, dispatch, parallel fan-out, aggregation, retries, and error handling. Use model judgment only for decisions that genuinely require it, such as decomposing a novel request, and constrain even those with schemas. Frameworks can provide graph and state primitives; keep control flow explicit. Guidance is in ai agents vs workflows and how to choose an ai agent framework.

Step 5: Build the shared state store

Persist task context, subtask assignments, intermediate results with provenance, decisions, and handoff records in a structured store keyed by task. Agents read what they need and write their results; context survives handoffs; the store supports checkpointing and resumption. See what is agent memory.

Step 6: Enforce budgets and validation

Set budgets per task on steps, tool calls, tokens, cost, and time, enforced by the orchestrator or runtime, with defined behavior on breach. Detect loops through repeated-state checks. Validate every handoff against its schema and policy rules, so a manipulated or malformed output cannot steer the rest of the system. Gate consequential actions regardless of which agent proposes them. Controls are in ai agent guardrails and the AI agent security architecture whitepaper.

Step 7: Evaluate at three levels

• Agent level: each agent against its contract on its own golden set.
• Handoff level: information preservation and schema conformance across boundaries.
• Trajectory level: end-to-end task success, cost, steps, latency, and safety, with adversarial cases at every untrusted entry point.

Trajectory success is the business metric; agent metrics locate the cause when it drops. Harness design is in how to build an agent evaluation harness.

Step 8: Trace trajectories in production

Log each task as one linked trajectory: orchestrator decisions, agent inputs and outputs, tool calls, handoff records, validation outcomes, gate decisions, and cost per step. Provide a trajectory view for debugging. Alert on budget breaches, loop detections, validation failures, and cost anomalies. The model is in the AI observability whitepaper.

Worked example: a vendor risk assessment system

A procurement team needs vendor risk assessments assembled from financial data, security questionnaires, news, sanctions lists, and internal history. A single agent produced incomplete assessments as tool count grew. The rebuilt system uses an orchestrator that creates typed subtasks for financial review, security review, adverse media, sanctions screening, and internal history; workers run in parallel with narrow tools; a critic checks each section for unsupported claims and missing citations with a two-pass limit; and the orchestrator assembles the assessment and routes it to a procurement analyst for approval. Budgets cap cost and time per vendor; the shared state store holds every source consulted. Evaluation covers each worker on its own labeled set, handoff schema conformance, and end-to-end assessment completeness, and the trajectory trace lets analysts see exactly which sources informed each section.

What does it cost to run?

Multi-agent tasks cost several times a single-agent task in model calls; budgets, parallelism, and model routing per worker control it. Measure cost per completed task against the quality gain that justified decomposition. Drivers are in cost of running llms in production.

What are the common mistakes?

• Starting multi-agent without measuring single-agent failure.
• Agents defined by role names rather than tool and expertise boundaries.
• Free-form agent conversation with no schemas.
• Orchestration delegated to a model that improvises control flow.
• No budgets or loop detection.
• Evaluating only final outputs, never handoffs or trajectories.

How FISTA Solutions builds multi-agent systems

FISTA Solutions builds multi-agent systems to this playbook: single-agent baseline first, decomposition along real boundaries, typed contracts, deterministic orchestration with shared state, budgets and handoff validation, three-level evaluation, and trajectory tracing. The AI agents practice delivers the system, AI enablement the runtime, gateway, and observability platform, and forward deployed engineers embed with your domain experts to define boundaries and contracts. The record behind the work is 150+ projects with 99.9% uptime.

To scope a multi-agent system, or to stabilize one that has become hard to control, message FISTA on WhatsApp, or read what is an orchestrator agent for the core pattern.