AI Agent Security Architecture: A Whitepaper

An AI agent is a program that reads untrusted content, decides what to do, and does it using credentials you gave it. Described that way, it is obvious why agents need a security architecture rather than a safety prompt. This whitepaper presents a threat model for autonomous agents and a defense-in-depth architecture that FISTA Solutions applies to every production agent. It is the engineering counterpart to the agentic AI governance whitepaper.

What is the threat model for AI agents?

Agents differ from conventional applications in one decisive way: their control flow is influenced by data. Whatever text an agent reads, whether a retrieved document, an email, a web page, a tool result, or a user message, can attempt to change what the agent does. Combined with credentials and tools, this produces a distinct threat model.

Threat	Vector	Impact
Direct prompt injection	Malicious user input	Agent ignores policy, reveals data, performs unauthorized actions
Indirect prompt injection	Poisoned documents, emails, web content, tool outputs	Same as above, triggered without the user's knowledge
Credential abuse	Over-broad tokens, shared service accounts	Lateral movement, data access far beyond the task
Data exfiltration	Outputs, tool calls to external destinations, logs	Leakage of sensitive or regulated data
Unsafe actions	Manipulated or mistaken tool use	Financial loss, record corruption, customer harm
Supply-chain compromise	Malicious models, libraries, plugins, MCP servers	Backdoors, data theft
Multi-step manipulation	Gradual steering across a trajectory	Evades single-request checks
Denial of wallet	Loops, large contexts, retries	Runaway cost

Background on the first two is in what is prompt injection and on the broader landscape in AI agent security risks. The OWASP Top 10 for LLM Applications is the most widely used public taxonomy and aligns with this table.

What are the principles of the architecture?

Five principles organize the defenses:

1. The agent is a principal. It has an identity, an owner, and permissions granted against a spec.
2. All content is untrusted. Instructions come only from the system configuration; everything else is data.
3. Capability bounds blast radius. What an agent cannot do, it cannot be tricked into doing.
4. Every action is validated and attributable. Policy is enforced outside the model, and every action is logged.
5. Assume compromise; contain it. Design so a manipulated model produces a contained incident, not a breach.

How should agent identity and credentials be designed?

Each agent authenticates as its own principal in the identity provider. Credentials are:

• Scoped per tool and per action class, with read and write separated.
• Short-lived, issued at run time from a secrets manager rather than embedded in configuration; see AI secrets management.
• Bound to the agent's spec, so a permission review compares granted scopes to required scopes.
• Revocable independently, so one agent can be disabled without affecting others.
• Never shared across agents or with human users.

Where the agent acts on behalf of a user, it should use delegated, user-scoped authorization so that it cannot access more than the user could. This is the foundation on which every other control rests; without it, a prompt-level defense is guarding an open door. The pattern aligns with zero-trust principles described in AI and zero trust architecture.

How is prompt injection defended in depth?

There is no complete fix for prompt injection at the model layer, so defense is layered:

Layer	Control	What it stops
Structural	Separate system instructions from all data; label content sources; strip or neutralize instruction-like patterns in retrieved content	Simple and moderate injections
Detection	Classifier or heuristic screening of inputs and retrieved content for injection patterns; adversarial test suite in CI	Known attack families
Capability	Least-privilege tools; no tool that can exfiltrate to arbitrary destinations; egress allowlists	Impact of successful injection
Validation	Policy checks on every proposed action and output before execution	Unauthorized actions regardless of cause
Oversight	Approval gates on consequential actions; confidence-based escalation	High-impact actions under manipulation
Monitoring	Trajectory anomaly detection; alerting on unusual tool sequences or destinations	Multi-step and novel attacks

The capability and validation layers are the ones that matter most, because they hold even when the model is fully manipulated. A comprehensive checklist is in the prompt injection defense checklist.

How should tool execution be sandboxed?

Tools are where agent decisions become real-world effects. Sandboxing limits what a tool call can reach:

• Code execution runs in ephemeral, isolated environments with no production credentials, restricted network egress, CPU and memory limits, and time limits.
• File access is confined to task-specific scratch locations; no access to shared drives beyond what the spec names.
• Network access is restricted by egress allowlists so that exfiltration to arbitrary hosts is impossible.
• Database access goes through parameterized, permission-scoped interfaces, never raw connections with broad rights.
• Third-party integrations and MCP servers are inventoried, pinned, reviewed, and run with their own minimal credentials; see what is model context protocol and AI supply chain security.

Sandboxing is what turns "the agent was tricked into running a script" from a breach into a log entry.

How are outputs and actions validated?

Policy enforcement lives outside the model, in a validation layer that inspects every proposed action and every output before it takes effect:

1. Schema validation ensures tool arguments and outputs match expected structure and types.
2. Policy rules check actions against the spec: allowed tools, allowed targets, value limits, prohibited operations, rate limits.
3. Content checks screen outputs for sensitive data, policy violations, and unsafe content before they leave the system.
4. Consequence classification routes irreversible, costly, external, or regulated actions to approval gates.
5. Deterministic fallbacks define what happens when validation fails: block, escalate, or degrade.

Because these checks are code, not prompts, they are testable and auditable. Detail is in LLM output validation and AI agent guardrails.

How are consequential actions gated?

Approval gates are a security control as much as a governance one. Any action that moves money, changes records of legal or regulatory weight, contacts external parties, deletes data, or falls below a confidence threshold pauses for a human with context. The gate presents the proposed action, the evidence, and the trajectory so the approver can decide quickly. Gates are only effective when the queue is staffed and the approver has authority; see how to build a human review queue.

How is data protected across the agent lifecycle?

Data protection spans four points:

• At retrieval: permission filtering so the agent only sees what the requesting user may see; see the enterprise RAG reference architecture.
• In prompts: redaction or tokenization of sensitive fields before content reaches a third-party model, per data-handling policy and provider agreements.
• In outputs and tool calls: leakage checks and egress controls, so data cannot leave through a response or a webhook.
• In logs: redaction of sensitive content while retaining enough for forensics; access controls and retention aligned with regulation.

Related guidance is in AI data leakage prevention and AI data privacy compliance.

What monitoring and detection does an agent need?

Agent monitoring must reason about trajectories, not requests. Effective detection includes:

• Full step-level logging: inputs, retrieved content with sources, tool calls with arguments, outputs, validation decisions, gate decisions, and approvers.
• Baselines for normal tool sequences, data volumes, destinations, and cost per task.
• Alerts on deviations: unusual tool chains, first-seen destinations, spikes in data retrieved, repeated validation failures, cost anomalies.
• Correlation with identity and network telemetry in the security operations center.
• Immutable retention for reconstruction and regulatory record keeping; see how to build an AI audit trail.

Observability infrastructure is described in the AI observability whitepaper.

How should agents be tested for security?

Security testing is continuous and adversarial:

• An adversarial test suite of injection payloads, exfiltration attempts, and policy-violation cases runs in CI on every change.
• Red-team exercises against high-consequence agents on a schedule, covering indirect injection through realistic content channels; see AI red teaming guide.
• Penetration testing of the surrounding infrastructure, tools, and integrations; see AI penetration testing.
• Permission audits comparing granted scopes to spec requirements.

Findings feed the golden set and the validation rules, so each discovered attack becomes a permanent regression test.

What does incident response look like for agents?

The response playbook mirrors conventional incident response with agent-specific steps: contain by revoking the agent's credentials or dropping its autonomy level, reconstruct from the trajectory log, assess data exposure and actions taken, remediate effects and notify as required, root-cause across spec, permissions, model, content sources, and validation rules, and prevent by adding tests and tightening controls. Pre-writing this playbook is part of readiness; see the AI incident response checklist.

What does the reference architecture look like end to end?

Component	Security role
Identity provider and secrets manager	Agent principals, scoped short-lived credentials
LLM gateway	Central logging, model pinning, rate and spend limits, content screening
Retrieval layer with permission filtering	Authorized context only
Tool registry and policy engine	Allowed tools, scopes, validation rules, consequence classification
Sandboxed execution environments	Contained code, file, and network actions
Human review queue	Approval gates and escalation
Trajectory logging and SIEM integration	Detection, alerting, forensics
Evaluation and adversarial test pipeline	Continuous verification

Each component is shared across agents, which is why building it as a platform is cheaper and safer than securing agents one at a time.

How FISTA Solutions secures production agents

FISTA Solutions builds agents on this architecture as standard within its AI agents practice: per-agent identity, least-privilege tooling, sandboxed execution, policy validation outside the model, approval gates, trajectory logging, and adversarial testing in the pipeline. Forward deployed engineers work alongside your security team to integrate with your identity provider, secrets management, and SIEM, and the AI enablement practice delivers the shared gateway and policy platform. The result is agents that can be granted real autonomy because their blast radius is bounded by design, backed by a record of 150+ projects with 99.9% uptime.

If your agents currently hold broad credentials and rely on prompts for safety, message FISTA on WhatsApp to scope a security review. For the complete pre-launch control list, see the LLM security checklist.