Who Is Accountable When an AI Agent Fails?

The wrong email goes out, the wrong payment is approved, the wrong answer reaches a customer, and the first question in the room is whose fault it is. Companies that have not answered the question in advance answer it badly, usually by blaming the technology. This guide gives executives an accountability model for agent failures: why the company stays accountable, how to layer ownership internally, what vendors actually owe, and the records that make the position defensible. It is general guidance, not legal advice.

Why does accountability stay with the company?

Because customers, employees, and regulators deal with the company, not with its model provider or its agent. An agent's action is the company's action in every relationship that matters. The company may have recourse against a vendor under contract, but that is a separate question from its responsibility to the people affected. FISTA's general counsel's guide to AI and agentic AI covers the legal dimensions; this piece covers the organizational model.

How should accountability be layered?

Layer	Accountable for	Not accountable for
Business owner	The outcome; the scope and authority granted to the agent; the supervision level; exception handling; the decision to expand or retire	The system's engineering; the model's behavior on inputs outside the specification
Technical owner	The system: architecture, permissions as implemented, evaluation before release and on schedule, monitoring, incident response	The business decision to grant authority; the process design
Functional executive	Approving authority within appetite; funding; resourcing operations	Individual agent behavior
Accountable executive and CEO	The risk appetite; the governance structure; the evidence standard; whether the layers above exist and function	Individual agent behavior
Board	Oversight that the structure exists and operates	Management of the program

The assignment is recorded in the inventory before launch, so the question "whose fault" has an answer that was written down when nobody was under pressure. The executive guide to AI agent governance describes the inventory.

What does the vendor owe?

What the contract says: service levels, security commitments, data-handling terms, deprecation notice, warranties, indemnities, and whatever else was negotiated. A vendor that breached those terms is accountable under them. A vendor that delivered a model that behaved as models do, on an input the company never evaluated, with permissions the company granted, has not breached anything. In practice, most agent failures trace to the deploying company's permissions, evaluation, or oversight, which is why the AI vendor contracts matter but do not substitute for controls.

Is accountability for a probabilistic system fair?

It is, when it attaches to decisions rather than to the system never being wrong. Nobody can be accountable for a model's every output. Everyone in the layers above can be accountable for the judgment they exercised: what authority they granted, what evidence they required, what supervision they applied, what monitoring they funded, and how they responded when it failed. This distinction is what makes accountability compatible with the blameless analysis that improves systems. The AI incident postmortem template separates the two.

What records make the position defensible?

The records that show reasonable oversight was designed and exercised: the inventory entry with owners and risk tier; permissions and approval gates in force at the time; evaluation results before release and on schedule; the full trace of the failing run; supervision and oversight records; the incident response and remediation; and the decisions that set the agent's autonomy, with the evidence cited. A company with these records can show it acted reasonably; a company without them is arguing from memory. The AI observability explained for executives piece explains where the trace comes from.

What happens in the first hours after a failure?

Contain (stop the agent or withdraw the affected authority), assess (what happened, to whom, how widely), communicate (to affected parties, honestly), and then investigate blamelessly while preserving the records. Assigning blame in the first hour is both unfair and unhelpful; assigning ownership of the response is essential. The what executives should do in the first hour of an AI incident guide covers the sequence.

What should executives ask before the failure?

• For each agent, is the business owner, technical owner, and approving executive recorded?
• What authority did we grant, and what evidence supported it?
• Could we produce the trace, the evaluation results, and the oversight records for any run?
• What does our vendor contract actually commit the vendor to?
• Do our people know that accountability attaches to decisions, not to the model being perfect?

How can FISTA Solutions help?

FISTA Solutions builds AI agents that arrive with the accountability structure in place: named owners in the inventory, permissions and gates recorded, evaluation evidence, tracing, and oversight records, and works with executive and legal teams through its AI enablement practice to establish the layered model across existing agents. Since 2017, FISTA has delivered 150+ projects for 50+ companies across 12+ countries.

To confirm that every agent you run has a recorded answer to "whose fault," talk to FISTA on WhatsApp, or read AI audit and accountability for the audit view.