THE SHORT ANSWER
Agents fail when goals are ambiguous, models make unsupported assumptions, context is stale, tools or permissions fail, observations are misread, retries compound errors, or stopping rules are weak. Because later steps depend on earlier state, evaluate the full trace and final environment outcome.
Locate failure in the loop
| Stage | Failure | Example control |
|---|---|---|
| GOAL | Ambiguous, conflicting or impossible objective | Clarify scope and acceptance |
| UNDERSTAND | Stale context or wrong assumption | Source provenance and freshness checks |
| CHOOSE | Poor plan or wrong tool | Allowed actions and representative evaluation |
| ACT | Invalid arguments, missing permission or tool outage | Validation, least privilege and explicit errors |
| OBSERVE | Partial result mistaken for success | Structured status and environment checks |
| VERIFY | Model confirms its own unsupported work | Independent deterministic or human checks |
| CONTINUE / STOP | Loop, runaway retry or premature completion | Hard budgets and stopping rules |
Small errors can become workflow state
A model may invent a customer identifier, pass it to a tool, interpret an empty result as no account and then draft the wrong response. The final text is only the last visible symptom. Trace evaluation reveals the original assumption and every point where the system could have rejected it.
Read Why Does AI Hallucinate? for the model-level problem. Agent design adds tool, state and control failures around it.
Evidence & context: NIST
Expect the environment to be unreliable
- A service times out after completing an action.
- Permissions change between planning and execution.
- A file is edited while the agent works.
- A tool returns a partial page of results.
- Two actions succeed in a different order than expected.
- A retry repeats a non-idempotent operation.
Return explicit status, preserve operation identifiers and verify the environment before retrying. Recovery logic should distinguish ‘not attempted’, ‘unknown’, ‘failed’ and ‘completed’.
Evaluate trace and outcome together
A fluent final message can claim completion when the environment never changed. Conversely, an unusual path may still reach the correct state. Agent evaluation should inspect tool calls and intermediate decisions while grading the final outcome independently.
Evidence & context: Anthropic
Sources & further reading
- Generative Artificial Intelligence Profile (NIST AI 600-1)
NIST. Risk-management guidance, including confabulation. It does not establish a universal error rate.
- Building effective agents
Anthropic. A provider's engineering taxonomy of agents and workflows, not a universal industry definition. We use the conceptual distinction, not its changing product recommendations.
- Demystifying evals for AI agents
Anthropic. A provider's engineering guidance on multi-turn agent evaluation, checked 13 September 2026. Examples inform evaluation design but do not establish universal pass thresholds.
Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.
A correction, a counterexample or an experience worth sharing?
Join the conversation ↗