Why AI agents break on long tasks, and how to design ones that don't.
Small error rates compound over many steps. The arithmetic behind unreliable agents, and the two design levers that make them dependable in production.
Published · 7 min read
An AI agent that handles a three-step task flawlessly in a demo can fall apart on a twenty-step task in production. The model has not become less capable. The task has become longer, and small error rates compound. Understanding that arithmetic is the most useful thing a team can do before putting an agent in front of real work.
The arithmetic of long tasks
Suppose each step of a task, such as reading a document, calling a tool or deciding what to do next, succeeds 95% of the time. That sounds reliable. But a task only succeeds if every step does, so the probabilities multiply. Over twenty steps, 0.95 multiplied by itself twenty times is about 0.36. Roughly two tasks out of three contain at least one error.
| Number of steps | 99% success per step | 95% success per step | 90% success per step |
|---|---|---|---|
| 1 | 99% | 95% | 90% |
| 2 | 98% | 90% | 81% |
| 3 | 97% | 86% | 73% |
| 4 | 96% | 81% | 66% |
| 5 | 95% | 77% | 59% |
| 6 | 94% | 74% | 53% |
| 7 | 93% | 70% | 48% |
| 8 | 92% | 66% | 43% |
| 9 | 91% | 63% | 39% |
| 10 | 90% | 60% | 35% |
| 11 | 90% | 57% | 31% |
| 12 | 89% | 54% | 28% |
| 13 | 88% | 51% | 25% |
| 14 | 87% | 49% | 23% |
| 15 | 86% | 46% | 21% |
| 16 | 85% | 44% | 19% |
| 17 | 84% | 42% | 17% |
| 18 | 83% | 40% | 15% |
| 19 | 83% | 38% | 14% |
| 20 | 82% | 36% | 12% |
| 21 | 81% | 34% | 11% |
| 22 | 80% | 32% | 10% |
| 23 | 79% | 31% | 9% |
| 24 | 79% | 29% | 8% |
| 25 | 78% | 28% | 7% |
| 26 | 77% | 26% | 6% |
| 27 | 76% | 25% | 6% |
| 28 | 75% | 24% | 5% |
| 29 | 75% | 23% | 5% |
| 30 | 74% | 21% | 4% |
The curve is unforgiving in both directions. At 90% per step, a twenty-step task succeeds about one time in eight. At 99% per step, it still fails nearly one time in five. Even a very good model, left to run a long chain on its own, will produce a steady stream of wrong results.
Real systems are not quite this simple. Some errors are harmless, some steps are much easier than others, and a capable model sometimes notices and corrects its own mistakes. But the direction holds, and it matches what teams see when they move from a demo to production volume: problems appear in the long tail of tasks that need many steps.
Two levers that change the outcome
The arithmetic also points to the fix. Reliability depends on how many steps are left to the model and on what happens when one of them fails. That gives two levers.
Give the model fewer steps
Much of what an agent does in a loop does not need a language model at all. Fetching a record, calling an API with known parameters, formatting a file and writing to a database are ordinary code: deterministic, testable and effectively 100% reliable. Every step moved from the model into code leaves the chain shorter.
In practice, this means designing a workflow rather than handing an open goal to an agent. The model handles the parts that need judgment or language, such as reading an unstructured document, classifying a request or drafting a reply. Code handles everything around them.
Check every step that remains
The second lever is to catch errors before they spread. After each model step, check the output: validate it against a schema, reconcile numbers against the source, run the generated code's tests, or ask a second model to review it against explicit criteria. When a check fails, retry the step once, then escalate.
Checks do not need to be perfect to help. In the worked example below, a check that catches four errors out of five raises the effective reliability of each step from 95% to almost 99%. Across twenty steps, that moves the task from 36% to 79%.
Combined, the two levers turn a system that is wrong most of the time into one that is right more than nine times in ten. The remaining failures are no longer silent, either: they surface at a check, where a person can look at them.
What a production workflow looks like
Put together, a reliable agent looks less like an autonomous assistant and more like a well-designed process with a few intelligent steps. Here is the shape we use for document-heavy back-office work, such as processing supplier invoices.
- CodeFetch the invoice and its purchase orderPlain API calls, no decisions.
- ModelExtract amounts, dates and line itemsOutput constrained to a strict schema.
- CheckValidate the schema and reconcile totalsOn failure: one retry, then escalate.
- ModelExplain differences from the purchase order
- HumanApprove differences above a thresholdRoutine cases pass straight through.
- CodeWrite to the ERP and log every step
Three properties matter more than the specific steps.
- The model's output is always structured. Free text is hard to check. A strict schema, with required fields, types and allowed values, turns "is this right?" into a test that code can run.
- Escalation is part of the design. A person reviews the cases the system cannot settle: a failed check after a retry, an amount above a threshold, a supplier it has never seen. Routine cases pass straight through, and people spend their time on the exceptions.
- Every step is logged. When something goes wrong, you need to know which step failed, with what input, and what the model returned. Without that trace, an agent cannot be debugged, audited or improved.
Measure the right thing
Agent quality is usually reported per answer or per step. What the business experiences is the success rate per task. Before deployment, build a set of real tasks with known correct outcomes and measure how many the system completes correctly from start to finish, and how many it escalates. Track both over time. A rising escalation rate is often the first sign that the inputs have changed.
This also gives you a way to make decisions. If a new model raises per-step accuracy, the task-level measurement tells you whether it matters. If a step fails often, you can see whether to improve the prompt, add a check, or move that step into code.
Where to start
Pick one workflow that is repetitive, has clear correct answers and currently takes real time, such as triaging inbound requests, extracting data from documents or reconciling records. Map its steps, decide which ones need a model, and design the checks before you write a prompt. Our applied AI work follows this approach, and you can describe your workflow to us to see whether it is a good candidate.