The Hidden Cost of Agents

Every cost model we have written about on this blog assumes a shape: a request goes in, an answer comes out, and the bill is a function of the two. Agents break that shape. An agent is not a request; it is a program that issues requests — planning calls, tool calls, reflection calls, retries — and the number of calls is decided at runtime by the model itself. The per-token price did not change. The thing being priced did.

The consequence is that agent costs are not predicted by the pricing page. They are predicted by the loop: how many calls one user action causes, how long each call is, and how often the loop fails and starts over. Teams that budgeted agents like chat features discover the difference in the first invoice.

This post is the cost anatomy of that loop — where the tokens go, which parts are waste, and how to budget a workload that decides its own length.

Four places, in rough order of surprise. The conversation itself: every agent call re-sends the accumulated context — the system prompt, the tool definitions, the history so far — and the context grows with every step. A loop of a dozen steps is not a dozen small requests; it is a dozen requests whose average size is the whole conversation so far.

Tool definitions ride along too. Every call carries the full schema of every tool the agent might use, whether or not this step uses any of them. Large tool sets are a standing tax on every single call.

Then the failed attempts: the malformed tool call, the parse error, the retry. Each failure is a full request that produced nothing the user will ever see.

And finally the reflection: the model checking its own work, summarizing, planning the next step — tokens spent on process, not on answer. None of these are line items on the pricing page. All of them are on the invoice.

Because the loop has no natural length. A chat answer ends when the model stops talking; an agent ends when it decides the task is done, and that decision is a judgment call made by the same model that is being paid by the step. Under ambiguity, the loop runs long. Under failure, it runs again.

The multiplication is visible in the arithmetic: if one user action causes a dozen model calls and each call carries a growing context, the cost of that action is not the cost of one answer but the cost of a small campaign. The rate-limit post made the same observation from the capacity side — chatty architectures multiply requests — and the cost side is the same observation in dollars.

Retries are the quiet multiplier. A tool call that fails validation is retried with the error message appended, which makes the retry longer than the original. A loop that retries generously converts one failure into several paid attempts, and the user experience does not improve in proportion.

Tool calling is where agents earn their keep, and it is also where the bill hides. Every call carries the tool schemas; every tool result is appended to the context; every step pays for the privilege of the next step. A tool-heavy agent can spend more tokens describing what it is about to do than doing it.

Two disciplines blunt the tax. Keep the tool set minimal — an agent with twenty tools pays for twenty schemas on every call, and most steps use one. And keep tool results small: return the field the agent needs, not the whole record. A search tool that returns the top result with a snippet is a different cost profile from one that returns the top fifty with full text, and the agent rarely needs the difference.

The deeper fix is architectural: not every step needs the frontier model. The planning step, the reflection step, and the final answer may deserve different strengths — the same right-sizing argument from the hard-coding post, applied inside a single loop.

You budget the loop, not the call. Measure steps per completed task, tokens per step, and cost per completed task — the framework from the cost-per-task post, with the task redefined as the whole agent run. That number is the only one that survives contact with the invoice.

Then set the guardrails from the budgets post at the loop level: a step cap, a token cap per step, and a spend alert on the agent feature specifically. An agent that cannot exceed a step budget is a product decision; an agent that can is a liability with a nice demo.

And log the failure taxonomy. Steps that fail, retry, and succeed are different from steps that fail and are abandoned, and both are different from steps that succeed slowly. The invoice cannot tell these apart; your logs must.

Honestly: when the task cannot be done in one call, and the multi-step version is worth the multiple. That is a narrower set than the demos suggest. If a single well-prompted call with retrieval answers the question, an agent is a more expensive way to get the same answer — the fine-tune-prompt-or-RAG post makes the same point about levers.

Agents earn their cost when the work is genuinely sequential: the answer depends on the result of the previous step, and no amount of prompt engineering collapses the sequence. Research that requires reading then reasoning, operations that require calling then reacting — these are loops by nature.

And the honest test is the eval: run the same task set through the agent and through the best single-call alternative, and compare cost per completed task. If the agent does not win on quality by enough to justify its multiple, it is a feature, not a strategy.

The teams running agents profitably share three habits. They cap the loop — steps, tokens, and spend — before launch, not after the first invoice. They right-size the models inside the loop, because a reflection step does not need the same strength as the final answer. And they treat the failure taxonomy as a product metric, reviewed with the same seriousness as latency.

None of this makes agents cheap. It makes them predictable, and predictability is what turns an agent from a cost surprise into a cost line. The surprise is the expensive part.