Disclosure: BlueBear builds an AI agent platform and an MCP gateway, so we have a commercial interest in this topic. The worked example below is illustrative and labelled as such — it is not a measurement of any customer system.
The short answer: count the spans, not the requests
The mental model most people carry into agent work comes from chat: one question, one answer, one charge. Agents break it completely. A single user-visible request routinely triggers six to fifteen model calls, and the number is not fixed — the model decides at runtime how many tool round trips to take, so the same endpoint can cost four times as much for a hard input as an easy one.
This is not a defect and it is not overhead to be eliminated. It is what makes agents work on problems a single prompt cannot solve. But it means every cost intuition you have from per-call pricing is wrong by a multiple, and the multiple is not knowable without measuring.
The anatomy of one request
Here is a representative decomposition of a support-triage agent answering one customer question. Not every system has every stage, and some have stages this list omits.
| Stage | Model calls | What it carries | Notes |
|---|---|---|---|
| Input guardrail | 0–1 | The user input, a policy prompt | Often the same model as the agent, which is usually a mistake — this is classification. |
| Planner / router | 0–1 | Task description, available capabilities | Sometimes a model, sometimes deterministic. The deterministic version is free. |
| Agent iteration 1 — tool selection | 1 | System prompt + all tool schemas + user input | The prefix that gets re-sent forever after. |
| Retrieval | 0–1 | Query embedding, or a model-generated query | Embedding calls are cheap; a model rewriting the query is not free. |
| Agent iterations 2–n | 2–8 | Everything above, plus every prior tool result | Where the cost actually is. |
| Summarisation / compaction | 0–2 | The accumulated context | Pays for itself only if it runs before the context has already been sent several times. |
| Retry after a failure | 0–3 | The whole run again | Full price, no partial credit. |
| Verification / self-check | 0–1 | The draft answer plus criteria | Classification again. Rarely needs the expensive model. |
| Output guardrail | 0–1 | The final answer | Same. |
| Final response | 1 | Full context | The one call the user experiences. |
Fourteen is not a worst case. It is an ordinary Tuesday for an agent with two guardrails, retrieval, four tool round trips, one retry and a verification pass.
Why the arithmetic is worse than the count suggests
Counting calls understates the problem, because the calls are not the same size. Each iteration of a tool-calling agent carries the entire conversation so far: system prompt, every bound tool's JSON schema, every prior tool call and every prior tool result.
So if we call the accumulated content at step 1 one unit, step 2 carries two, step 3 carries three, and the total for an n-step run is n(n+1)/2 units rather than n.
| Steps | Calls | Relative input tokens | vs. 2 steps |
|---|---|---|---|
| 2 | 2 | 3 | 1.0× |
| 4 | 4 | 10 | 3.3× |
| 6 | 6 | 21 | 7.0× |
| 10 | 10 | 55 | 18.3× |
Illustrative: a simplified model assuming each step adds a comparable amount of context. Real growth depends on the size of individual tool results, and prompt caching materially changes the effective price of the repeated prefix.
Two consequences follow, and they are the reason this article exists.
Development does not predict production. In testing you use clean inputs that resolve in two or three steps. In production the hard cases take eight. If your test cases average 3 steps and production averages 6, your real cost is not double your estimate — it is closer to three and a half times it.
The largest tool result is the most expensive object in your system. A tool returning a 6 KB unfiltered API response at step 2 of a seven-step run is paid for six more times. Projecting that response down to the four fields the agent reads is frequently the single highest-yield change available, and it improves the model's signal at the same time.
Where to cut, in order of payoff
- Shrink what tools return. Filter inside the tool, not in the prompt. Highest yield, improves quality, no evaluation needed to justify it.
- Move the classification stages to a cheap model. Guardrails, verification, routing and intent detection are classification, and classification is where small models hold up best. See which steps need a frontier model.
- Reduce iterations by narrowing tool choice. An agent choosing among twenty tools iterates more than one choosing among four, and pays a schema tax on every iteration for the privilege. Splitting one agent into narrower stages attacks both.
- Make the stable prefix cacheable. Stable content first, volatile content last. Verify by diffing the rendered prompt across two consecutive requests.
- Cap the worst case. An explicit iteration limit does not reduce the median at all. It converts an unbounded tail into a known one, which is what makes the cost forecastable.
- Delete steps that do not change the answer. The uncomfortable one. Run twenty real requests with and without the verification pass and compare acceptance. Sometimes it is carrying the system; sometimes it has been agreeing with the draft for six months.
Making step count visible
You cannot manage this without recording it per run. The minimum useful record is: number of model calls, input and output tokens per call, the largest single message in the conversation, whether the run succeeded, and whether the user regenerated.
The last one is worth a sentence of its own. A regeneration is the cheapest quality signal you will ever get — the user is telling you the answer was inadequate, at no measurement cost. It is a field in our own route-outcome payload for that reason, alongside a flag for whether a tool call failed. A change that cuts model cost 20% and raises regenerations 30% has made the product worse, and a cost dashboard alone will report it as a success.
If you use OpenTelemetry, the GenAI semantic conventions give you a shared vocabulary for these spans, which is worth adopting simply so the numbers mean the same thing in two years.
The question this reframes
Once you can see that one request is fourteen calls, "which model should we use?" stops being one question and becomes fourteen. Some of those fourteen genuinely need the strongest model available. Most do not. That is the whole opportunity in agent cost work, and it is invisible until you count.
It also reframes routing. A router that picks one model for the whole request is solving a coarser problem than the one you have; the useful granularity is per step. That is why eligibility, ranking and fallback are separate decisions in a serious routing policy, and why route plans are per request rather than per tenant — the routing policy guide works through the fields.
Do this next
Open one trace for your highest-volume agent and count the spans. Write down the number and the largest single message in the conversation. Those two facts, for one real request, will reshape your cost work more than any pricing comparison — and if the trace does not exist, that is the finding.
Questions people actually search for
- how many llm calls does an ai agent make per request
For a single-turn chat completion, one. For a tool-calling agent, it is one per iteration plus any guardrail, planning, summarisation and verification calls you have added — commonly six to fifteen for one user-visible answer. The number is not knowable from the code alone because it depends on how many tool round trips the model chooses to take, which is why you have to measure it per run rather than reason about it.
- why does agent cost grow faster than the number of steps
Because each iteration re-sends everything before it. Step 3 carries the system prompt, the tool schemas, and the results of steps 1 and 2. So the total input tokens for an n-step run scale with roughly n(n+1)/2 rather than n. Doubling the steps roughly triples the input tokens. This is the single most important cost fact about agents and it is why per-call pricing intuitions do not transfer.
- how do I count the model calls in my agent
Open a trace for one representative request and count the spans, rather than reading the code. Whatever tracing you use — OpenTelemetry GenAI semantic conventions, a framework's built-in tracing, or your own — the count for one real run tells you more than an hour of reasoning about the graph, because the model chooses the iteration count at runtime.
- is a multi step agent worth the cost
Sometimes clearly yes, and the honest test is whether the steps change the answer. Take twenty real requests, run them through the full agent and through a single well-constructed prompt, and compare acceptance. On genuinely decomposable work the agent wins convincingly. On work that was always one retrieval and one generation, teams routinely find the extra steps buy very little — which is a much cheaper thing to discover from twenty examples than from an invoice.