Disclosure: BlueBear builds an AI agent platform and an MCP gateway, so we have a commercial interest in this topic. This article describes failure modes and a test method; it contains no model comparison numbers, because we have not run a benchmark at a scale that would justify publishing one.
The short answer: selection breaks before syntax does
Tool calling is two distinct skills, and they fail in different orders.
Schema adherence is emitting arguments that satisfy the tool's JSON schema — right field names, right types, required fields present. Smaller models are broadly competent at this, and constrained decoding or structured-output modes narrow the remaining gap by making malformed output impossible rather than merely unlikely.
Selection is choosing the right tool from the ones available, and calling it at the right moment with the right values. This is the skill that degrades, and it degrades with the number and similarity of your tools.
That ordering matters for cost, because the two failures have very different prices. A schema failure is caught immediately by validation and retried — one wasted call. A selection failure is not caught at all: the tool executes successfully and returns the wrong thing, the agent reads it, and it takes at least one more iteration to recover. And every iteration carries the accumulated context, so late-run recovery is the most expensive kind.
The economics of a wrong tool call
Illustrative arithmetic, not a measurement. Suppose a small model costs a fifth of a larger one per token, and picks the wrong tool 15% of the time against the larger model's 3%.
Each wrong pick costs: the call that made it, the tool execution, the tool result (which is then carried on every subsequent iteration), and at least one additional iteration to recover — itself carrying everything before it. In a run where accumulated context grows with each step, an extra iteration late in the run costs several times what an early one does.
The point is not the exact multiple, which depends entirely on your context sizes. It is that a 12-point difference in selection accuracy is not a 12% cost difference. Selection errors are amplified by the same quadratic context growth described in one request, many model calls, and a naive per-token comparison will therefore recommend the wrong model.
Testing it properly
The standard demo — three well-separated tools, obvious cases — tells you almost nothing. It is the easiest possible instance of the task.
A test that predicts production has four properties:
| Property | Why |
|---|---|
| Your real tool count | Selection accuracy is a function of the choice set. Testing 3 of your 22 tools measures a task you do not have. |
| Your real descriptions | Overlapping descriptions are the dominant cause of wrong picks, and they only exist in your catalog. |
| Real inputs, including ambiguous ones | Sample from production. Invented cases are systematically less ambiguous than real ones. |
| Multi-step traces, not single calls | Selection quality changes as context accumulates. A model that picks well at step 1 may not at step 6. |
Record three metrics separately, because they have different fixes: selection accuracy (right tool), argument validity (schema satisfied), and argument correctness (right values in a valid shape). The third is the sneakiest — a call that validates and executes with the wrong date range returns real data that is silently wrong, and no amount of schema tightening catches it.
The full acceptance discipline — criterion first, real traffic, blind grading, non-inferiority margin — is in is a cheaper model good enough? and applies here unchanged.
Designing tools that stay cheap
Most tool-calling cost problems are catalog problems, not model problems. Five changes, in order of payoff:
1. Return less
The single highest-yield change in any tool-calling agent. A tool that returns a raw API response hands the model dozens of fields it will never read, and pays for them again on every remaining iteration. Project the response down inside the tool. This improves selection quality and cost simultaneously — the model's signal gets cleaner as the payload shrinks.
2. Bind fewer tools per step
Schemas are re-sent on every iteration, so a large catalog is a fixed tax on every call in the run — and it degrades selection at the same time. Splitting one agent into narrower stages, each binding only the tools it needs, attacks both at once. This is usually a bigger win than any model change.
3. Make descriptions mutually exclusive
Read your tool descriptions as if you had no context. If a competent new colleague could not choose between two of them, neither can any model. Say what each tool is not for. search_documents versus search_knowledge_base is the canonical failure; naming them by their data source and saying so in the description usually fixes it outright.
4. Make failure loud
A tool that returns an empty result for both "no matches" and "your query was malformed" teaches the model nothing. Distinct, explicit error messages let it correct in one iteration rather than exploring for three.
5. Cap the loop and count the cap
An explicit iteration limit bounds the worst case. Hitting it should be recorded as an event, not swallowed — a rising cap-hit rate is a selection regression showing up before anyone reads the invoice.
Make tool failure a first-class signal
Tool-call failure deserves its own field on every run record, next to whether the run succeeded and whether the user regenerated. In our own route-outcome payload it is exactly that — a toolCallFailed flag reported alongside userRegenerated, the executed model, the token counts and the observed cost.
The reason to keep it separate rather than folding it into a general success rate is that it is the metric that moves first when a model is substituted. Overall success may hold steady for a while because retries paper over selection errors; tool-call failure moves immediately. If you are going to watch one number after a model change on a tool-calling step, watch that one. Treating repeated tool failure as a fallback trigger rather than a retry loop is the wider pattern in multi-model resilience.
And when a model is changed, the record has to say which model actually ran — not which one was recommended. A tool-failure rate attributed to the wrong model is worse than no data, because it is confidently wrong. That is the whole argument of plan-honoured accounting.
A note on MCP
If your tools are exposed over the Model Context Protocol, everything above still applies — MCP standardises how tools are described and invoked, not how well a model chooses between them. Two things change in practice. Your catalog may grow faster, because adding a server adds all of its tools at once, which makes the selection problem worse without anyone deciding to make it worse. And tool descriptions now come from someone else, so the mutual-exclusivity discipline has to be applied at the point where you decide which servers and which tools to expose to a given agent. The governed integration catalog covers admitting a server deliberately rather than by default.
Do this next
Count the tools bound to your highest-volume agent step. If it is more than a handful, read every description in one sitting and mark any pair you could not reliably choose between. Rewriting those descriptions is free, needs no evaluation, and typically improves both cost and correctness — which almost nothing else in this space does.
Questions people actually search for
- do cheaper models handle tool calling well
They usually handle the mechanics well — emitting syntactically valid arguments against a schema — and degrade first on selection: choosing the right tool from several plausible ones. Because selection failures cost an extra iteration each time, and iterations carry accumulated context, a small model that picks wrong 15% of the time can end up more expensive than a larger model that picks right, even at a fraction of the per-token price.
- how many tools can an llm handle
There is no fixed number, and any figure you are quoted was measured on somebody else's tools. What is consistent is the direction: selection accuracy falls as tool count rises, and it falls faster for smaller models and for tools whose descriptions overlap. The actionable version is to measure your own selection accuracy at your own tool count and treat it as a metric you watch, not a constant.
- why does my agent call the wrong tool
Most often because two tool descriptions are close enough that the correct choice is genuinely ambiguous from the text —
search_documentsandsearch_knowledge_baseis a classic pair. Before blaming the model, read your descriptions as if you had no context. If a competent new colleague could not choose between two of them from the descriptions alone, the model cannot either, and a bigger model will only guess more confidently.- how do I reduce the cost of tool calling
Three changes, in order: return less from each tool, because a large tool result is re-sent on every remaining iteration; bind fewer tools per step, because schemas are re-sent every iteration and choice quality falls with count; and record tool-call failures as a first-class metric so a selection regression shows up as a number rather than as a slightly higher bill.