Agent web search: budget and engine outweigh the model

The conversation about web search in agents is stuck on the wrong question. Everyone asks which language model to use. The question that decides the result is a different one: how many sources that agent will touch, and with which engine. In the 2026-08-12 benchmark that compared the Exa, Parallel and Perplexity engines on the task of searching for an agent, the result changed less when the reasoning model was swapped than when the exploration budget changed. The model is the last layer to decide. Depth and engine decide first, and by then everything is already decided.

Why does the result of an agent that searches the web suffer more from the budget than from the model?

Because the LLM only interprets the context that reaches it, and it is the search budget that decides which context reaches it, which places collection ahead of reasoning in the value chain of the result. A brilliant model over a short and badly chosen set of sources produces a confident and wrong answer. A mediocre model over a broad and well curated set produces a modest and correct answer, which is what pays the bills in production. The bottleneck is not in the reasoning layer, it is in the collection layer.

An agent that queries the web makes three choices before the model writes a single sentence. It chooses the engine, the depth, and the size of the context it will carry into reasoning. None of the three is made by the model. They are made by the tool configuration, by the function calling rollout, and by the token ceiling the application imposes.

This should not surprise anyone who has watched an agent hallucinate with confidence about a fact that a second source would refute in ten seconds. The second source simply did not exist in a search run with minimum depth.

The cost also fits this reading. Each additional query costs search tokens, latency and, when the engine is paid, money per call. Search budget is, at the end of the month, a line on the ledger. The tool that spends the most inside the agent's loop is web search, and it is the tool almost nobody governs.

What did the search-engine benchmark for agents really show?

It compared three engines, Exa, Parallel and Perplexity, on the task of feeding an agent with context to answer a question, and, with reasoning held constant, switching engines shifted the score by around 10 points, while the gap between a top reasoning model and a cost effective one reached around 15 points. The engine matters, but it matters less than the model. And the two together move less than the depth of the search budget. Each engine has its own way of cutting the web, of ordering relevance and of returning context, and that way is what decides the raw material of the agent.

The way the test was designed matters. It was not a test of "who found the right link". It was a test of "who returned the context the agent needed". One engine can find a perfect source and return it buried in a noisy package; another can find a regular source and return it clean. For the agent, the second is more useful, because what reaches reasoning is a truncated text, not a ranking of links to click.

A mandatory reading note before taking the numbers seriously: the results are reported by the vendor that ran the test, not independently verified. So what is worth extracting is not the absolute winner, it is the direction. And the direction the benchmark reconciled with its own vendor-reported title, "while the engine matters, the model matters more", points to a single thread: switching only the engine, with the model held constant, changes less than switching the model, and switching the pure model, without touching the search budget, leaves the biggest gain on the table. Result quality is born from the combination, not from the choice of a single component.

It is here that the technical conversation turns into an architecture conversation. If the engine and the depth dominate, then the company should not lock a single engine into code. It should have the ability to switch engines per call, per cost, per reliability and per the depth requirement of the question at hand. That is routing. Only routing, in this case, does not cover the model, it covers the tool.

Why does the search budget weigh more than the language model choice?

Because all the value of a reasoning model is wasted when the context it receives is shallow, noisy or biased. An excellent LLM does not solve missing information. It writes too good a sentence over too poor information, and that is the worst possible production outcome, because it is undetectable in the flow.

Think about what depth does at three levels. At level one, the agent queries a single page and answers; at level two, it opens the reference links of that page. It is there, at level two, that new material is born. At level three, it follows chains across trusted domains and cross-checks versions. Each level costs more in latency and in search tokens, but each level removes a class of error that the previous levels cannot even name.

Latency is the silent adversary. A high-depth search can take tens of seconds, and the user of a B2B agent does not have half an hour of patience for a routine query. So the decision is not "search more", it is "search more when the question deserves it". A billing question deserves high depth. A form-filling question does not. Deciding when to escalate depth is a policy, and policies are exactly what an AI gateway knows how to apply.

Cost per query closes the argument. Each depth level multiplies engine calls and context tokens. In an operation with thousands of queries a day, the difference between minimum and high depth is the difference between a small bill and a bill that requires two levels of sign-off, the same cost-composition dynamic as the operational cost of the gateway in production. Without a spending cap on search, the search budget is infinite until the day the invoice arrives.

How does web search behave as a tool inside the agent's loop?

It behaves like any other tool exposed by function calling or by a connection protocol, an entry point that the agent decides to call in the middle of its reasoning. For the gateway, web search is not ontologically different from a call to a calculation API, a database, or any connector. It is a tool with a signature, a cost and a behavior profile.

And that is the detail almost everyone misses. When web search is inside the loop, it is called on demand, sometimes dozens of times in a single response. Each call is a routing decision waiting to happen: which engine serves this query, at what depth, within which spending cap. Today that decision is made by default, in the agent's code, locked in at the moment someone chose the engine and never looked again.

The connection protocol made this design even more mature. An agent connected through a standard of message exchange between systems, the same one that brings MCP and tools into an agent's loop, exposes tools in a structured way, with description, parameters and machine-readable limits, the model that the MCP protocol generalizes to any system. This means the layer that intercepts the agent's calls can see every tool, every parameter and every predicted cost before letting the call pass. The visibility exists. What is missing is applying policy to that visibility.

It is here that the gateway stops being a model router and becomes the layer that governs the agent's entire call. The yardstick does not change because the target is now a tool. Only the object changes.

How does the routing policy extend from the model to the search tool?

The same policy the gateway applies to the LLM, routing by cost, by reliability, by latency and by depth, is applicable to the search tool, with the same criteria and the same fallback math. The gateway records the cost-per-query tiebreaker, the automatic failover when an engine degrades, and the spending cap per key, per project or per agent, exactly as it does per token. The extension does not ask for a new product. It asks to apply the existing policy to a new target, the same way endpoint evaluation works as a routing policy for the model.

Consider cost routing. Search engines for agents charge per query at scales that can differ by multiples. For a low-value query, the router picks the cheap engine. For a query that signs a contract, the router picks the expensive engine, because the cost of a wrong answer is greater than the cost of the right query. That is the same "which model for which request" logic, only at the tool level.

Reliability routing follows the same design. An engine stays stable for three days and degrades on a Wednesday. With tool failover, search traffic migrates in milliseconds to a backup engine, and the end user sees nothing beyond a response that arrived on time. Without failover, the agent returns an engine error that looks like a product error.

Depth routing is the newest and the most neglected. It resolves the conflict between quality and latency: simple question, minimum depth; consequential question, high depth. For this the gateway needs to read the intent of the call, which it already does in the intent classification layer it uses for model selection. The tool inherits the classification the model already had, and it is this set of criteria that should be on the table of anyone still evaluating how to choose a gateway.

What changes in practice by routing only the LLM instead of routing the model and the tool?

What changes is what the gateway sees, what it governs, and what it can audit in every agent call. Routing only the LLM lets the company master the token cost and the reasoning failover, but leaves search, the most expensive part of the loop, running on a single engine, with fixed depth and no cap. It is governing the smallest bill and ignoring the largest. The largest decides the risk.

The right-hand column is the one the Nexforce Router applies. The Router governs the cost and reliability of every agent call, including the search tool, with a spending cap per key, per project or per agent and with failover that moves traffic in milliseconds. It does not choose the engine for you. It applies the same routing policy it already applies to the LLM.

The practical argument closes in one sentence: whoever routes only the model is solving the problem of 20% of the cost and 20% of the risk, while the noisy invoice and the single point of failure of the loop sit in search. Extending the policy to the tool is the difference between governing the call and governing only its appearance.

How do you start applying tool routing to your agent's web search?

Starting does not require rewriting the agent. It requires changing one integration point and inspecting the tool. The path is incremental and fits in a sprint.

1. List the tools the agent calls today and measure the cost of each, starting with web search, which is usually the most expensive.
2. Unify the call entry point into a single gateway, which then sees the model and each tool called inside the loop.
3. Define the depth policy per question type, separating the routine query from the consequential query.
4. Configure engine failover for the degradation case, with a backup engine to absorb the switch in milliseconds.
5. Set a spending cap per key or per project that covers search, and not only the model.
6. Audit the real calls for a week and adjust the intent classification that decides depth.

Step six is what separates a design from a policy that works. The quality of tool routing comes from observing what the agent really calls, at what cost, with what failures, and adjusting the yardstick. Without call telemetry, the same kind of traceability that sustains LLM observability, routing is a bet with good arguments and no evidence.

And there is a gain that does not appear in the table. When the policy covers the tool, the same routing decision that saves on cheap search and invests on expensive search becomes the basis of a complete audit of every call. Each call is traceable, each selection criterion is explainable, and each failover leaves a trail. For a B2B buyer who needs to explain what the agent did, this is worth more than any quality percentage point.

Frequently asked questions about web search and routing in agents

Is the language model not what decides the quality of the agent's response?

The model interprets the context, but the search budget decides which context reaches it. A good model over shallow and badly chosen sources produces a confident and wrong response. Switching the model without touching depth leaves the biggest gain on the table: the isolated engine moves the result less than the model, and the depth of the budget moves more than both.

What is an agent's search budget?

It is the set of choices about how the agent collects information: which search engine it uses, how many pages and link levels it traverses, and how much context it carries into reasoning. Each level costs latency, tokens and, on paid engines, money per query.

Is web search inside the agent a tool that can be routed like the LLM?

Yes. Inside the loop, search is a tool exposed by function calling or by a connection protocol, with a signature and a cost. The gateway applies to it the same routing policy it applies to the model: cost, reliability, failover and spending cap.

Why does the Nexforce Router govern web search if it is an LLM router?

The Router governs the cost and reliability of every agent call, including the tools it calls inside the loop, and web search is the most expensive of them. It extends the same LLM routing policy to the tool, without becoming an agents product.

Do I need to rewrite the agent to route the search tool?

No. The path unifies the call entry point into the gateway, defines depth per question type, configures engine failover and sets a cap that covers search. The change is incremental and fits in a sprint.

References and Further Reading

The point about extending the routing policy to the agent's tools, treating call evaluation as a policy decision, speaks directly to endpoint evaluation as a routing policy. For those in the buying phase, the guide on how to evaluate and choose an AI gateway is the supporting read. The cost base this account rests on is in the survey on the operational cost of the gateway in production. To understand how an agent's tools enter the loop through a standard of message exchange between systems, the text on MCP and tools continues the chain. And the telemetry that makes step six work, tracking every call, is the heart of the LLM observability guide. Each one follows a thread of the argument.

A good model does not fix a bad context

There is a well-worn optimization hierarchy that almost every AI team learns the hard way. Swapping the model is easy, writing a better prompt is easy, and measuring how much information search left on the table is what nobody does. The same team that spends six weeks evaluating model vendors leaves the web-search engine locked in a line of code nobody revisits, and that is exactly where the worst cost and the worst risk of the agent's loop live.

Search routing is not an engineering whim. It is the same discipline the team already applies to the LLM, cost, reliability, failover, cap, only applied to the tool the loop uses most and spends the most on. The company that understands this does not need to choose between an expensive and a cheap engine, nor between high and low depth. It needs a policy that chooses for it, per call, looking at the intent, the cost and the consequence of each question.

The Nexforce Router exists for this: to govern the cost and reliability of every agent call, including the search tool, with the same policy that already governs the model. Whoever routes only the model solves half the problem. Whoever extends the policy to the tool starts governing the entire call, from intent to the last source touched. The next step is to change the integration point and start measuring. The data will do the rest.