Cost per request: 26% of traffic is an AI decision

A quarter of the traffic through a production gateway is already decision, not text generation: in October 2026, 26% of the requests went to decision models, which accounted for 0.53% of spend. The data moves the question. The AI cost per request, not the price per token, is what decides the invoice.
How was the index measured, and where does it stop?
The index measures anonymized, aggregated traffic from a gateway, above 100 trillion tokens per month, with the series closed in October 2026. Spend is estimated at list prices. It is not the AI market, and it is not the invoice of whoever reads this text.
The data comes from the November 2026 production index, published by Vercel on November 9. Two limitations matter. The perimeter is one gateway, not the market. And the money is an estimate at list prices, because negotiated discount, annual contract and commitment credit do not enter the public price table the source uses to estimate spend.
Neither limitation invalidates the reading.
Table 1. What the index measures and where it stops
| What | Unit | Limit |
|---|---|---|
| Requests by model class | % of total requests | Only the traffic that passes through this gateway |
| Spend by class | % of total spend | Estimated at list prices, not an invoice paid |
| Period | Monthly series, closed in October 2026 | Published on November 9, 2026 |
| Base | Above 100 trillion tokens per month | Aggregated and anonymized, with no view of the reader's load |
A mix portrait does not need to be an invoice to say which share of the volume and which share of the money each model class occupies. That proportion is what survives the fact that the books belong to a vendor.
What the index shows: 26% of requests and 0.53% of spend
Decision models went from zero in August to 26% of all requests in October, and accounted for 0.53% of spend. In the same move, language model requests fell by more than a fifth. A new class now carries a quarter of the volume.
A decision model does not write text: it classifies, scores or chooses between options. The class entered the gateway in mid-September and reached October with 15 models from 8 vendors.
The three numbers of the month:
- Decision models accounted for 26% of requests and 0.53% of spend, against zero in August.
- Language model requests fell by more than a fifth in the same move.
- 11% of paying teams used a decision model without sending a single request to a text-generating model.
That is the strange number of the series. There is AI operation in production that generates no text at all, and almost a third of paying teams already used a decision model in production in October.
Table 2. Requests against spend by class, October 2026
| Class | Share of requests | Share of spend |
|---|---|---|
| Decision models | 26% | 0.53% |
| Language models, embeddings, reranking and media | 74% (derived by subtraction) | 99.47% (derived by subtraction) |
The chart is the whole piece: a quarter of the requests and half a percent of the money. Whoever looks only at spend concludes that decision is irrelevant; whoever looks only at requests concludes the opposite.
How much does a decision request cost compared with a generation one?
The arithmetic closes an order of magnitude: a decision request costs about 66 to 70 times less than a language model request that does the heavy generation. The source publishes that this cost is about one seventieth. The two readings, one calculated here and one published, point to the same place.
The arithmetic uses 100 requests and 100 units of spend. The decision class takes 26 requests and 0.53 of spend: 0.53 divided by 26 gives 0.0204 per request. The remainder takes 74 requests and 99.47: 99.47 divided by 74 gives 1.3442. The ratio between the two is 65.9, or approximately 66.
About 66 to 70 times: the percentages are rounded and the residual carries embeddings, reranking and media. Order of magnitude, not measurement.
The second account is narrower. Almost all of the decision request volume went to Jev, from TypeSafe AI, with almost four times the request volume of Anthropic. For every US$ 1 spent on Jev, teams spent US$ 64 on Anthropic models. Four requests per US$ 1 give US$ 0.25; one per US$ 64 gives US$ 64. With the two approximations of the source, the ratio sits at about 1/256.
Table 3. Relative cost per request between two routes
| Route | Requests | Spend | Cost per request |
|---|---|---|---|
| Jev, from TypeSafe AI | Almost 4 times | US$ 1 | About US$ 0.25 |
| Anthropic models | 1 | US$ 64 | About US$ 64 |
| Ratio between the routes | About 1/256 |
With more than two orders of magnitude per request, choosing a route by the price table per million tokens measures the variable that weighs least. The model that returns only decisions explains why the class exists; the account above explains why it changes the invoice.
The cost per request that the token price does not include
The inference cost per request has five components: input and output tokens, retry amplification, fallback delta, operation of the routing layer and amortized observability. The price per token table covers the first and ignores the other four, so a table that covers only the first describes a fraction of what the invoice charges.
The second component, retry amplification, is the most ignored one. Each new attempt pays in full, and a request that fails twice before succeeding costs three times what the table promises. Cost per attempt turns retry into money, and the record has to prove the attempts of each request.
The third is the fallback delta: the difference between standby and primary, applied to the fraction of requests that migrate. The standby is rarely the cheapest, because it exists for the day the primary does not answer. Model fallback criteria say when the migration happens; the account says what it costs.
The fourth is the LLM gateway overhead: normalization, intent classification, model selection, distribution and failover, with a fixed part and a part that follows the volume. The fifth is observability: traces, evaluations and storage, divided by the requests of the period. Amortized, it looks small; not amortized, it becomes a platform line nobody associates with inference.
The formula:
cost per request = (t_in × p_in + t_out × p_out) × r + f × delta + c + o
t_in and t_out are input and output tokens, p_in and p_out the prices per token of each route, r the average number of attempts, f the fraction that falls on the standby, delta the cost difference between standby and primary, c the cost of the layer and o the amortized observability. No term has a universal value.
Table 4. The five components of cost per request
| Component | What enters | Who controls it |
|---|---|---|
| Input and output tokens | Volume consumed per route and published price | Model vendor |
| Retry amplification | Each new attempt pays in full | Application engineering |
| Fallback delta | Difference between standby and primary, times the migrated fraction | Availability policy |
| Routing layer | Normalization, classification, selection and failover | Platform operation |
| Observability | Traces, evaluations and storage, amortized | Platform operation |
This article converts traffic mix into an account per request; latency, memory and the overhead of the layer are in the operational cost of an AI gateway.
Why the adoption cohort changes the number
Adoption rises with volume: a little more than one in five teams below 10 million tokens per month uses a decision model, almost half between 100 million and 1 billion, and seven in ten above 10 billion. Below the first step, the layer tends to be a new fixed cost.
The ladder describes a threshold. Whoever runs 10 million tokens per month has the volume to justify a classifier and does not have the volume to dilute the cost of keeping the layer. The account changes sign as the denominator grows.
Table 5. Decision model use by volume cohort
| Token cohort per month | Teams that used a decision model |
|---|---|
| Below 10 million | A little more than one in five |
| Between 100 million and 1 billion | Almost half |
| Above 10 billion | Seven in ten |
Model concentration changes with size. The 1% that spends the most uses a median of 29 different models. The tenth that spends the least uses one.
There is a reading of the index that runs against the conclusion it sells. The gateway vendor sells routing, and the easy reading is that routing always pays. The ladder contradicts it: at the base, most teams do not use a decision model, and the probable explanation is the arithmetic of a small single-model load, where the new layer costs more than it returns.
The six-week half-life changes the question
More than half of the language requests in October went to models less than 60 days old, against fewer than three in ten before September. The source sums the effect in one expression: a half-life of about six weeks. A frozen routing policy, in this regime, is a cost decision by omission.
Turnover has its own numbers. A new version arrives every 61 days in the family median, a typical model reaches its peak in about a month, and four weeks after the peak three in four teams that used it had already migrated. Half of the teams with 10 million tokens or more changed their most used model in October alone, and six in ten went to another lab.
Table 6. Model turnover in October 2026
| Indicator | Value |
|---|---|
| Language requests on models less than 60 days old | More than half, against fewer than three in ten before September |
| Interval between versions in the family median | 61 days |
| Time to the peak of use of a typical model | About one month |
| Teams that migrated four weeks after the peak | Three in four |
| Teams with 10 million tokens or more that changed their main model in October | Half |
| Migration to another lab | Six in ten |
The denominator the buyer measures in January is not the same one from March. A price per token frozen in a twelve-month contract ages faster than the model it describes, which is why the test below repeats the measurement in 30 days.
The five-step test before closing the routing architecture
The test uses the load itself as a sample and takes five steps: count requests by class, calculate the full cost, measure retry and failover, compare the accuracy of the classifier on both model types and repeat in 30 days. It fails if the classifier shows no gain or if retry consumes the difference.
Each step has a verifiable output and a failure criterion. Without both, the test turns into a meeting.
- Count requests by class over seven days, separating decision, generation, embedding and media, with a trace per call. Without an LLM call trace, the aggregated log hides the class and the retry. Fails if the count does not separate the classes.
- Calculate the full cost per request with the five components, using the invoice actually paid, not the table. Fails if the number comes from the pricing page.
- Measure the retry rate and the failover rate and convert them into extra requests. Fails if the amplification consumes the difference between the new route and the current one.
- Run the classifier on both model types over a labeled sample of the operation itself and compare the accuracy. Fails if it does not beat the rule already in the code.
- Repeat in 30 days and record what changed. Fails if the difference between the two measurements has no identifiable cause in the load, because that indicates the measurement did not capture what changed in the operation.
The test does not decide whether routing models by task is good. It decides whether the route the buyer is about to contract is better than the one already in place, with their own load. A failing result is information.
What this changes for buyers in Brazil
For whoever pays in reais, the cost per request is the unit that survives the exchange rate and the import taxes, because both enter the same denominator. The invoice in reais with a tax document is what allows comparing the list-price estimate with the outlay of the month.
A price per token in dollars says nothing about cash until it passes through exchange rate, taxes and payment date. The cost per request, measured on the invoice that arrived, has already passed through all of it. It changes currency. It does not change logic. It is the line the buyer defends in front of finance.
This is where Nexforce Router enters, as a route decision product, not as the source of any number in this article. Router exposes the main model families behind an API compatible with the one the application already consumes, with routing by cost, performance, latency and context, automatic failover, a spend cap per key and a trace of each call. The index is third party and does not measure Router; Router delivers the place where the routing policy stays written, with an owner and with a limit.
Frequently asked questions
Does cost per request replace cost per token?
It does not replace it, because the two accounts answer different questions. Cost per token is the raw material of a route; cost per request is what the invoice charges per request, after retry, fallback and layer. The accounts of 66 to 70 times and of about 1/256 only exist per request.
Does the index apply to my load?
It applies as a reference for mix, not as a forecast. The 26% of requests and the 0.53% of spend describe aggregated traffic from one gateway, estimated at list prices. Applying those percentages to a specific company only makes sense after counting requests by class in its operation.
Does a decision model serve any stage?
It serves stages with a discrete answer, such as classifying, scoring or choosing between options. It does not serve to write the final answer to the customer. The index shows 11% of paying teams using a decision model with no request at all to a text-generating model, which points to entire operations built on decision.
References
Primary source consulted on 2026-11-16:
- November 2026 production index (Vercel, 2026-11-09): https://vercel.com/blog/ai-gateway-production-index-november-2026
Further reading, consulted on 2026-11-16:
- Operational cost of an AI gateway: https://nexforce.ai/en/blog/measure-ai-gateway-operational-cost
- Cost per attempt: https://nexforce.ai/en/blog/ai-inference-cost-per-attempt
- TypeSafe launches Jev: https://nexforce.ai/en/blog/typesafe-jev-calibrated-decisions-llm-routing
- AI model fallback: https://nexforce.ai/en/blog/ai-model-fallback-acceptance-criteria
- LLM call trace: https://nexforce.ai/en/blog/llm-call-trace-audit-request-token
The index is monthly and revises the previous months, so the percentages on this page may change in the next edition.
The account that changes the next architecture meeting
The index does not say that routing pays. It says that a new class accounts for a quarter of the requests with half a percent of the money, and that the unit able to explain that difference is the request, not the token. Whoever walks into the meeting with the account per request in hand discusses cost. Whoever walks in with the price table discusses price, and leaves the room believing they decided.

Accelerate your company'sbusiness and operational efficiency
We design the technology of tomorrow to boost your business operational scale
Talk to a SpecialistRelated articles

Gemini 4 Hydrogen: flat pricing and cost per request
Google DeepMind released Gemini 4 Hydrogen on 12 November 2026 with a flat price across the entire 1 million token window. With a constant rate, the cost of a request depends only on input and output volume, and comparing models no longer fits inside a price per million.
Read more
Pix installments at SaaS checkout: cash, timing, chargeback
A comparison of the three receiving rails (card installments, credit at Pix initiation and installment boleto) from the point of view of whoever sells subscriptions to Brazilian companies: payout timing, early settlement cost, dispute risk and reconciliation effort, with the Central Bank rules cited at the official source and an enablement test for finance.
Read more
Pipedrive pricing in Brazil: seats, limits and tax invoices
Pipedrive bills per seat, even when empty, and caps automation per company. Check published prices, subscription exclusions and first-year costs in Brazil.
Read more