LLM speed without a price change: when the price freezes, the route moves

Seven models got faster in one week, none got more expensive, and for anyone deciding an LLM route in production that combination is more uncomfortable than it looks. That is seven models with a speed reading, out of the ten the panel tracks for price. Artificial Analysis, in the panel read on 2026-09-21, records output-speed gains between 13.0% and 26.0% across nearly every model it follows. Over the same period, the price per million tokens did not move for any of the ten tracked models. When cost per token stops discriminating, ties are no longer settled by price, and the variable left standing is response time.

The current read is that a speed metric for LLMs changes no contract at all. It does, and for an accounting reason. A route chosen by price is only optimal while price separates the options. On the day two models cost the same per million, the math ties, and what decides is latency, measured at peak hours, not in the panel average.

The finding in one sentence: with price per million tokens flat, latency and response variance become the route-decision variable, and a gateway that only compares price stops deciding anything.

How was the data collected?

The numbers come from two readings of the public Artificial Analysis panel in tokens per second, one on 2026-09-14 and one on 2026-09-21, at artificialanalysis.ai/leaderboards/models. That is seven models with a speed reading, out of the ten the panel tracks for price, and all seven appear in both readings. The percentage variation compares the two snapshots per model, and the price per million tokens was checked on the same panel with no change across any of the ten.

Two weekly snapshots are not a time series. They are one photo against another. A simultaneous jump across nearly every model is equally consistent with a change in the measurement window, a method adjustment in the panel, or a round of inference optimization at the providers. Nothing here distinguishes those hypotheses, and this piece does not pretend it does. What can be asserted with the data at hand is narrower and still useful: there was a measured output-speed gain, the price stayed flat, and the combination changes the question anyone routing has to answer.

One technical cut deserves naming, the panel does not settle on its own. Tokens per second is generation speed, the time between the first and the last output token. In a short-prompt call, what the user feels is something else: prefill, provider queue time and network latency dominate the total, and the generation gain shows up diluted. Measuring tokens per second and calling that latency is the confusion that makes a route decision look right on the chart and wrong in the product.

How much speed did the models gain in one week?

The short answer is that the gain ranges from 13.0% to 26.0% across the tracked models, and the largest percentage jump is not the largest absolute gain. GPT-5.6 Sol rose 26.0% from a low speed base. Gemini 3.8 Flash, already the fastest on the list, added 53.4 tokens per second in absolute terms, close to a whole second model of difference over the same interval.

The table below carries the pair of values per model. No price row appears because none moved: the variation column would be a string of zeros, and that is exactly the information that drives the thesis.

The pattern that jumps off the table is not the average. It is the asymmetry. The fastest model on the list got 19.2% faster, and the second fastest, 13.0%, which means a route chosen by raw speed on September 14 is still the raw-speed choice on September 21, without anyone having to swap anything. The aggregate gain did not reshuffle the speed ranking. It held the order and lifted everyone in the same direction, which is the behavior that motivates the measurement-window caveat.

Why does a flat price change the route-decision logic?

Because the tiebreak criterion disappears. A cost-based route works by comparing input and output price per million tokens. If two models cost the same, that criterion returns a tie, and the system deciding by price needs a second criterion it may never have implemented. The work that guides route promotion by traffic evidence already exists and is described in how to decide the LLM route with real traffic evidence, but it needs the right metric to measure. When the price is flat, that metric is response time.

The opposite case, price in motion, has its own post on this blog: what to do when the token price changes covers exactly the case where cost starts discriminating again. The two regimes complement each other. When the price moves, the math runs the decision. When it freezes, the math ties and latency takes over. No company operates in one of the two regimes forever, and the architecture that survives the switch is the one already measuring both variables all the time, instead of discovering one on the day it starts to decide.

There is a reason to distrust average speed as a criterion. The latency distribution of an LLM provider has a long tail, and the value the user perceives sits in the tail, not the center. A model averaging 68 tokens per second can have a p99 of four seconds on a peak afternoon, while another with a lower average never goes past two seconds. The average hides precisely the event that ruins the experience. Anyone deciding a route from a panel number decides from a statistic the user never sees.

What exactly should be measured, then?

Three things, and none of them is the panel's average tokens per second. Time to first token, which defines the feeling of immediate response. Total time to last token, which defines whether the task finishes. And the distribution of those two metrics by hour, with p90 and p95 made explicit, because it is in the tail that the wrong decision gets expensive.

1. Time to first token (TTFT). The interval between sending the request and the first output fragment. It dominates perception on short prompts and is what average tokens per second ignores by construction.
2. Total time to last token. What decides whether a long task, such as generating a report or reviewing a code file, finishes within the operation's time budget.
3. p90 and p95 by hourly window. The metric that separates a stable provider from one that works well at ten in the morning and badly at three in the afternoon, when the queue fills.

The detail almost no one instruments is the variance across providers of the same model. The same model served by two different providers has two distinct latency curves, because the hardware configuration, the batching policy and the region load are not the same. Choosing the model is not choosing the route, the same way choosing the destination is not choosing the road. The Artificial Analysis panel measures one reference provider, and treating that number as if it held for every provider is a mistake that only shows up in production.

It is worth stating where this text disagrees with the most common read. The current read treats latency as a user-experience metric. That underestimates the problem. Latency, when the price is flat, is an operational-risk metric: a long tail on one provider can blow the timeout of an agent step, burn an entire attempt and reprocess the call, and the cost of that reprocessing shows up on the invoice through a path nobody mapped. The savings the flat price promised are handed back as rework.

Does the gateway that only compares price decide anything when the price is equal?

It decides nothing. And that is the point where the gateway layer stops being an infrastructure detail and becomes the piece that carries the decision. Nexforce Router is an LLM gateway that routes across more than three hundred models through a single API, and its intelligent routing considers cost, performance, latency and request context. When cost ties, that combination is what keeps producing a decision instead of a technical tie.

What matters is not the feature list, it is the order in which those features operate under a flat price. Measure latency per provider and per model, apply routing by measured performance, and run automatic failover when a provider slows down mid-window, with configurable fallback to an equivalent model. Failover stops being a safety net against an outage and becomes a response to response-time degradation, a failure mode that is quieter and more frequent than full unavailability. The gateway also keeps a cache of responses and embeddings, which attacks latency by another path: the fastest response is the one that does not have to be generated again.

There is a second job the gateway does once the route is decided by response time. It has to prove afterward why it decided what it decided. An auditable trail of every call, with the provider, the model and the latency logged, is what lets a team answer in an operations meeting whether the route degraded on Tuesday afternoon or was always like that. Without that record, the route decision becomes folklore. With it, it becomes input for the next route promotion.

One point deserves insistence: adding latency as a criterion does not retire cost control. The spend ceiling still holds and is still what keeps a faster and more expensive route from consuming the budget. Budget per unit, covered in caller identity in the gateway and budget per unit, is what keeps the latency decision inside an economic limit. Latency routing without a spend ceiling is an elegant way to trade a 50% saving for a larger invoice.

Where does cost per task come in when the price ties?

It comes in as the unit that reconciles the two metrics. Price per million tokens is an input unit. Cost per completed task is an output unit, and it is the one that captures the effect of rework. A cheap and slow model that blows the timeout and forces a second attempt can cost more per task than an expensive and fast model that closes on the first call. The analysis in cost per task decides the route goes deeper on that unit, and it is the final judge when the price per token ties.

The practical consequence is an inversion of priority. The priority inverts. On a price dashboard, the expensive model is the default suspect. On a cost-per-task dashboard, with a flat price and latency in play, the model that looks expensive can be the cheapest on the list, because it finishes the task and does not hand the problem back to the queue. Measuring by task is measuring what the company actually pays.

How do you decide the route when the price does not decide?

A concrete sequence, in the order the decision resolves:

1. Confirm the price tie on the panel, per model and per provider, before treating latency as the tiebreak. If the price still discriminates, the math runs the decision and latency is secondary.
2. Instrument TTFT, total time and p95 by hour, per provider and per model, on your own production traffic. A third-party panel number is a starting point, never the final criterion.
3. Define the time limit the task tolerates before it fails. It is that limit, not the average, that turns latency into a routing rule.
4. Promote the route on accumulated evidence, with failover configured to the fastest equivalent model available at the moment of degradation.
5. Keep the spend ceiling active per key or per project, so that latency routing does not turn into an invoice with no brake.

The most common mistake in that sequence is skipping step two and deciding from the panel number. The panel is not the company. The panel measures one reference provider in a fixed window. The company's traffic measures the provider it actually uses, at the hour it actually uses it, with the load it places on that provider over the course of the day. The difference between the two is where the decision lives.

What changes for anyone running LLMs in production?

The question changes, not the tool. The question stops being which model is cheapest per million and becomes which model finishes the task inside the time the operation tolerates, at the lowest cost per completed task. That question has no panel answer. It has an answer from continuous measurement on the route itself, and that answer only exists if the layer making the decision is measuring response time all the time, not only when someone opens the speed chart.

The week's data reinforces the thesis and limits it at the same time. The gains from 13.0% to 26.0% are real as a reading, and the measurement-window caveat still stands. A team that decides a route treating those numbers as a verdict treats a seven-day photo as if it were a permanent regime. A team that measures its own traffic, by contrast, uses the photo only as a signal and its own data as the decision criterion. The second team does not depend on it. It does not need to know whether the jump came from the provider or from an adjustment in the measurement window, because its decision rests on its own traffic either way.

That is why the piece lands on the gateway. Latency routing is not a feature you switch on. It is a posture of measuring, deciding and reversing on top of data that changes every week. A flat price is not the end of cost optimization. It is the beginning of time optimization, and that only works if the decision layer is watching the clock, not just the invoice.

Frequently asked questions

What does a flat price per million tokens mean? It means the tracked models kept the same input and output price per million tokens between the two readings, on 2026-09-14 and 2026-09-21. With no price difference, the cost criterion ties between options of different speed, and the tiebreak starts coming from response time.

Is output speed in tokens per second the same thing as latency? It is not the same thing. Tokens per second measures generation speed between the first and the last token. Perceived latency includes time to first token, provider queue time and the network. On short prompts, perceived latency is dominated by those other factors, and the generation gain shows up diluted.

Why does a measurement-window change matter for this analysis? Because two weekly readings do not distinguish a real speed gain from an adjustment in the panel method. A simultaneous jump across nearly every model is consistent with both hypotheses, and no company should change its routing architecture based on two photos. The caveat is part of the finding, not a footnote.

Does latency routing replace cost control? No. The spend ceiling per key, per agent or per project stays active and is what keeps a fast and expensive route from consuming the budget. Latency and cost per task operate together: latency decides between options of the same price, and the spend ceiling imposes the economic limit of the decision.

How many providers can the same model have, and does that affect measurement? The same model can be served by several providers, and each one has its own latency curve by hardware, batching and region. The third-party panel measures one reference provider. The measurement that decides should come from the provider the company actually uses, at the hour it uses it.

References and Further Reading

• Artificial Analysis, model leaderboards, read on 2026-09-21, primary source of the tokens-per-second data.
• Nexforce Router, LLM gateway with intelligent routing, automatic failover and call observability.
• How to decide the LLM route with real traffic evidence, the method for promoting a route on your own data.
• What to do when the token price changes, the opposite case, with price in motion.
• Cost per task decides the route, the decision unit when the price per token ties.

Next step: measure the clock, not just the invoice

The 2026-09-21 data says the price stopped and the speed rose. What it does not say, and no third-party panel says, is the response time of the provider your operation uses at three in the afternoon on a Tuesday. That number lives in your traffic, and the layer deciding the route is where it has to be captured and turned into a rule. While the price per million stays flat, whoever measures response time decides; whoever only compares price ties. The clock decides. To see how Nexforce Router handles latency, failure and cost in the same decision, the product page is the starting point.