Multi-model orchestration: half the cost, 2 points lower

Does multi-model orchestration trade quality for cost?

Artificial Analysis measured a cost of $5.59 per task for GitHub's composition, against $11.42 for Claude Code with Opus 5.5 in the maximum configuration. The orchestration landed two points lower in the index. Yes, it trades: the saving is real in the measurement, and so is the lower score.

The conclusion is not that mixing models always wins. It is that picking a model by its rank buys an average, while the bill arrives per task. The engineering lead needs to know whether the gap falls inside the work the team cannot get wrong.

The thesis is this: multi-model orchestration, or model composition, earns the trade only when the saving per task survives the acceptance test of the real workload. Without that test, a lower price is a proposal, not a decision.

What does the index actually measure?

The Coding Agent Index v2.0 gathers Terminal-Bench 4.0, FrontierCode and SWE-Atlas-QnA-AA, with equal weight for each benchmark. The aggregate result lets you compare configurations on the distribution the measurement chose. It does not say how much an organization will save, nor which error it will meet in its own application.

In the maximum configuration, Copilot Fusion in the GitHub Copilot CLI leads Terminal-Bench 4.0 with 71. In FrontierCode it sits third with 51, behind 56 and 54. In SWE-Atlas-QnA-AA it records 57, according to the Artificial Analysis measurement. The same aggregate result holds different positions depending on the test.

That detail changes the purchase. The benchmark has to represent the work, and one lead does not erase the disadvantage in the other tests.

Two points in the index are not two percent less quality. The relation between score and damage depends on the task, on the acceptance criterion and on the consequence of the error, none of it recoverable from a single aggregate average. Whoever turns that difference into a failure rate without measuring is selling precision the index does not offer.

What changes in model composition?

In model composition, the decision stops being concentrated in the early choice of one model for all the work. The system selects models during execution. That changes how spend and result are evaluated, but the specific mechanism has to be known before anyone attributes merit to any step.

The measured case is Copilot Fusion. GitHub describes the selection and coordination of Anthropic and OpenAI models at run time to balance performance, cost and latency per task, as Artificial Analysis reports. Token billing follows the models actually used to complete the request, not a single tariff for the composition.

An architecture that picks models step by step evaluates each pass by its own goal: read the input, produce a change, verify the result. That is a design possibility, not a description of the internal decisions of the product measured. The source does not publish the step-by-step breakdown, which limits any statement about AI model routing, and it does not authorize saying which model ran which part.

There is a difference between switching models and finishing well. The cheaper call may need a correction, another call or human review, and that goes into the bill. A routing error moves the spend to after the first result.

A trace can link the call to the execution context, but it does not prove the result was accepted, nor does it replace the provider invoice.

How do you compare the published numbers?

The comparison puts cost, consumption and benchmark results side by side inside the same Artificial Analysis measurement. The values describe the configurations tested, not a universal tariff and not the deployment cost in another company. Where the source gives only a relation between consumptions, the table keeps that relation.

Source of the table: Artificial Analysis, measurement published on October 28, 2026. The relative position in the index does not reveal the absolute values of the configurations.

The third configuration in the same measurement, Devin Fusion with Opus 5.5 in the maximum configuration and GPT-6 Luna on high, delivers 62 points at $8.00 per task, 43% above the $5.59 of GitHub's Fusion. Two composition architectures therefore sit in different cost bands, and the architectural label does not determine cost per task.

The measurement compares agent products, not gateway policies. A gateway picks one model per request. An orchestration inside the product issues several calls per task, with state. The agent harness provides tools, context and cache. The saving of an agent product does not size a gateway decision without its own measurement.

The consumption of 7.2 million tokens per task is roughly half the consumption of Opus 5.5 in Claude Code, according to Artificial Analysis. Fewer tokens help explain the difference, but the cause is not single: harness, tool design, cache reuse and model tariffs all compete for the explanation.

Which measurements separate real savings from apparent ones?

A useful comparison starts with the same workload, the same definition of acceptance and a bill that includes the work needed to reach the result. The published average guides the investigation. It does not replace the measurement of quality, cost per step, tokens and recovery when routing goes wrong.

1. Quality by task type. The owner splits tasks by relevant characteristics and defines who accepts the result, with verifiable criteria. Then the groups are compared, not the aggregate. The average hides failures in the most sensitive tasks.
2. Data eligibility. Before the price, the team marks the tasks that cannot go to one of the providers because of information classification, data residency or a contractual clause. The route that does not comply is out, however cheap it is.
3. Cost per step and per task. The team ties calls and attempts to the originating task, identifies the models used and applies the billing terms in force. When billing does not allow detailed attribution, the value stays unknown at that level, and unknown is not zero cost. Cost per task in model routing details that bill by route.
4. Tokens per task. Input and output, when available, should be tracked next to the model and the price rule. The count shows where consumption sits; it is not the invoice. Short and long tasks need a separate comparison.
5. Cost of the routing error. The team records rejection, correction, retry and human intervention, including the cost of unaccepted tasks. If the measurement excludes those cases, a route that fails and abandons work can look cheaper than one that finishes.

The formula for cost per accepted task requires declaring the cohort and the minimum size: add the costs of the tasks included, with rejected and abandoned ones treated per the defined policy, and divide by the number accepted in the same cohort. The account needs dispersion and an interval, because a small cohort turns noise into difference. With no accepted tasks there is no average; without reliable attribution, the account is partial.

Latency enters the same bill. If the route meets the quality criterion and finishes outside the process deadline, the discount does not restore the service. The buyer looks at the distribution of times, not the average.

When is a single model still the right choice?

A single model remains the right choice when composition does not reach the acceptance criterion, does not meet the required latency or does not offer enough evidence to attribute its costs. The advantage of a simpler route is not aesthetic: it reduces the need to justify intermediate choices without observability.

The objection to multi-model orchestration deserves to be taken seriously. A system that chooses between models introduces additional decisions and demands investigation when the result fails. If the measured gain is small against the cost of evaluation, integration and operation, keeping the existing configuration may be the correct economic decision. The route consumes its own call, and switching models invalidates the prompt cache.

Vendor concentration is the cost that does not show up in the tariff. An orchestration couples two or more contracts, with their own data terms, limits and price adjustments, and the selection policy is opaque by construction: it can be neither reproduced nor contested in an audit. Before adopting, ask for terms per vendor, a versioned policy and an estimated exit cost.

That does not turn the single model into an automatic winner. The Artificial Analysis measurement shows that multi-model orchestration reduced cost per task, with a bounded difference in the index, inside that distribution. For another workload, the answer has to come from your own test.

The decision should be reversible. Fixing the policy without monitoring the task types it serves turns a point-in-time evaluation into a permanent rule, and a change in the work mix requires redoing the account.

FAQ about multi-model orchestration

<!-- FAQPage -->

The useful doubt is not which label wins. It is what the measurement allows you to conclude and what the company still has to measure. An index helps choose the test. It does not define, on its own, the quality acceptable for a specific application.

Is multi-model orchestration always cheaper?

No. Artificial Analysis found $5.59 per task for GitHub's composition against $11.42 for Claude Code with Opus 5.5 in the maximum configuration, on the index tested. In another workload, consumption, errors and review are different, and the comparison has to be redone there.

Does the difference in the index amount to an error rate?

No. The two points of difference reported by Artificial Analysis are points of the aggregate index, not a percentage of quality and not an error rate per task. Operational risk depends on the distribution of work types and on the team's own acceptance criteria.

Do fewer tokens guarantee a lower cost?

No. Tokens are one part of the bill, and the price depends on the models used and on the billing terms. Artificial Analysis reports 7.2 million tokens per task for GitHub's composition, but does not detail the cost of each step nor prove the isolated cause of the difference.

How do you compare cost per task across routes?

The team should run the same workload with the same acceptance criterion, attribute the relevant costs to the cohort and declare how rejections and abandonments are handled. It also needs to compare latency, define the minimum cohort size and record unknown costs. An average proves nothing when the billing changes.

References and Further Reading

The primary source gathers the cost-per-task comparison, the token consumption and the benchmark results for the configurations evaluated. The limits of the measurement follow any application of that data outside the tested set. The references are these:

• Artificial Analysis: model orchestration, performance and cost per task.
• AI model routing in production: what changes when the route starts being chosen by task.

What decision comes before changing the route?

The IT team should select a representative workload, declare the acceptable result and compare composition with the single model in the same cohort, with minimum size, dispersion and interval. Only then does it decide whether the saving pays for the lower quality and the rework. An inconclusive pilot is valid.

The Nexforce Router places the discussion in routing infrastructure, not on a benchmark scoreboard. To evaluate it, or any alternative, the buyer should ask for evidence about the data needed for the company's own account and confirm what the implementation records. The measurement does not prove specific capabilities of the Router.

If composition does not meet the acceptance criterion for critical tasks, the team keeps the single model, as long as it meets it. If the workload is not eligible for one of the providers, the decision is made before cost. If it passes the criterion and reduces the cost of the cohort, the chosen route pays its own bill.