NVIDIA brings NVFP4 to DeepSeek V4.1-Flash: GPU cost

NVIDIA published on October 30, 2026 an NVFP4 variant of DeepSeek-V4.1-Flash-1017, a MoE with 552 billion parameters in the backbone and context of up to 1 million tokens. The card compares checkpoints of approximately 400 and 475 GiB. On GPQA Diamond, the editorial calculation points to +0.16 percentage point. Less storage does not guarantee savings on GPU.

The checkpoint is the data set that records the weights and other components of the model. The difference between the sizes equals approximately 16% of the original checkpoint. The card publishes both totals and does not break down that difference nor attribute it to the conversion of the experts. Freed capacity only has operational value when there is a compatible workload and room within the latency limit. On GPQA Diamond, the card reports 92.77% for the original and 92.93% for the variant; the difference is an editorial calculation, with no conclusion of statistical significance.

Before changing a production route, the operation needs to measure answer quality and the cost of keeping the engine compatible.

What did NVIDIA publish about NVFP4 on October 30?

NVIDIA published a quantized version of DeepSeek-V4.1-Flash-1017 that uses NVFP4, a low-precision numeric format, in the routed experts, and prepared it with NVIDIA's Model Optimizer. The conversion was not uniform: the card describes different conversion paths for different components.

The card uses the word quantization for two distinct operations. The Engram tables lost bit width, from FP8 to MXFP4. The experts swapped one 4-bit format for another 4-bit format with a finer scale, from MXFP4 to NVFP4. Only the first one removes bits.

In a mixture of experts, or MoE, execution selects experts, but the remaining weights stay in the checkpoint. According to the card, the 384 routed experts per layer across the 40 layers moved from MXFP4 to NVFP4, with W4A4 weights and activations and quantization groups of 16 values.

The Engram tables moved from FP8 to MXFP4; other components kept the original precision. According to the card, the expert weights preserved dequantized values, with signed-zero canonicalization. The Engram conversion was lossy; preserving the weights does not guarantee identical answers.

Authorship also has clear boundaries. The original model card belongs to DeepSeek AI, and NVIDIA states that it did not develop or own the model. NVIDIA published the quantized variant, built for a third party's requirements, and declares an MIT license and commercial and non-commercial use.

How much space does the checkpoint take, and why does that affect GPU cost?

Consulted on November 2, 2026, the NVIDIA card compares approximately 400 GiB in the quantized checkpoint with 475 GiB in the original. The difference of approximately 75 GiB equals about 16% of the original size and can reduce expenses if the deployment uses less paid hardware. Size on disk does not prove savings.

The card publishes both totals and does not publish the breakdown of the 75 GiB. The calculation we did shows why the conversion of the experts cannot explain that difference: MXFP4 and NVFP4 are 4-bit formats with the same E2M1 element, and NVFP4 uses an FP8 E4M3 scale every 16 elements, against an E8M0 scale every 32 in MXFP4. That more frequent scale costs 4.5 bits per element in NVFP4 against 4.25 in MXFP4, or about 6% more per element (4.5 divided by 4.25), and not less. Applied to the 552 billion parameters of the backbone, the same calculation gives about 293 GB in MXFP4 against about 310 GB in NVFP4, that is approximately 17 GB more in the experts, and never 75 GiB less. Those volumes are in decimal GB; the checkpoint sizes published in the card are in GiB.

The only conversion that removes bits is the one on the Engram tables, from FP8 to MXFP4, and it is also the one the card declares lossy. That is the plausible explanation for the published reduction.

The total parameter count explains the file size. The card declares 552 billion parameters in the backbone, 8 billion activated in prefill, 16 billion in decode and 196 billion in conditional Engram memory. Weight traffic per token follows the activated routed experts; resident memory follows the total plus the cache. That is why 552 billion parameters at 4.5 bits per element occupy about 289 GiB, and the 196 billion of Engram in MXFP4 add about 97 GiB, values coherent with the approximately 400 GiB published. Averaged over the file, that gives about 0.78 byte per backbone parameter, in our calculation.

The comparison separates checkpoint size from price per token: the cards consulted do not publish cost per million tokens, and size does not determine the hardware required.

The table gathers the formats and sizes declared in the cards, plus the context window of the original model; it does not report runtime consumption or contracted price.

Another measure is GPU memory at runtime. Beyond the model components loaded on the device, the engine reserves space for cache, buffers and structures the requests need; weight distribution, copies and offloading to the CPU change the arithmetic. That is why dividing the checkpoint size by the nominal memory of one GPU does not determine a production configuration.

For the expense, the capacity limit matters. When the measured deployment can use less reserved hardware, there is a possibility of cost reduction; when it keeps the same set of GPUs, the freed space can serve other workloads or increase concurrency, as long as the configuration supports the usage and the required latency. If there is no use for that capacity, the reduction on disk does not necessarily turn into savings.

Even when the workload does not use all the available memory, the reserved hardware makes up the expense. The comparison needs to include processing time and the tasks completed with quality; the card does not quantify savings for a specific operation.

Did accuracy hold up on the benchmarks?

The numbers in the NVIDIA card were reported by the publisher across six evaluations and do not demonstrate statistical equivalence, because error margins are missing to determine the relevance of the differences. The results of the original and quantized checkpoints are close, but they were not presented as an independent measurement.

Source: NVIDIA model card, consulted on November 2, 2026. Results go up or down. The tests cover science, long context, code, instructions, vision and terminal tasks, without replacing validation of the application.

The methodology varies. The card reports different numbers of repetitions and a separate configuration for Terminal-Bench 2.1; there is no support for attributing the same variability to the tests.

The card also reports that 24 of 46,080 projection entries used fallback activation scales. The conversion needed a fallback scale on real projection entries, and those entries are exactly what prevents treating the result as a faithful copy of the original.

The team needs to test both variants on representative tasks. The acceptance criteria need to be the same. Human corrections or new calls change the cost per task; the publisher does not measure that effect.

What is the operating cost of the patched vLLM build?

The card points to a vLLM build with the changes from the support pull request to load the Engram tables in MXFP4. The operation needs to test the build, track updates and plan security fixes. The publisher does not quantify that work, and the documented requirement does not anticipate support for future versions.

Maintaining that build takes work. The reported results use the image vllm/vllm-openai:v0.31.0 with the pull request applied, and the checkpoint ships in 48 shards; the pull request was open and unmerged on November 2, 2026. The team needs to obtain or build a compatible image, record dependencies, test updates and define how to receive security fixes; it also needs a rollback path if an update prevents the checkpoint from loading or changes application behavior. The card does not assign a financial value to those activities.

The published reference uses four NVIDIA GB300 Blackwell GPUs, vLLM with FlashInfer MLA sparse attention and offloading of the Engram tables to the CPU. That is the configuration of the reported results, not a universal capacity recommendation. Nor does it allow attributing the whole checkpoint size to GPU memory.

The card declares microarchitecture compatibility with NVIDIA Blackwell and declares no support for any other. Outside it, the comparison does not apply without a hardware swap; the card does not describe behavior on other architectures, and this article does not infer any.

SGLang, another engine cited in the card, was not validated with the checkpoint. That does not prove incompatibility, but it requires the operator to verify loading, response under load and failure recovery before assuming support.

What changes in inference routing?

A smaller checkpoint only changes the buyer's cost if the deployment or the terms of the service turn that reduction into a measurable advantage. For the operator of the GPUs, the comparison includes reserved resources, effective usage and maintenance. For whoever buys inference, it depends on billing and on the calls needed to finish the task.

The hardware restriction comes before the memory arithmetic. The card declares support for NVIDIA Blackwell only, so a previous-generation fleet, such as Hopper or Ada, only benefits from this variant after deciding on a hardware refresh, not on swapping the checkpoint.

The pass-through is not automatic.

A route with lower cost per call may require more attempts or corrections to deliver the same result. In that case, the cost per accepted task can rise, despite the smaller checkpoint size. If quality holds and the operation uses the freed capacity, the team has a reason to revisit traffic distribution, respecting latency limits.

The production comparison requires records for both variants:

1. Measure GPU and CPU memory, execution time and utilization under the same load.
2. Evaluate answers with the same criteria and record failures, repetitions and corrections needed.
3. Divide the cost of serving or buying the calls by the number of accepted tasks, including build maintenance when the operation controls the engine.

Further reading: the DeepSeek-V4.1-Flash-1017 launch; DeepInfer. The comparison is the cost of resources per accepted task.

Comparing routes requires cost per accepted task, quality and latency measured under the same load. Availability of this variant and its price must be confirmed with the provider. No savings for the customer is demonstrated.

Frequently asked questions

What is NVFP4?

NVFP4 is a low-precision numeric format used in the weights and activations of the routed experts of the variant published by NVIDIA. The conversion did not cover every component of the model; the card describes different formats for the Engram tables and the remaining components.

Are the 400 GiB the memory needed to run the model?

No. The approximately 400 GiB correspond to the checkpoint size declared in the NVIDIA card, consulted on November 2, 2026. Memory at runtime includes other reservations, such as cache and buffers, and depends on how the components are distributed between GPU and CPU.

Does the 16% reduction guarantee savings in the same proportion?

No. The approximately 16% derive from the approximate sizes of 400 GiB against 475 GiB for the checkpoints, not from a cost measurement. Reserved hardware, utilization, billing, task quality and build maintenance all influence the financial result in production.

Does the variant work with any vLLM installation?

The card says that standard vLLM images do not load the Engram tables in MXFP4 for this checkpoint and points to a build with specific support. The team must validate the chosen installation; that statement does not allow anticipating support for future versions.

How can availability and price of the variant be confirmed?

Availability and price of this variant must be confirmed with the provider. Before including it in a routing policy, the buyer needs to check whether a route is available, what the corresponding commercial terms are and the cost per accepted task.

References and Further Reading

The NVIDIA Model Optimizer repository documents the tool used in the quantization. The cards linked at the top of the article describe, respectively, the variant published by NVIDIA and the original model from DeepSeek AI; the consultation for this article took place on November 2, 2026.

The cards do not replace your own tests. The configuration and the build requirement must be evaluated under the load and the acceptance criteria of the application.

Where GPU savings meet CPU cost?

In the reference configuration described in the NVIDIA card, the Engram tables are offloaded to the CPU. The bill does not end at the GPU. So, for whoever pays for GPU capacity, a reduction in that expense can be offset by the cost of the CPU capacity that inference requires, depending on the contract and the load.

If the 16% reduction comes from the Engram conversion, it materializes in host memory and on disk. The card demonstrates a smaller checkpoint; it does not demonstrate a smaller HBM bill. And the bill that remains starts with the architecture: according to the card, without Blackwell this checkpoint does not load.