DeepSeek open-sources DeepInfer, the inference framework behind DeepSeek-V4.1-Flash
Archived — this story has rotated out of today’s deck. It is kept here in full.
The gist
Chinese AI lab DeepSeek released its production inference framework DeepInfer under the Apache 2.0 license.
It runs on Nvidia and Huawei chips; outside benchmarks of its gains are pending.
Background
DeepSeek is a Hangzhou-headquartered Chinese AI lab and start-up that developed both DeepInfer and the DeepSeek-V4.1-Flash model it serves. DeepInfer is described as the high-performance framework that actually powers DeepSeek's large-scale production services, not a stripped-down demo. The release is framed as the latest escalation in a race where DeepSeek gives away tools that OpenAI and Anthropic treat as closely guarded trade secrets. It also follows last month's DeepSeek-Huawei partnership to build programming tools for Ascend chips.
How it unfolded
- Oct 20, 2026DeepSeek released DeepInfer, its in-house inference framework, under the Apache 2.0 license; it supports DeepSeek-V4.1-Flash and runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.
- Oct 20, 2026DeepSeek also released deepseek-recipe, its MIT-licensed token encoding and decoding layer.
- Oct 20, 2026DeepSeek said on X, according to Bloomberg, that it is sharing its production stack to help the community lower compute costs and make AI more affordable.
- Oct 20, 2026The release provides offline inference only and leaves out a KV cache system, prefill and decode disaggregation, and load balancing across data-parallel replicas, which users must supply.
- Oct 20, 2026DeepSeek reports a single cluster serves 155 trillion tokens per day and overall revenue of US$38 per GPU-hour at valley pricing; decode tops 25,000 tokens/s/GPU on 64 GPUs in self-reported benchmarks.
Who’s saying what
- Official
- DeepSeek says it is sharing its production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
- Caution
- Bloomberg notes no outside party has yet benchmarked DeepInfer's efficiency gains, so the performance figures remain self-reported.
Still unverified
DeepInfer's efficiency gains have not been benchmarked by any outside party, according to Bloomberg. Whether the release pressures OpenAI or Anthropic into any specific public response remains to be seen, and nothing in the current record suggests either company has signaled it will.