DeepSeek Open-Sourced DeepInfer — and Gave Huawei's Ascend First-Class Support | CloudEmythos

Illustration: CloudEmythos. DeepSeek's DeepInfer supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs as first-class targets — a first for a frontier lab.

TL;DR — On October 20, 2026, DeepSeek open-sourced DeepInfer, the inference framework it says powers its own production AI services, under the permissive Apache-2.0 license. The headline nobody else is writing: the release gives Huawei's Ascend 950 the same first-class support as Nvidia's SM100 Blackwell GPUs — the first time a frontier lab has done that. It is built on vLLM, uses speculative decoding, and drew thousands of GitHub stars within hours. But the performance figures are DeepSeek's own claims, not independently tested — and it runs one model, offline only. Here's the honest breakdown.

Disclosure: this article contains no affiliate links. Every claim below is sourced from DeepSeek's announcement, its GitHub README, and independent coverage (SiliconANGLE, aiweekly, MarkTechPost, Tesla News Wire), linked in-line. Self-reported figures are labeled as such.

What Released

DeepSeek announced DeepInfer in a four-post thread on X on October 20, 2026: "our in-house inference framework, now fully open source!" The lab said it is "sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable," and that inference code for future models will land in the same repository.

The GitHub README describes DeepInfer as "the high-performance inference framework that powers DeepSeek's large-scale production services." It ships under Apache 2.0 — meaning anyone can use, modify, and sell it. Alongside it, DeepSeek also open-sourced deepseek-recipe, its MIT-licensed production token encoding and decoding component that matches the official API.

What's in the first release

One model: DeepSeek-V4.1-Flash, running in offline inference mode only. A KV cache system, prefill/decode disaggregation, and load balancing across replicas are not included — so this is not a complete serving stack yet. Two chip families: Nvidia SM100 GPUs (CUDA 13) and Huawei Ascend 950 NPUs (CANN 9.2). Built on vLLM: the README says DeepInfer was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM."

That last line matters. Eighteen months ago, in April 2025, DeepSeek said it had considered releasing its full inference engine but decided against it — the code was a heavily customized early fork of vLLM, tightly coupled to internal cluster tools, and the small team lacked the bandwidth to maintain a big open-source project. It promised to contribute pieces instead. DeepInfer is the reversal: the whole engine, not just the pieces.

Why this is a bigger deal than another open-weight model: Labs open-source model weights all the time. Opening the serving stack — the kernels, schedulers, and samplers that make inference cheap at scale — is far rarer. Inference is the part of AI that happens after training: serving answers to millions of people quickly and cheaply. DeepSeek's famously low prices have always rested on how well it does this. Now everyone can read the engine.

The Huawei First-Class Story

Here is the angle the English-language AI press mostly skipped: the README lists two supported targets — Nvidia SM100 and Huawei Ascend 950 — with no second-class distinction. Per aiweekly's coverage, this is the first time a frontier lab has given Huawei's Ascend silicon the same first-class framework support as Nvidia's flagship data-center GPUs.

The context sharpens it. DeepSeek said just last week that it plans to pre-train on Huawei Ascend chips by the end of 2026. DeepInfer's dual-target design is the serving-side half of that bet: one codebase, both hardware families. Build requirements tell the story plainly — Linux, GCC 13+, Python 3.12, and CUDA 13 (Nvidia) or CANN 9.2 (Ascend).

The honest read: This is DeepSeek building a non-Nvidia escape hatch into its production stack — in public, under Apache 2.0. Whether Ascend performance actually matches SM100 in practice is unproven (the benchmarks in the README used 4 GPUs for prefill and 64 for decode, on Nvidia hardware). But the intent is unambiguous: DeepSeek is designing its future around the possibility that Nvidia silicon isn't always available.

It also landed one day after the U.S. Department of Homeland Security ordered federal agencies to remove DeepSeek models from their systems — the exclusion order we covered in our DHS DeepSeek ban article. Washington is trying to wall DeepSeek out of government systems; DeepSeek responded by open-sourcing the engine that runs its service, with Huawei silicon as a first-class citizen. The timing may be coincidence. The symbolism isn't.

The Tech: vLLM, DSpark, and the Kernel Stack

For the engineers in the audience, the feature list reads like DeepSeek's recent systems papers made operational:

Custom kernels

FlashMLA, DeepGEMM, DeepEP, TileKernels, and DeepSelect, with TileLang in the mix — the low-level building blocks DeepSeek has published research on, now shipping as runnable code.

Parallelism

Attention data-parallel with balanced context-parallel, plus mixture-of-experts expert-parallel with EPLB (expert-parallel load balancing) — the machinery for splitting giant MoE models across chips.

DSpark speculative decoding

Accelerate decoding with draft tokens and use confidence estimates to schedule verification — guess several tokens ahead with a small draft model, then check them in one pass with the full model, instead of generating strictly one token at a time. Default maximum draft length is 5.

Plus: an asynchronous zero-overhead batch scheduler, batch-invariant inference mode, deterministic sampling ("identical outputs for identical input tokens and sampling parameters"), and structured outputs constrained by regex, JSON schema, or EBNF grammar.

The disclosure the brief demanded: Every performance number in this section's orbit — 155 trillion tokens per day per cluster, 96% input cache hit rate, ~400ms time-to-first-token, 230 tokens/sec (up to 300 for coding), $38 per GPU-hour at valley pricing — comes from DeepSeek's own production figures, as reported by the outlets covering the release. None of it has been independently tested. Treat these as the lab's self-reported numbers, not audited benchmarks.

The Timing: Post-Ban, Pre-Everything

DeepInfer arrives at a strange inflection point:

• One day after the DHS ban. The U.S. government just told its agencies to rip DeepSeek out. DeepSeek answered by giving the world its serving engine. Our DHS ban coverage has the full timeline.
• One week after Japan went the other way. Japan's Digital Agency deployed AI agents to 180,000 government workers — with published safety standards. Washington fears the supply chain; Tokyo deploys at national scale.
• Days after Nvidia's $10B Europe play. Nvidia is buying perches inside Europe's industrial-AI stack while DeepSeek builds the software case for a post-Nvidia world. The hardware war has a software front now.

The repo crossed ~8,000 GitHub stars within hours of launch — the biggest developer-thread of the day on Hacker News (2,136 points at last check). Developers aren't star-bombing a press release; they're bookmarking a codebase they intend to read.

CloudEmythos take: The smart money reads this as a recruiting and ecosystem play as much as a technical one. Every engineer who studies DeepInfer learns to think in DeepSeek's primitives — its kernels, its schedulers, its Ascend support. Open-source the stack and you open-source the talent pipeline. The Apache-2.0 license means competitors can fork it too — DeepSeek is betting the ecosystem gains outweigh that risk.

Related reading

• DHS Bans DeepSeek From Federal Agencies — What the Order Actually Says
• Japan Just Gave 180,000 Government Workers Their Own AI Agents
• NVIDIA's $10 Billion Europe Bet Is the Small Part of the Story

Published October 21, 2026. All performance figures are DeepSeek's self-reported production numbers, not independently verified. Chip-support claims per the DeepInfer GitHub README.