AI news Oct 21, 2026

Today · post, October 20

DeepSeek open-sources DeepInfer; figures are self-reported

DeepSeek open-sources DeepInfer, the software serving DeepSeek's V4.1-Flash model in production, and says DeepSeek serves 155 trillion tokens a day on one cluster, as relayed by swyx

Why it matters: DeepSeek opened its production serving stack as well as its weights. On DeepSeek's own figures, other providers may close in on frontier labs' serving efficiency.

DeepSeek is a Chinese AI lab. On October 20, DeepSeek's official account announced it is open-sourcing DeepInfer, DeepSeek's inference framework: "We're sharing our production stack to help the community lower compute costs" (DeepSeek, 2026-10-20). DeepInfer serves DeepSeek's own V4.1-Flash.

swyx, co-host of the AI engineering podcast Latent Space, relayed a passage from DeepSeek's official Zhihu page: 155 trillion tokens a day on a single cluster, and overall revenue of US$38 per card-hour at off-peak prices (swyx, 2026-10-20). A card-hour is one accelerator card running for one hour. swyx assumes a GPU-hour costs US$5 to US$10 and, against the US$38 of revenue, arrives at a serving gross margin of about 80%. That is swyx's arithmetic. On the same US$5 to US$10 assumption, the margin ranges from 73.7% to 86.8% by our calculation. swyx's 80% sits inside that range. The relayed passage gives no card count or hardware for the cluster, so the 155 trillion tokens and the US$38 can't be converted into each other.

The open-source inference framework vLLM said the next day that, using the decode settings DeepSeek published, vLLM's decode throughput on GB200 is within 6% of the result DeepInfer published (vLLM, 2026-10-21). That compares against DeepSeek's published numbers; it is not a side-by-side run. Tim Dettmers, author of QLoRA and a machine-learning systems researcher, writes that DeepSeek's inference system is on par with OpenAI's, with no comparison data attached (Tim Dettmers, 2026-10-20).

Not proven yet: revenue figures are DeepSeek's own; the original Zhihu page hasn't been reviewed How we checked →

Judgment update: Our read: if inference serving stops being where efficiency gaps open up, competition moves back to model quality and access to customers. That would weaken the argument that inference moats rest on scarce compute and heavy capex. This call is not settled. A same-day signal points the other way: Reuters reports that ByteDance, Alibaba and Tencent ordered more than US$35 billion of Nvidia chips, and Nvidia told customers it can't guarantee delivery (see Also happened item 3). While compute stays this tight, the moat may hold. The signal that would decide it is a team other than DeepSeek publishing throughput and cost figures using DeepInfer.