AI news Oct 21, 2026

Today · announcement, October 20

Google makes TPUs PyTorch-native; figures are vendors' own

Google makes the TPU a native PyTorch device, and companies founded by the creators of vLLM and SGLang co-develop TPU versions of both inference engines

Why it matters: Open models can move to TPU by changing settings in the model code, but the low-level kernels still need rewriting. Performance-per-dollar figures come only from Google.

On October 20, Google announced TorchTPU on its developer blog: the TPU becomes a native PyTorch device, so code written in ordinary PyTorch runs on it (Google for Developers, 2026-10-20). The same post launches TPU versions of the vLLM and SGLang inference engines, two widely used open-source inference frameworks. Google co-developed them with Inferact and RadixArk, which Google says were founded by the vLLM and SGLang founding teams.

Google says the engines already run for "preview customers," who aren't named. Google also self-reports a result on InferenceX, SemiAnalysis's public inference benchmark. There, Alibaba's open model Qwen 3.5 running on Google's Ironwood TPU posts the benchmark's lowest cost, 2.01 US cents per million tokens. The post names no rival and gives no rival figure.

A second Google post the same day says TorchTitan, Meta's PyTorch framework for training large models, is also joining the native TPU path. Hugging Face said the same day that models running on Transformers and Diffusers no longer need Optimum TPU, Hugging Face's own adapter layer (Hugging Face, 2026-10-20). Kernels still have to be written separately for TPU.

The September 13 issue reported SemiAnalysis saying TorchTPU would be open-sourced in mid-October. The installable public release went up on October 19 (PyPI torch-tpu), so SemiAnalysis's timing held. Google's April 7 blog post already said the GitHub repository was public, so "open-sourced" here means the installable release.

Not proven yet: every cost and performance figure comes from Google or Google's partners How we checked →

Judgment update: Our read: Google is bringing in the vLLM and SGLang founding teams, Meta's training framework and Hugging Face's model definitions. By Google's and its partners' account, moving an open model to TPU takes only config changes at the model-definition and framework layer. Kernels still have to be rewritten or tuned for TPU. Performance per dollar so far rests on Google-side numbers alone, with no independent measurement. SecondSource's September checkpoint is whether a named third party with no Google ties reruns InferenceX on the public release by November 7, 2026. There is no new evidence on it today. What would prove this wrong: a named non-Google customer publishes production costs after migrating to TPU, or the preview customers are still unnamed in March 2027.