Does Google's TPU beat Nvidia by up to 50% on performance per dollar? The claim, held up against the SemiAnalysis newsletter it came from

① Who said it, and what was said

SemiAnalysis is the independent analyst firm most often cited on semiconductors and compute infrastructure. On September 7 it announced that InferenceX, the inference benchmark SemiAnalysis runs and publishes, would start posting daily results for the TPU, the chip Google designs in-house. Founder Dylan Patel wrote on his own account that "$/token is better than B200 and B300," Nvidia's current data-center chips (@dylan522p, 2026-09-07). The firm's official post carried a six-line headline that included both "Up to 50% Better Performance per Dollar" and "Reducing CUDA Moat," the claim that Nvidia's software ecosystem is getting easier to displace (@SemiAnalysis_, 2026-09-07).

Among the voices this publication tracks, Patel's remarks were the most relayed this week. Of the 529 named speakers tracked this week, 259 were mentioned by at least one kind of source. Only 2 cleared three bars at once: other people were relaying them, a primary document existed to check against, and a sentence could be proven wrong. Patel was the more prominent of the two. Patel put out 6 original posts this week, and their average engagement was 2.4 times the median engagement of his past original posts. That figure compares an average with a median, so a single viral post can pull it up, and the two raw figures behind it are not published. SecondSource's main issues also cited Patel once each, on September 6 and September 8.

What the claim is checked against. Two primary documents: the full free section of SemiAnalysis's September 7 newsletter, and the original quarterly report Nvidia filed with the US Securities and Exchange Commission on August 26.

Which sentence could fall. Read as a general rule, "up to 50% better performance per dollar" falls, and the sentence that knocks it down is in the same newsletter: "Across most of the raw-performance curve, TPU does not outperform Nvidia GPUs."

② The receipts

The 50% is tied to one point on the curve, and the newsletter's own text marks that point (primary source, the hardest receipt here). The full free section of the newsletter is available (SemiAnalysis, 2026-09-07), and all five parameters that decide comparability can be pulled from it. The hardware is TPUv7 Ironwood against the B200 and B300. The model is Qwen3.5 397B. Precision is FP8 on both sides. The workload is 8k input tokens and 1k output tokens with single-token prediction. The serving mode is aggregated serving. Aggregated serving means the two phases of an answer are not split apart and run on the same set of chips. Single-token prediction means one token is generated at a time, without the speculative decoding that comes up below.

The 50% belongs to a single point: 20 tokens per second per user at 256 concurrent streams. At 100 tokens per second, the TPU's edge drops to 19% over the B200 and 34% over the B300. At a 20-second median response, the edge is 8% and 25%. Around a 30-second median response, the B200 wins. The newsletter puts it this way: "Across most of the raw-performance curve, TPU does not outperform Nvidia GPUs. However, that comparison does not account for TPU's lower TCO."

What wins the result back is total cost of ownership, and that cost is the benchmark author's own estimate. Total cost of ownership is everything a machine costs from purchase to retirement. The external figure the newsletter uses comes from SemiAnalysis's own cost model, not from a Google price quote. The variable that flips the conclusion therefore has the same source and the same author as the conclusion.

This benchmark is not independent, and the newsletter says so. The newsletter thanks 15 Google engineers by name, along with Inferact, a company that builds inference engines. The software the measurements ran on was written by Google for this exercise and currently sits in a private fork held by SemiAnalysis. That private version moves over only when TorchTPU, a public project that lets developers outside Google run models on TPUs, is released in mid-October. Disclosing this is better than hiding it, but public is not the same as independent.

The headline says the moat is shrinking; the body measures the opposite direction. The body says the TPU's external software is not yet optimized for disaggregated serving or speculative decoding. Disaggregated serving is the counterpart to the aggregated serving above: the two phases of an answer run on separate chip pools. In the comparison where both sides use disaggregated serving, Nvidia's GB300 rack still leads. The shrinking moat in the headline has no support in the body's measurements.

The same method, stripping off the headline and going back to the original for its conditions, works on another claim this week. A second headline number from the same firm in the same week fails in the same place as the first, and this time an independent primary document is available. From September 11, a figure began circulating that Nvidia's off-balance-sheet guarantees total US$530 billion. Nvidia's original quarterly report, pulled from the SEC's database and converted to plain text of 170,000 characters, contains "off-balance" 0 times and "530 billion" 0 times (Nvidia quarterly report, 2026-08-26). What the filing actually publishes is three tables built on different measures: future commitments of US$366 billion, additional commitments of US$56 billion, and maximum exposure under guarantees of US$108.5 billion. Together they add up to US$530.5 billion. Inside the largest table, future commitments, the memory and capacity purchase commitments alone come to US$279 billion, which is material Nvidia has committed to buy for its own products. What the statutory table actually classes as guarantees is US$108.5 billion. The problem is not the arithmetic; it is the label.

The weakest receipt in this column: the attribution of the second number can only be paraphrased. The framing behind that total comes from a paid newsletter, and this publication carries no verbatim quotes from paid sources. The sentence saying who added the three tables together and called the sum off-balance-sheet guarantees therefore has no verbatim original attached. ⚠️ That makes the attribution weaker than the four receipts before it: those four link to originals anyone can read, while this one rests only on SecondSource's own record in the September 12 issue. One more limit: every primary reading on the TPU side comes from the same firm, on the same day, in the same round of outreach. The newsletter's full text adds comparability; it does not add a second independent source.

③ The receipts say: unresolved

The claim being ruled on joins two lines of the SemiAnalysis headline: "Up to 50% Better Performance per Dollar" and "Reducing CUDA Moat."

The half that holds: the number is real, and its conditions are disclosed. All five comparability parameters are there, the operating point is clearly marked, and the acknowledgments and the origin of the software are stated up front. That is harder evidence than most benchmarks on the market offer. At that one operating point, the reading stands.

The half that doesn't: two steps are missing between the reading and the conclusion. First, across most of the curve the TPU does not win; the newsletter says so, but that fact never reached the headline. Second, the total cost of ownership that pulls the result back is an estimate set by the benchmark author, not a price anyone can check. As for the shrinking moat, the body measures the opposite direction. Our call is therefore unresolved: the measurement is credible, and the generality of the headline is not.

What would prove this wrong: ① After TorchTPU's public release in mid-October, a third party with no working relationship with Google reruns the same workloads on the public software and gets the same direction. The headline's generality would then have evidence behind it, and the call would change to holds. ② Conversely, if a measurement beyond that curve, one outside the sources reviewed for this column, shows the TPU ahead at most points, the call that the headline is stronger than the body is wrong. ③ If Google starts signing verifiable long-term contracts with outside customers and market share actually starts to move, an existing judgment here gets re-examined: that the bottleneck is externalization, not price-performance. This column's practical conclusion would change with it.

How to use it. A performance-per-dollar comparison means little until it names its point on the curve. Section ② already gives four points: 20 tokens per second per user at 256 concurrent streams, 100 tokens per second, a 20-second median response, and a 30-second median response. Which chip is cheaper for inference flips with tokens per second and with response time, so the same hardware can win by 50% at one point and lose at another. A procurement or investment decision built on a number like this holds only once the buyer's own service target is placed on that curve and the vendor supplies the reading at that point. The second claim has the same shape: a total of guarantees or commitments needs splitting into purchase and investment commitments the company will pay for its own use, and exposure that actually backstops someone else. Two boundaries. This column rules on how general the sentence is, not on whether the TPU is any good, and not on whether Patel is a credible analyst. Only two cases this week share the shape, and two are not yet a pattern.

④ Where the claim sits

The claim's place on the map uses the existing trend tree; no new tree is drawn.

This week's claim lands on the last segment of the tree. That custom chips have reached in-house inference is an existing judgment. This benchmark tries to push one step further and say the chips now win on price-performance as well. ⚠️ The dividing line for that step is not the chip but whether it is sold. The existing judgment here holds that whether the TPU can take share hinges on externalization. Externalization means something built for in-house use starts being sold or rented to outsiders. The sources reviewed for this column contain no external long-term TPU contract with a named customer and checkable capacity or term, and that absence is the starting point measured here. The next section opens a window on it.

⑤ Checking the answers

This issue opens two short windows. Both dates are observation windows SecondSource set, not deadlines anyone has promised. The first: by November 7, does a third party with no working relationship with Google rerun the same workloads on the public software released after the mid-October open-sourcing and measure the same direction on performance per dollar? If one does, the headline's generality gains independent evidence and this column's call changes to holds. If not, the weakness that the primary readings come from one firm still stands. The second: by October 31, does Google sign a verifiable external long-term TPU contract, with a named customer and capacity or term that can be checked? This window tests the point an existing judgment here rests on: what holds back share is externalization, not price-performance.

One update to an older entry, opened on July 14: training stays on GPUs while in-house inference moves to ASICs. This week's benchmark does not touch that entry's closing condition. It adds a condition that would prove the entry wrong, one the entry did not have before. If, after the public software is released, an unaffiliated third party measures custom chips ahead on performance per dollar while adoption by buyers outside the chip designers' own use stays flat, then the recorded cause of the two paths coexisting and diverging has to move from price-performance to supply terms. That window runs to January 31, 2027.

The nearest verdict date is September 15. All judgments and their verdict dates → the judgment index.

One open item remains: 1 short window that came due on 2026-09-01 is still unsettled on the ledger as of this issue (record).