(interview originally recorded May 22, 2026; academic cross-checks June/July 2026) Narrow-domain training's generalization surprised Vinyals — but his bet on model-as-judge is losing to the measurements.

From the Aug 2, 2026 daily brief

Overnight's 15 new papers are still unprocessed and this column has no single-day increment, so here is a claims-versus-academia pairing. Vinyals's self-reported changed mind from the past year: training on narrow domains where answers auto-verify, like math and coding (what the AI world calls RLVR, reinforcement learning with verifiable rewards), "creates this generalization. I think that is not something I quite predicted to work as well as it … did"; but he says plainly the question is open — is math-and-code alone enough to induce general problem-solving? "I don't know. I mean, I think it could go either way." His bet on the way through is letting the model be the judge, to open up domains that can't be auto-scored (Unsupervised Learning interview, recorded May 22, 2026). The academic measurements land squarely on that bet: a July 2026 evaluation study finds AI judges to be systematically over-lenient when no reference answer is available, with verdicts flipping in up to 85% of cases once reference answers are supplied (arXiv paper, July 2026); a June 2026 grader-design study finds that unit tests used as graders can have zero detection power for specific error types — and that a wrong grader's feedback is worse than no feedback at all (arXiv paper, June 2026). Side by side: the domains where you can't write a grader are exactly the domains where AI judging is currently least reliable. The bet isn't unplayable — it's that the measurements aren't on its side yet.

More in this section

• Research Jul 29, 2026 3 sources

  "Use a rubric as a training reward and the model games it too" now has two independent academic results behind it

  A newer route in AI training is rubric-based rewards: instead of training a separate rewar…

• Research Jul 28, 2026 1 sources

  (the primary release carries no explicit date — confirmable only to July 2026; picked up July 27 via commentary) New benchmark MirrorCode: an AI spends 14 hours and $251 on a software-rewrite task humans would need 2–17 weeks for

  Epoch AI, the AI data-research organization, and the evaluator METR released MirrorCode, a…

• Research Jul 26, 2026 1 sources

  Why do models keep getting better at solving problems yet worse at writing like people? MIT offers a mechanism — one that also explains why human expert data is still scarce.

  The current mainstay of post-training (the stage after base training where a model is taug…

• Research Jul 25, 2026 1 sources

  The real bottleneck in "AI writes low-level code" has moved: generating it is easy — trusting the number is hard.

  In the same AMD assessment, SemiAnalysis reports firsthand engineering observations: 2.5 e…