Research & models Oct 18, 2026

Trend · preprint October 9

TheoPhysEval's top agent hits 98%; it's not peer-reviewed

TheoPhysEval, 50 unpublished research-level theoretical physics problems with verifiable answers: the paper says the best agent scores 98% accuracy at under US$3 per problem, yet there is still no agent-led breakthrough in theoretical physics. The paper was submitted by 34 authors from multiple institutions (arXiv 2610.16733, 2026-10-09); only the abstract was read. Per the paper, the best agent is "GPT-6-Astra harnessed in Codex," meaning GPT-6-Astra run inside Codex, OpenAI's coding agent, with tools for literature search and numerical scripting. Disabling web search and background papers moved each agent's accuracy only slightly, which the authors read as significant memorization of the literature. Every problem is a computation with a verifiable answer, not open-ended discovery; 50 problems is a small sample, from one source, not peer-reviewed. In judging progress in AI for Science, "solving problems" and "making discoveries" need to be counted separately.