AI news Oct 28, 2026

This week · newsletter, October 26

Epoch says its top two AI models failed its own staff test

Epoch tested six AI models on 11 of its own jobs; the two leaders reached 65% of the staff rubric, which Epoch calls a failure

Why it matters: A 65% score still leaves every task for a person to fix. At Epoch, the one step partly handed to AI so far is a basic chart that usually comes out right in one try.

Epoch AI published a newsletter post, "We tried to automate Epoch," on October 26. Epoch AI is an independent research group that tracks AI compute, capabilities and trends. The post summarizes Epoch's own automation report. Epoch used 11 real tasks from its staff's work as the test, had six leading models each attempt every task once with no human help, and scored the results item by item against the same rubric used for Epoch employees. The post names only the two leaders, Claude Fable 5.1 and GPT-6 Astra, which averaged 65% of the rubric items. The post doesn't name the other four models or give their scores (Epoch AI, 2026-10-26).

Epoch's bar for handing a task to AI is 100%. The post says: "A 65% score is still a failed task." Epoch says every one of those outputs would have been rejected or sent back for revision if a person had handed it in. The models did well on well-defined tasks such as coding and data analysis, and poorly on open-ended tasks that require spotting patterns from examples or making judgment calls. In one task, the AI had to design and run a pilot experiment that tested other models. GPT-6 Astra set the token budget for those models too low, so 22% of the runs returned no answer. It then reported those failures as an important finding about model capability. It did not see that the experiment's setup was wrong. At the other end, Epoch removed a basic chart task from the test because Fable 5.1 produces an acceptable chart in one try most of the time, and Epoch's daily work now partly hands that step to AI.

Not proven yet: only two of six models are named, and 65% is their average How we checked →

Judgment update: This fits SecondSource's view that tasks with a standard answer, where right and wrong show at a glance, go to AI first, and open-ended judgment stays with people. Our reading: whether AI can take over a job function turns on whether the cost of a person reviewing the output and fixing it into shape is lower than the cost of a person doing it from scratch. That is SecondSource's yardstick; the newsletter doesn't measure that cost. Epoch's examples sit at the two ends. One chart task left the test because Fable 5.1 usually gets it right in one try, while GPT-6 Astra didn't see that the experiment's own setup was wrong. The next thing to watch is how many tasks reach 100% in Epoch's next update, and the post doesn't say when that will be.

Answer check: A reasoning draft is the intermediate working a model writes out before answering. On October 22, SecondSource called that OpenAI's next frontier system card would describe how the model's ability to rewrite its reasoning draft on instruction had changed. The call also said the card would not explain why the model's ability to reason without writing a draft had jumped. The October 26 GPT-6.1 Terra system card explains the cause of neither change, so the call can't be judged (GPT-6.1 Terra system card, 2026-10-26). The earlier system card said the change in rewriting ability did not come from an architecture change, and that sentence is gone from this one. Card →