Trend · preprint, October 25

Best agent passes 47.7% of learning stages; AI graded it

In a preprint, Haoyu Zhao and co-authors from UCL, UNC and other institutions built a "continual learning exam" to measure whether frontier LLM agents improve from experience, and the best system passed 47.7% of the stages. The benchmark has 120 professional tasks written by people, each with five stages, 600 stages in all. Each stage unlocks only after the previous one is passed, and later stages require experience gathered earlier (arXiv 2610.33956, 2026-10-25). According to the abstract, across 17 harness-and-model pairings the best system passed 47.7% of stages, and only 23.3% of tasks were passed on all five. Giving diagnostic feedback after a failure and allowing one fix raised the median completion rate by 5.3 percentage points. Grading is done by an evaluator agent against rubrics people wrote. Neither the grader's reliability nor the 17 pairings appear in the abstract. SecondSource chose to place this paper next to item 2. The two use different tasks and metrics and don't corroborate each other. In item 2 an agent won high placements in one game by building an environment, while in this paper agents learning directly from experience pass fewer than half the stages on this benchmark. How we checked →

Concept illustration: seen from behind rows of auditorium seats, a large screen on a stage shows five doors in a row; the two on the left stand open, the middle red door is closed with a laptop on the floor in front of it showing a speech-bubble icon, and the two beige doors on the right are closed. A person in a long dark coat holding a small device stands at the right of the stage looking at the screen, below two black signs at the upper right lettered "UCL" and "UNC".

Source and checks

This item ran in the brief of Oct 27, 2026, alongside 12 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "Best agent passes 47.7% of learning stages; AI graded it." https://secondsource.io/en/research/2026-10-27-3/. Accessed access date.