Computer-use AI will pass 60% on the long-task OSWorld 2.0 test by mid-2027 — resolves 2027-06-30

The debate is whether AI that operates a computer, reading the screen, clicking and typing, is held back by practice grounds that cannot be replayed or by how long a task runs. If every replayable ground gets trained up, companies can predict which computer work gets automated first, and if length is the wall the timeline moves out. SecondSource expects the replayable long-task test to be trained up as fast as the first version was, while the other reading says longer tasks stay hard even when they can be replayed.

SecondSource’s reading: progress in computer-operating AI follows whether the practice ground can be replayed. Where a test can be rerun with repeatable results, models get trained up: on the first OSWorld test, Sonnet 4.6 scored 72.5%, level with the human 72.36%.

OSWorld 2.0 can also be replayed, but its workflows are longer. The best reading in the June 2026 paper was only 20.6% (Claude Opus 4.8), measured as binary completion, where a task counts only if every step is finished.

We bet it climbs fast along the path the first version took: before 2027-06-30, the top binary completion rate on the official leaderboard reaches 60% or more.

The original material behind this call has not yet been linked here in a form readers can open and check.

The highest binary completion rate listed on the leaderboard of the official OSWorld 2.0 website (osworld-v2.xlang.ai), with a 30-day grace period.

What would overturn it

If the top binary completion rate on the official OSWorld 2.0 leaderboard is below 60%, the claim in sections four and six that replayable grounds get trained up must be rewritten: length is a harder wall than replayability.