Research & models Oct 21, 2026

Today · post October 20

DeepSeek tops GPT-6 Sol in cyber test; Vals will post scores

Preliminary results from third-party evaluator Vals AI: DeepSeek V4.1 Flash 1017 beat GPT-6 Sol on multi-step offensive security tasks in a simulated corporate network. GPT-6 Sol was tested through a trusted-access program, and scores and methods will be published in a few weeks. Vals AI is a company that publishes third-party evaluations of models. Vals writes that on its own multi-step offensive security tasks, DeepSeek V4.1 Flash 1017 beat GPT-6 Sol. GPT-6 Sol was tested through the Daybreak trusted-access program, which the post doesn't explain, and the setup of that access channel could affect the result. The test environment is a simulated corporate network. Full results will be published in a few weeks (Vals AI, 2026-10-20). Only these two models were compared, and the post contains no scores. Nathan Lambert, a post-training researcher at Ai2 and author of Interconnects, responded to the Vals result: AI security risk looks more like an ecosystem-wide problem than one specific to open-weight models. Lambert also says the same model is probably good at defense too, but nobody posts defensive results for clicks (Nathan Lambert, 2026-10-20; Nathan Lambert, 2026-10-20). Lambert is an advocate for open models, and both posts are Lambert's opinion with no data attached. In our view, a preliminary result on a single task says nothing about the security capability of open-weight models as a whole. How we checked →