SecondSourceJudgment rebuilt from primary sources

OpenAI lets four outside evaluators examine models still in training

Concept illustration: on the left, a large display wall lettered OPENAI beside a white interlaced knot-shaped emblem has rack-style panels with small red and white lights on both sides and a central dark screen showing a speech bubble with grey lines and three small squares; in front of it a long table holds four open laptops whose screens show grey lines, where a man in a suit types on one laptop and a woman stands with her hands behind her back, looking up at the screen. On the right, a sign on a stand lettered EVALUATORS stands in front of a doorway, and a man in dark clothes wearing a badge on a lanyard stands behind a foreground desk that holds a red folder lettered REPORT and a black pen. Printed on the image: Evaluators get source code and training data; OpenAI can redact and require delayed publication.

Four outside evaluation groups are working inside OpenAI to examine internal models still in training. OpenAI announced this first phase of "embedded assessments" on November 16 (OpenAI, 2026-11-16). The evaluators can see models in training, but OpenAI keeps the power to redact confidential information and to require delayed publication.

The evaluators get access to in-training model versions, source code, training and evaluation data, and monitoring systems. METR, which evaluates AI autonomy and monitoring, is looking at OpenAI's largest frontier reinforcement-learning run. Apollo Research, which evaluates whether models covertly go against instructions, is testing four OpenAI claims in two stages, such as the claim that training doesn't push models toward covert defiance. Transluce and Goodfire work on interpretability, the study of how a model works internally.

OpenAI also keeps several powers: it can redact confidential information and ask for publication to be delayed, and what employees say in interviews may not be quoted publicly. Disputes go to the safety committee of the OpenAI Foundation's board. OpenAI says the program fulfills a commitment under the White House Accord on Super Intelligence.

Expert view · Daniel Kokotajlo (researcher at the AI Futures Project, former OpenAI employee; interests: no public disclosure found): "Apollo embedded evaluators will only be allowed to talk to a specific whitelist of OpenAI employees chosen by OpenAI, … and they aren't allowed to publish the results without OpenAI approval and OpenAI is allowed to redact whatever it wants, … Extremely weaksauce." (Daniel Kokotajlo, 2026-11-16)

Not proven yet: how independent the evaluations are depends on the reports published later How we checked →

Next to watch: how much is redacted from the first reports the four groups publish, and the results of Transluce's roughly eight-week evaluation in early 2027.

Source and checks

This item ran in the brief of Nov 17, 2026, alongside 13 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "OpenAI lets four outside evaluators examine models still in training." https://secondsource.io/en/news/2026-11-17-4/. Accessed access date.