The only way anyone can see what an AI is thinking may be failing, and nobody has the standing to check

Today's main line

1. [This week] (events September 1–2) The guardrail that lets anyone see what an AI is thinking got fought over for a day; what is left when the fight ends is that nobody has the standing to check

Reasoning models of the past two years work out an answer before they give it, and they write that working out step by step in language a person can read. The industry calls it chain of thought. Because the reasoning arrives as human language, a safety team can read it, and can aim a second model at it to scan for a model planning something it should not. It is the only oversight technique deployed at scale anywhere in this industry, and the only one that has ever caught anything.

The story arrives in three beats. On September 1, OpenAI's official account announced that its forthcoming model, Astra, "represents a significant advance in cybersecurity capability, reaching the Critical threshold under our Preparedness Framework" (@OpenAI, 09-01). The Preparedness Framework is OpenAI's own risk-grading and deployment-control regime for its models; it is a company policy rather than a law, and no outside body verifies the grade. Critical is the top rung, and it carries a commitment not to deploy until safeguards are in place. Early on September 2, The Information, a subscription technology business publication, reported that Astra uses a breakthrough that could destroy the monitorability of chain of thought, citing a source who said OpenAI is currently "limiting the use of the technique". At 5:30 that morning, OpenAI's chief scientist Jakub Pachocki answered in person, with a number anyone can hold him to: "The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4." He also conceded that chain-of-thought monitoring "is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes" (@merettm, 09-02; his title was confirmed publicly in the same thread by OpenAI researcher Noam Brown). Computation-graph depth is roughly how many layers of arithmetic sit inside the model, so a figure that has not exploded means the reasoning has not been relocated somewhere invisible.

Verification: three unaffiliated parties moved separately, and all three point away from the reading that OpenAI got caught. One, the technical read. Sebastian Raschka, a machine-learning educator and the author of Build a Large Language Model (From Scratch), with no commercial stake in either side, took apart the technique the report pointed at: run the same stack of layers twice, keeping the model the same size and nearly doubling the compute. His conclusion was that "reusing layers does not by itself suppress visible chain of thought," and he said outright that the journalist may be describing a different technique or may have misunderstood this one (@rasbt, 09-02). He also named the one worry that survives: more passes could mean the model emits less intermediate reasoning text. His own rebuttal to that is that scaling the model up does the same thing — "we would get the same effect if we were scaling up the model size" — so the worry is not specific to this architecture. Two, the competitor. Jack Lindsey, an interpretability researcher at Anthropic, wrote that "this isn't a dunk on OpenAI -- I have no evidence that OpenAI has allowed CoT monitorability to degrade substantially," and offered a judgment of his own: the white-box approach that skips the text and reads the model's internal activity directly is "unlikely" to "provide as much monitorability as CoT currently provides within a year" (@Jack_W_Lindsey, 09-02). Three, the reporter. The Information's Stephanie Palazzolo said publicly that there is "more nuance here than people are initially getting from the headline" (@steph_palazzolo, 09-02). ⚠️ We have not read The Information's article — it sits behind a paywall — so every description of its contents here reaches you through someone else's account of it. ⚠️ Pachocki's "within a factor of two" is a ratio, not a level. He never said what GPT-4's depth is, and every absolute figure circulating online is somebody's inference.

Judgment update: the point of this item is not that a company did something wrong. It is a public technical dispute, and the hole it exposes. The policy scholar Dean Ball put the argument most plainly: a listed company's financials do not get settled by people shouting at each other on a social platform, they get settled by an auditing profession and audited statements. "It would be like if we argued about what every publicly traded company's financials were by posting hyperventilating on the timeline rather than relying on the institution of auditing and the audited financial statements that institution produces," he wrote (@deanwball, 09-02). Quoting him requires one disclosure: he now works at OpenAI, which he said himself. From the opposite corner, the AI safety researcher Geoffrey Irving asked the obvious question back: "Yes, regulation would be great here, but also...you could just tell someone independent what the architecture is?" (@geoffreyirving, 09-02) The two men prescribe opposite remedies and land in the same place: no independent body has the standing to verify a frontier lab's architecture decisions, which is why a question like this can only be fought out on a social platform. That is the thing we intend to watch for a long time. The question to carry into your own team: if our AI vendor did this exact thing tomorrow, could we check it ourselves? The answer is almost certainly no, and what belongs in the contract is therefore today's work, not next year's. What would prove this wrong: a white-box technique that outperforms chain-of-thought monitoring overall inside a year makes Lindsey's judgment wrong. In the other direction, any frontier vendor publicly admitting that its own chain-of-thought monitorability has degraded substantially turns this risk from a guess into a fact. Verdict date: September 2, 2027, set off Lindsey's "within a year" window and not off any date an institution has committed to.

Investor note: the prevailing story assumes AI safety is something labs can manage internally, and that failures surface. This evidence weakens it. The one guardrail actually in service is called fragile by the side defending it, and a competitor says nothing can replace it for a year. The leading signal to watch is not the next leak. It is whether any lab hands its architecture details to an independent person at all.

2. [This week] (events September 2) The capability nobody could move for years finally moved — and what keeps agents out of production was never intelligence, it is the failure rate

Anthropic released Claude Fable 5.1 on September 1 with its own scores attached: "On Terminal-Bench 4.0, it scores 55.8% against 42.0% for Fable 5" (@claudeai, 09-01). The release is not what you should read today. What matters is what several unaffiliated third parties measured the day after, each on their own.

The first finding is that a slope changed. LlamaIndex builds document parsing and retrieval infrastructure and maintains ParseBench, a parsing benchmark drawn from "2k+ real-world documents across finance, legal, insurance, and more". It measured Fable 5.1 beating the previous generation on tables, content faithfulness, formatting, charts and visual grounding, and wrote the sentence that matters: this is "one of the first frontier models in recent memory where document OCR performance actually improved between model generations!" Before it, "the latest models (Opus 5, 5.6-Sol, 3.7 Flash) have stagnated on visual understanding tasks" (@jerryjliu0, 09-02). An enterprise's real data lives in PDFs, scans, tables and handwritten forms. A model can score as high as it likes on reasoning tests; if it misreads the number in a table, everything downstream of it is wrong. That dimension has barely moved for two years while reasoning raced ahead.

The second finding puts a number on the price. Signal65, an independent technology performance testing outfit, ran its PINNACLE enterprise-workflow benchmark and measured Opus 5 and GPT-5.6 Sol failing 14 workflows in every 100, against 7 for Fable 5.1. Asked something the documents cannot answer, Fable 5.1 invents an answer 0.7% of the time; Opus 5 does it 7.6% of the time. The cost is stated just as plainly: a correct answer from Fable 5.1 costs $2.46 on the full API bill, 1.8 times Opus 5, and 80% of that is the context the agent re-reads at every step, with no setting available to turn it down. At scale, ten thousand correct tasks a month runs about $25,000 on Fable 5.1, $14,000 on Opus 5 and $5,000 on GLM-5.2, the open-weight model from China's Zhipu, run on leased GPU hardware (@Signal_65, 09-02).

Verification: ⚠️ The ParseBench post gives only the summary claim of beating the previous generation "across the board"; the per-metric scores sit behind a shortened link we could not open. ⚠️ The measurer's identity matters more: LlamaIndex is both the benchmark author and a competing vendor, and the same post sells its own product. The bias runs in this item's favour, though, because the commercial interest is in saying frontier models are not good enough. It said the opposite this time. It also said the part that suits it: Fable 5.1 as a document recognition engine "costs 15c a page - which is 10-15x more expensive than our default agentic mode on LlamaParse (which performs similarly)," and that last clause is self-assessment to be discounted. ⚠️ Signal65's revenue model may include vendor-commissioned testing, and we cannot confirm whether this round was. ⚠️ That $5,000 comes from a different population altogether (renting your own GPUs for an open-weight model, against calling an API), and no failure rate for GLM-5.2 was published, so nothing follows about it being cheap and equally good.

Judgment update: the Signal65 argument deserves to survive intact: "The thing that keeps AI agents out of production is not intelligence, it is the failure rate. Every workflow an agent fails is one a person has to catch." At 14 failures per 100, an enterprise keeps human review on everything. At 7 per 100, with invented answers down to a tenth, "review every task" becomes "review the exceptions". We are moving the gate on agent deployment out of the capability column and into the failure-rate and unit-cost column, and this round turned those two into an explicit trade: half the failures, 1.8 times the bill, not both improving at once. Two things follow immediately. If you run document workflows, re-test now, because the dimension that had stalled has moved. But do not use it as your document recognition engine, at 10 to 15 times the price. That 80% has a second layer worth knowing: it is not intrinsic to the model. It is what the software wrapped around the model decides about trimming context, which makes it an engineering problem someone can solve, with no switch available today. What would prove this wrong: a next generation that drops the failure rate and the cost per correct answer together, at which point the trade disappears.

Investor note: the prevailing story assumes stronger models mean agents roll out. This evidence redirects it rather than strengthening or weakening it: the capability gain is real, but what stands in the doorway of production has always been the failure rate and the bill per correct answer, and only one of those changed. Watch for anyone starting to disclose cost per correct answer as a line item. Once that becomes a standard disclosure, every model comparison table has to be redrawn.

Also happened — not verified by us yet

1. A benchmark called FrontierHarness, published September 2, holds the model, the tasks and the runtime fixed and swaps only the software wrapped around the model. The industry calls that layer the harness, and it decides which tools the model can reach, how context gets trimmed, and what happens on a retry. Across nine harnesses and 360 runs: pass rates from 50% to 67%, cost per pass from $1.05 to $18.34, a 17.5-fold spread (@guanlan, 09-02). ⚠️ We are not writing this up as a judgment today, because "the same model" is the controlled variable of the whole experiment and which model it was has not been published. Worth following.

2. The research team at Ramp, an enterprise spend management platform, says 80% of OpenAI's and Anthropic's enterprise revenue comes from 1% of their customers, with that top 1% skewing heavily toward the tech sector and AI products and services. We have not obtained the underlying report, and the data covers only businesses that use Ramp rather than the whole market (@arakharazian, 09-02).

3. Rui Ma, an analyst covering China's technology industry, says close to 40% of new cars sold in China in the first half of 2026 shipped with L2+ or L2++ driving assistance (the first is broadly highway-focused automated driving, the second extends onto city streets), increasingly bundled into the car with no subscription. Her comparison points in the US are Tesla FSD at $99 a month, Ford BlueCruise at $50 and GM Super Cruise at $40. She does not give a source for the 40% (@ruima, 09-02).

4. [This week] (events September 2) Two frontier vendors arrived independently at the same way of handling cybersecurity capability, and the question worth asking today is a different one: Google, Gemini, $0.75 (Google DeepMind's announcement, 09-02). ⚠️ There was room for one line only today; the detail is at the link above.

Named commentary

[This week] (published September 2) Zvi Mowshowitz ranks today differently than we do. He has run an AI weekly for years, tracking frontier lab releases and safety questions item by item, with no commercial stake in any lab. On the dispute in item 1 above, his call was "Playing with fire is also my read. The house has not burned down." (@TheZvi, 09-02), which matches where we landed: a dispute, not an incident. But in a separate post he said plainly that "Today still has to be Fable 5.1 Model Card Day though." (@TheZvi, 09-02) He thinks the most important thing today is the item we ranked second, not the item we ranked first. We are leaving our order where it is, for this reason: readings on model capability arrive every week, while "no institution has the standing to verify" is something supporters and opponents said out loud together for the first time this year. The disagreement is worth recording, and you can pick a side yourself.

From the archive

No archive pick this issue. We have used up the older material worth reusing from our own back catalogue — the last pick ran on July 30, and this is the fifteenth consecutive issue with the column empty. We would rather leave it blank than replay an item we have already run.