Models tied to their coding tools: the format layer is fixable with strict mode — resolves 2026-11-12

The same AI model does well inside its maker’s coding tool and worse inside someone else’s, and people want to know why.

If a model only works well with its own vendor’s tool, choosing a model also means choosing the tool, and switching vendors gets harder.

One camp says models are trained to know only their own tool’s format; SecondSource reads the cause as formats that nobody checks strictly, which can be fixed.

SecondSource’s reading: there is a nameable reason a model and its tool are hard to separate, but it is not that the model only knows its own format. It is that nobody enforces the format.

Anthropic’s own Claude Code silently drops extra fields a model writes and does not turn on strict mode, so during training the model is never penalized for inventing fields.

The format layer can be fixed with one switch: Anthropic’s strict tool use, which forces output to match the tool’s format exactly, already supports Opus 4.8, Sonnet 5, Opus 5 and the Fable 5 models.

The real cost sits with the coding tool, which must keep a format table for every model and lags each new model generation. Whether models that are used to their own vendor’s tool succeed less often elsewhere has not been measured at all.

Two parts of the original claim do not hold. The two supposedly unrelated sources were not from the same day: the explanation came from a blog post seven weeks earlier, the numbers from an outside contributor’s unmerged code change.

It also credited the tool for a 58.4% versus 63% success gap, but on 113 tasks a 4.6-point gap is smaller than one sampling error and tells us nothing.

The original material behind this call has not yet been linked here in a form readers can open and check.

By 12 November 2026, check whether the open-source coding tool Pi turns on strict mode or the native editing tool for Claude by default, whether anyone publishes how often Opus 5 or Fable 5 invent fields in Pi’s original format, and whether any tool maker publishes per-model format failure rates. Longer term, compare success with strict mode on and off for the same model, tool and task set.

What would overturn it

If editing success with strict mode on still clearly trails Claude Code, the problem is not only the format and the one-switch fix is wrong. If newer models invent almost no fields, the claim that the newest generation is worse was noise.

Read the full deep dive →