← All verdict cards · Public version history ↗ · First public version ↗
Can independent coding tools clear 3 adoption and model-switching tests by July 2027? — resolves 2027-07-14
Can independent coding tools clear 3 adoption and model-switching tests by July 2027? — resolves 2027-07-14
Scheduled check: 2027-07-14
Background
A coding harness is the software around an AI model that directs its work. Independent harnesses could let buyers switch model suppliers; tools tied to one supplier could keep buyers dependent on that supplier.
This card tests whether independent tools can demonstrate both large-enterprise adoption and reliable performance after a model switch.
The call
This card tests whether three conditions all hold by 14 July 2027: (1) at least two independent coding-tool vendors support models from different suppliers; (2) each has public adoption testimonials from at least three Fortune 500 or comparably sized companies; and (3) benchmark scores fall by less than 5 percentage points when the underlying model is changed. Percentage points are the direct difference between percentage scores: a fall from 80% to 75% is 5 points, which does not meet “less than 5.”
SecondSource estimates a 40% probability for this whole event, below even odds. All three conditions must hold for a hit; failure on any one is a miss. A miss leaves in place the view that suppliers providing both the model and the tool retain an advantage—the “vertical-integration” view mentioned below. Present-day adoption and switching scores have not been verified.
The original rule does not fully define comparably sized companies, equivalent benchmarks, or which model pair must be compared. Those choices need clarification at assessment. A major change in benchmark methods that makes switching scores incomparable voids the forecast under the original rule.
Sources behind the call
The original material behind this call has not yet been linked here in a form readers can open and check.
The receipt
Check the harness vendors' own case study pages and the historical SWE-bench Verified (or comparable hard benchmark) leaderboard records, comparing scores before and after a model swap.
What would overturn it
Failing any one of the three thresholds resolves this as a miss and leaves the vertical-integration reading in place.