Today · published October 16

Databricks cuts npm malware misses to 4%; it's unreplicated

Databricks security research: cheap models miss npm malware hidden in binary files, and about 1,800 tokens of domain knowledge given before reasoning cuts their missed human-review referrals from 50% to 4%. npm is the JavaScript package registry. The test set is 70 runs over ten kinds of binary malicious payload. The comparison is the same budget-tier models, on the same inputs, with and without that knowledge. A failure here means a malicious package's verdict didn't even reach the threshold for human review. The comparison models, Sonnet 4.6 and Kimi K3, caught all ten payloads without the added knowledge, at about three times the per-run cost of the budget tier (Databricks, 2026-10-16). Multi-model voting didn't help: "correlated errors converge on the wrong answer rather than canceling". Gemini, one of the models tested, flagged all ten as malicious but invented addresses that weren't in the scanned input. This is Databricks' own research promoting Databricks' own approach, with no independent reproduction.

Concept illustration: at the left, a little above the middle, is a light grey sign with a red stacked-layer icon beside the lettering DATABRICKS; on the right, in front of a pale pink circle, an open laptop sits on a beige surface, with a hand in a black sleeve reaching in from the lower left to rest its fingers on the keyboard. The laptop screen shows, from top to bottom, a red bar with white lines, a grey panel headed NPM with grey lines and one thick black line, and a beige panel of grey lines.

Correction details (Model watch). Model watch item 1 said: "How the six to seven months is derived from scores, and the full leaderboard, aren't in Epoch's post. GPT-5.4's score isn't given either." That was true of the post. Epoch's data page published the same day (Epoch AI, 2026-10-16) gives the method: it takes the two closed-weight models whose scores most closely bracket GLM-5.3. GPT-5.4, released March 5, scores below GLM-5.3; Anthropic's Mythos Preview, released April 7, scores above it. As of Epoch's data cutoff those releases were seven and six months earlier, which is where six to seven months comes from. The page lists Cyber ECI scores of 157.04 for GPT-5.4 and 160.10 for GLM-5.3, offers the scores as a CSV download, and is licensed CC BY (free to reuse with credit). It also notes that GLM-5.3's score comes from only three benchmark results. The Cyber ECI is still Epoch's own index; the method is published by Epoch and hasn't been independently checked. The email sent on October 17 is the original version. How we checked →

Source and checks

This item ran in the brief of Oct 17, 2026, alongside 15 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "Databricks cuts npm malware misses to 4%; it's unreplicated." https://secondsource.io/en/research/2026-10-17-3/. Accessed access date.