SecondSourceJudgment rebuilt from primary sources

A preprint finds that planting fake risk warnings in content an AI agent reads makes Gemini refuse legitimate tasks 86.3% of the time, against 9.8% without them.

Concept illustration: an open book stands on a beige surface below a black sign lettered PREPRINT on two posts; on the left page, from left to right, are a flat red block with a black band in a black tray, a white box lettered GEMINI holding up a card of grey lines, and a tall beige block with grey lines on top in a black tray. On the right page, from left to right, are a tall red block with a black band in a black tray, a second white box lettered GEMINI whose card of grey lines has a black tag lettered FAKE WARNING clipped over its top, and a short beige block with grey lines on top in a black tray. Printed on the image: 86.3% Gemini refusal rate on legitimate tasks with fake warnings vs 9.8% without; one author's small test.

The more cautious an agent is, the more likely a fake warning is to stall it, and this is a single author's small-scale experiment. Author Charles Xiao tested 3 model families on 51 sets of tasks. According to the abstract, prompts telling models to put caution first amplify the attack, while completion of clean tasks is unaffected. The author's proposed defense is to check a warning against authoritative records or the original content before refusing, which assumes such records exist. The abstract gives neither the Gemini version nor the other two models' numbers (arXiv 2611.14864, 2026-11-11). How we checked →

Source and checks

This item ran in the brief of Nov 14, 2026, alongside 14 other items. How we checked it, the quote checks and what was still missing that day are on that issue's How we checked page.

Cite this item

SecondSource (2026). "A preprint finds that planting fake risk warnings in content an AI agent reads makes Gemini refuse legitimate tasks 86.3% of the time, against 9.8% without them.." https://secondsource.io/en/research/2026-11-14-2/. Accessed access date.