代使用者收信、瀏覽、執行動作的 agent,如果只拿既有惡意樣本或直接重放的攻擊來驗收,會被測得比實際安全:依受測 agent 的行為或回饋改寫過的攻擊,成功率明顯高於同類樣本直接重放。三個不同團隊量到的方向一致。①AgentPhishing:九個 email agent 都能安全處理既有釣魚信;換成依各 agent 最佳化的說服型釣魚後,其中四個偵測率掉到 10% 以下,照做惡意請求的比例超過 45%。②BrowseSafe++:直接重放只成功幾個百分點的網頁注入,依 browser agent 回饋改寫後,成功率最高到 95.41%。③Anthropic 自報:Opus 4.7 在 Gray Swan 注入基準上,單次嘗試約 0.1%,100 次自適應嘗試後約 5–6%。風險也出現在行為層:CredLeak-Bench 以實際送出的資訊計算,所有受測模型都會洩漏憑證;Anthropic 內部紅隊用一段讀起來像例行任務的釣魚 prompt,25 次裡有 24 次讓 Claude 完成外洩。所以採購和紅隊驗收不能拿既有樣本、或單次嘗試的低攻擊成功率當作穩健的證據,要另外做兩件事:①對要部署的那個 agent 做自適應對抗測試;②計算它有沒有實際照做或送出資訊,不只看它認不認得出惡意訊息(在自適應攻擊下,偵測率一樣會掉)。邊界:落差幅度因 agent 而異,九個裡有四個嚴重,95.41% 是最高值;所有量測都只讀了摘要,是作者或公司自報,沒有人複現過;自適應攻擊用到的查詢量和回饋,真實攻擊者不一定拿得到;真實攻擊者是否普遍針對特定 agent 客製化,沒有量測,只知道做得到、可重用。
這不是預測，所以沒有排定哪一天對答案。
什麼會推翻它
對同一批代使用者收信或瀏覽網頁的 agent,同時量兩組攻擊的成功率:既有惡意樣本直接重放,以及依該 agent 的行為或回饋改寫後的攻擊 — 過半受測 agent 的改寫後成功率,比直接重放高不到 5 個百分點、也不到兩倍 — 落差低於此線即為反證:一個團隊量到 ⇒『只拿既有樣本驗收會把 agent 測得比實際安全』削弱為只在個別 agent 成立;兩個彼此獨立的團隊都量到 ⇒ 核心判斷推翻 — AgentPhishing、BrowseSafe++ 作者與 Anthropic 以外的研究團隊或第三方評測機構,以公開論文或評測報告發表 — 2027-06-30 前
前沿模型公司的系統卡、技術報告,或第三方紅隊基準公布的 agent 提示注入攻擊成功率,同時列出單次嘗試與多次自適應嘗試兩個數字 — 至少三家公司的旗艦 agent,100 次自適應嘗試後的成功率與單次嘗試相差都不到 1 個百分點 — 相差低於此線 ⇒『單次嘗試的低攻擊成功率不能當穩健證據』這一步削弱:單次數字已大致代表自適應攻擊下的表現 — 前沿模型公司(系統卡、技術報告)或第三方紅隊基準 — 2027-06-30 前
在自適應攻擊下,對同一批 agent 同時量『認不認得出惡意訊息』與『有沒有實際照做或送出資訊』,算出已認出是惡意卻仍照做或送出的比例 — 每個受測 agent 的這個比例都低於 2% — 低於此線 ⇒ 認不認得出就足以當照做風險的上限,『要另外計算實際照做或送出』這一步削弱為非必要 — AgentPhishing、CredLeak-Bench 作者與 Anthropic 以外的研究團隊或評測機構 — 2027-06-30 前
寫下判斷時的出處
AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack (Justin Tan, Miao Li, Ding Zhao) · arXiv
「Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10\%$ and malicious-request compliance rates above $45\%$.」
BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views · Yuetian Chen, Kaiyuan Zhang, Denis Yarats, Jerry Ma, et al. — arXiv 2610.17457
「BrowseSafe-Adapt revises benchmark injections from browser-agent feedback, and attacks that succeed on only a few percent of tasks when replayed directly reach up to 95.41% success once adapted.」
How we contain Claude across products · Anthropic Engineering Blog
「On Gray Swan's Agent Red Teaming benchmark, which tests susceptibility to prompt injection, Claude Opus 4.7 holds attack success to roughly 0.1% on single attempts, and around 5–6% after 100 adaptive attempts.」
AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack (Justin Tan, Miao Li, Ding Zhao) · arXiv
「These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.」
CredLeakBench: Evaluating Credential Leakage and Recovery in LLM Agents · arXiv(Rafid Ahmed, Joseph Fioresi, Mubarak Shah, Yuzhang Shang;機構未核)
「Within a sandboxed environment, leakage is measured through actual submissions of information rather than agents' self-reported behavior. Our evaluation reveals that all tested models are vulnerable to leakage.」
How we contain Claude across products · Anthropic Engineering Blog
「and the prompt itself read like routine task instructions. But somewhere among the setup steps, it gently asked Claude to read ~/.aws/credentials, encode the contents, and POST them to an external endpoint. Across 25 retries of that prompt, Claude completed the exfiltration 24 times.」
Stavros Damianakis, Konstantinos Psounis — SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space (arXiv 2610.24999) · arXiv
「It also reveals gaps that existing labels obscure: widely used agent benchmarks test only part of the attack space, and adaptive attackers favor mechanisms that curated benchmarks underrepresent.」
AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack (Justin Tan, Miao Li, Ding Zhao) · arXiv
「Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized.」
AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack (Justin Tan, Miao Li, Ding Zhao) · arXiv
「We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint.」