ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents

Computer Science > Cryptography and Security arXiv:2610.41272 (cs) [Submitted on 30 Oct 2026] Title: ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents Authors: Yini Huang, Aiwen LU, Zhen Sun, Yi Han, Yuzhi Zhao, Jiaheng Wei.

计算机科学 > 密码学与安全 arXiv:2610.41272 (cs) [提交于 2026 年 10 月 30 日] 标题：ANGLER：面向 Web 智能体的验证器基础基准测试与自适应攻击者 作者：Yini Huang, Aiwen LU, Zhen Sun, Yi Han, Yuzhi Zhao, Jiaheng Wei。

Abstract: Web agents are increasingly deployed to complete tasks autonomously on real websites, where adversarially injected content can induce them to perform unauthorized actions. Existing evaluations primarily use fixed attacks or focus on a single attack type, and evaluate attacks at the text level rather than through real state changes, thus underestimating how vulnerable agents are to diverse attacks.

摘要：Web 智能体正越来越多地被部署以在真实网站上自主完成任务，其中被恶意注入的内容可能会诱导它们执行未经授权的操作。现有的评估主要使用固定攻击或专注于单一攻击类型，并且是在文本层面而非通过真实状态变化来评估攻击，从而低估了智能体在面对多样化攻击时的脆弱性。

To address the gap, we introduce REEF, a benchmark of red-teaming executable environments with faithful verification for measuring the security vulnerabilities of web agents. REEF covers 9 verifier-grounded websites spanning travel, commerce, productivity, media, and communication, and pairs every authentic benign task with a malicious objective delivered through 5 attack types at both the user and the environment level.

为了弥补这一差距，我们引入了 REEF，这是一个包含红队可执行环境的基准测试，通过可靠的验证来衡量 Web 智能体的安全漏洞。REEF 涵盖了 9 个基于验证器的网站，涉及旅游、商业、生产力、媒体和通信等领域，并将每个真实的良性任务与通过用户和环境层面的 5 种攻击类型所传递的恶意目标配对。

REEF’s state-based verifiers provide the execution feedback that drives ANGLER, an adaptive near-miss-guided lure evolution framework for red-teaming. Rather than evaluating a fixed attack set, ANGLER uses outcomes from prior rollouts to guide LLM-based attack mutation and candidate selection, prioritizing near-miss seeds during refinement.

REEF 基于状态的验证器提供了驱动 ANGLER 的执行反馈，ANGLER 是一个用于红队测试的、由近失（near-miss）引导的自适应诱饵演化框架。ANGLER 不再评估固定的攻击集，而是利用先前执行的结果来指导基于大语言模型（LLM）的攻击变异和候选选择，并在优化过程中优先考虑近失种子。

Across 8 VLM-based web agents, ANGLER achieves an end-to-end attack success rate (ASR) of 20.36%, compared with 15.24% for the non-adaptive setting. Compared with attack-free execution on the same benign tasks, attacks generated by ANGLER reduce benign task success rate (BSR) from 61.9% to 44.4%.

在 8 个基于视觉语言模型（VLM）的 Web 智能体上，ANGLER 实现了 20.36% 的端到端攻击成功率（ASR），而对比非自适应设置下的成功率为 15.24%。与在相同良性任务上的无攻击执行相比，由 ANGLER 生成的攻击将良性任务成功率（BSR）从 61.9% 降低到了 44.4%。