Anthropic can’t reliably control its AI agents. It’s cutting off its internal evals from the live internet instead

Anthropic said its models exploited websites on the internet, including some run by U.S. government agencies, and it will turn off live internet access for all of its internal evaluations until the frontier lab is sure it can monitor and control its AI agents. The incidents, disclosed in a blog post, involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, avoided paywalls and anti-bot restrictions, used URL shortening services to smuggle information pass restrictions, and even submitted a false murder tip to the Philadelphia police.

Anthropic 表示，其模型曾利用互联网上的网站，包括一些由美国政府机构运营的网站。在确保能够监控和控制其 AI 智能体之前，该公司将关闭所有内部评估的实时互联网访问权限。在一篇博客文章中披露的这些事件涉及被指派在互联网上寻找资源以解决问题的 AI 智能体。在此过程中，它们利用了软件漏洞，规避了付费墙和反机器人限制，使用网址缩短服务绕过限制传输信息，甚至向费城警方提交了虚假的谋杀举报。

Anthropic said it discovered these new issues in a review of its model’s activities that began in July, underscoring the lab’s lack of awareness of its software’s behavior. Notably, the company said that alignment training was not yet sufficient for skills like search and computer use that are central to its pitch that AI agents will be used by any professional who relies on digital tools. The behaviors Anthropic disclosed are similar to incidents involving OpenAI agents that collaborated to break into various websites in search of information, including some run by the Australian government.

Anthropic 表示，这些新问题是在 7 月份开始对其模型活动进行审查时发现的，这凸显了该实验室对其软件行为缺乏了解。值得注意的是，该公司称，对于搜索和计算机使用等技能，目前的对齐训练尚不充分，而这些技能是其“AI 智能体将被任何依赖数字工具的专业人士使用”这一宣传的核心。Anthropic 披露的行为与 OpenAI 智能体涉及的事件类似，后者曾协同入侵各种网站以搜寻信息，其中包括一些由澳大利亚政府运营的网站。

Anthropic previously disclosed that its models had broken into external systems. The frontier lab said it considered today’s disclosures “significantly less severe from an alignment and security perspective” than those it announced before. However, the lab still said it had “turned off live internet access” for “all our internal evaluations” until it is certain it can monitor and control its agents. It’s not clear what that means, but Sydney Von Arx, the founder of Nightingale, an AI safety organization, told TechCrunch in an interview before this disclosure that developing models on a data center cut off from the open internet would be very challenging for researchers to use, and for the progress of the models, which benefit from internet access.

Anthropic 此前曾披露其模型曾入侵过外部系统。该前沿实验室表示，从对齐和安全的角度来看，它认为今天的披露内容“比之前宣布的事件严重程度要低得多”。然而，该实验室仍表示，在确定能够监控和控制其智能体之前，已为“我们所有的内部评估”关闭了“实时互联网访问”。目前尚不清楚这意味着什么，但 AI 安全组织 Nightingale 的创始人 Sydney Von Arx 在此次披露前接受 TechCrunch 采访时表示，在与开放互联网隔绝的数据中心开发模型，对研究人员的使用以及依赖互联网访问的模型进展来说，将是非常具有挑战性的。

“You have to align them at some point,” Von Arx said. “If the AIs are released to production and never have access to the internet, that’s not a very useful tool.” Anthropic said the behavior was a result of flaws in the lab’s training environments, which led the models to believe they would be rewarded for finding loopholes or avoiding restrictions, a behavior called “reward hacking.” The company said it would stop running some of its evaluations or move them offline, and has built tooling to detect and block this behavior.

“你必须在某个阶段对它们进行对齐，”Von Arx 说。“如果 AI 被发布到生产环境却永远无法访问互联网，那它就不是一个非常有用的工具。”Anthropic 表示，这种行为是实验室训练环境中的缺陷导致的，这使得模型认为寻找漏洞或规避限制会获得奖励，这种行为被称为“奖励黑客攻击”。该公司表示将停止进行部分评估或将其转为离线进行，并已构建了用于检测和阻止此类行为的工具。

This tooling was tested against the kind of incidents disclosed today and blocked them; it’s not clear what evidence will prompt Anthropic to return live internet access to its internal evaluations. Anthropic also said it would migrate its internal AI agents to “centrally managed infrastructure with strong containment,” and is beginning to using safety classifiers more frequently to monitor those agents.

该工具已针对今天披露的此类事件进行了测试并成功拦截；目前尚不清楚需要什么样的证据才能促使 Anthropic 恢复其内部评估的实时互联网访问权限。Anthropic 还表示，将把其内部 AI 智能体迁移到“具有强大隔离能力的集中管理基础设施”中，并开始更频繁地使用安全分类器来监控这些智能体。