Anthropic gives outside AI watchdog “employee-like” access

Anthropic has granted external evaluators “employee-like access” to assess the risks of its models, following through on a commitment to increase safeguards over concerns the technology could run catastrophically out of control. METR, a nonprofit that has worked with leading labs including OpenAI, Anthropic, Google and Meta, now has a team embedded in the Claude maker’s office with access to its hardware and internal systems.

Anthropic 公司已授予外部评估人员“类似员工的访问权限”以评估其模型的风险，此举旨在兑现其加强安全保障的承诺，以应对人们对该技术可能失控并造成灾难性后果的担忧。METR 是一家曾与 OpenAI、Anthropic、谷歌和 Meta 等领先实验室合作的非营利组织，目前已在 Claude 开发商的办公室内派驻了一个团队，并获准访问其硬件和内部系统。

A greater role for third-party evaluators is among the main concrete steps the labs have proposed in response to rising concern about the dangers more capable AI systems could pose to the economy, cyber security and humanity at large. Anthropic chief executive Dario Amodei achieved a rare moment of unity with arch-rivals Sam Altman and Elon Musk in September when the OpenAI and SpaceX bosses rallied behind his proposal to embed external auditors in the leading labs. Thursday’s announcement marks the first step to implementing these pledges.

在应对更强大的 AI 系统可能对经济、网络安全和全人类构成的危险这一日益增长的担忧时，让第三方评估机构发挥更大作用是各实验室提出的主要具体措施之一。Anthropic 首席执行官达里奥·阿莫代（Dario Amodei）在 9 月份与其主要竞争对手萨姆·奥特曼（Sam Altman）和埃隆·马斯克（Elon Musk）达成了罕见的共识，当时这两位 OpenAI 和 SpaceX 的掌门人支持了他关于在领先实验室中嵌入外部审计员的提议。周四的公告标志着落实这些承诺迈出了第一步。

OpenAI has yet to lay out plans to bring in evaluators after Altman said Amodei’s proposals were a good idea. “As models have become more powerful, they’ve required stronger safeguards and more challenging risk assessment,” Anthropic said on Thursday. “We want to be publicly accountable for the decisions we’re making. But we can’t publicly share all the information relevant to our safety work.”

尽管奥特曼曾表示阿莫代提出的建议是个好主意，但 OpenAI 尚未制定引入评估人员的计划。Anthropic 在周四表示：“随着模型变得越来越强大，它们需要更强的保障措施和更具挑战性的风险评估。我们希望对所做的决定承担公开责任，但我们无法公开分享与安全工作相关的所有信息。”

Critics of Anthropic’s proposal to use evaluators have argued the arrangements could lead to conflicts of interest, pointing to issues in the accounting sector, and that greater government regulation is required. Anthropic argued that sharing information publicly could benefit malicious actors or competitors and added that its evaluators will operate similarly to bank supervision, whereby regulators assign resident examiners to work inside the largest银行. “If a bank’s risk-taking could threaten the public, third parties try to see that in advance,” the company said.

批评 Anthropic 评估人员提议的人士认为，这种安排可能会导致利益冲突，并指出会计行业存在的问题，认为需要加强政府监管。Anthropic 则辩称，公开分享信息可能会让恶意行为者或竞争对手受益，并补充说其评估人员的运作方式将类似于银行监管，即监管机构指派常驻审查员在大型银行内部工作。该公司表示：“如果一家银行的风险承担可能威胁到公众，第三方会尝试提前发现这一点。”

Anthropic is also under pressure to reassure investors that safety issues will not overshadow its initial public offering, expected this year at a more than $2tn valuation. It devoted a large part of investor presentations this week to laying out its safety practices. METR was one of the organisations that investigated the hack of Hugging Face by rogue OpenAI models, reviewing thousands of logs and receiving company laptops.

Anthropic 还面临着向投资者保证安全问题不会掩盖其首次公开募股（IPO）的压力，预计该公司今年将以超过 2 万亿美元的估值上市。本周，该公司在投资者演示中花费了大量篇幅来阐述其安全实践。METR 曾参与调查 OpenAI 模型违规入侵 Hugging Face 的事件，当时该组织审查了数千条日志并接收了公司笔记本电脑。

Anthropic has now onboarded six METR employees, giving them company hardware, office space and “access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to.” The lab said it has encouraged employees to share information with the evaluators as they would with a fellow employee working in risk. Anthropic is not paying METR for this arrangement.

Anthropic 目前已入职了 6 名 METR 员工，为他们提供了公司硬件、办公空间以及“与我们自身风险评估团队所拥有的权限大致相当的内部工具和系统访问权限”。该实验室表示，已鼓励员工像对待从事风险工作的同事一样，与这些评估人员分享信息。Anthropic 并未为此安排向 METR 付费。

The research group does not accept donations from AI companies or their staff. It is funded by donations from individuals and philanthropic organisations, as well as government bodies such as the UK’s AI Security Institute. The evaluators will publish their analysis, and Anthropic cannot veto publication. However, it can require them to redact information that falls into certain categories, such as customer data, details covered by attorney-client privilege or information about model architectures.

该研究小组不接受来自 AI 公司或其员工的捐款。其资金来源于个人和慈善组织的捐赠，以及英国 AI 安全研究所等政府机构。评估人员将发布他们的分析报告，Anthropic 无权否决发布。不过，该公司可以要求他们对属于特定类别的信息进行删减，例如客户数据、受律师-客户特权保护的细节或有关模型架构的信息。

Anthropic added it was still working with Accenture, which was named a partner on safety testing last month, with each committing at least $1 billion to set up new safeguards. Another third-party tester is SecureBio, which reviews chemical and biological weapon risk for the company. “We’re embedding evaluators voluntarily, and we believe every frontier AI company should do the same,” Anthropic said. METR said: “We stand ready to work with all frontier AI developers to bring better evidence to the public.”

Anthropic 补充说，它仍在与上个月被提名为安全测试合作伙伴的埃森哲（Accenture）合作，双方各承诺投入至少 10 亿美元用于建立新的保障措施。另一家第三方测试机构是 SecureBio，负责为该公司审查化学和生物武器风险。Anthropic 表示：“我们是自愿嵌入评估人员的，我们认为每一家前沿 AI 公司都应该这样做。”METR 则表示：“我们随时准备与所有前沿 AI 开发商合作，为公众提供更好的证据。”