Microsoft says AI agents are like insider threats as ‘frontier models can’t be trusted’

Microsoft on Monday introduced new security features designed to protect enterprises from the growing threat of AI agents. The launch comes at Microsoft continues to warn enterprises about the dangers of AI and agents. Perhaps the most notable addition is Guardian Agents, a new feature intended to monitor the AI agents a company uses. Guardian Agents is part of Project Perception, a product introduced in June.

微软周一推出了新的安全功能，旨在保护企业免受日益增长的 AI 智能体威胁。此次发布正值微软持续向企业发出有关 AI 和智能体危险的警告之际。其中最引人注目的新增功能是“守护者智能体”（Guardian Agents），这是一项旨在监控公司所使用 AI 智能体的新功能。守护者智能体是 6 月份推出的产品“感知项目”（Project Perception）的一部分。

Project Perception was Microsoft’s answer to powerful hacking models like Anthropic’s Mythos and Fable and OpenAI’s security vulnerability scanning system Daybreak. Project Perception originally launched three classes of agents: a “red team” that acted like hackers and attacked, a “blue team” that detected the red team and assessed the vulnerabilities they found, and a “green team” tasked with fixing them. The product was designed around the assumption that AI hackers would come from outside the company.

“感知项目”是微软针对 Anthropic 的 Mythos 和 Fable 等强大黑客模型，以及 OpenAI 的安全漏洞扫描系统 Daybreak 所做出的回应。该项目最初推出了三类智能体：充当黑客进行攻击的“红队”、检测红队并评估其发现漏洞的“蓝队”，以及负责修复漏洞的“绿队”。该产品的设计初衷是基于 AI 黑客来自公司外部的假设。

But lately, Microsoft CEO Satya Nadella has been warning that the AI enemy may be the very agents enterprises have deployed and are paying for, much like a rogue human employee, otherwise known as an “insider threat.” Earlier this month, in his private blog, Nadella warned that it’s time to stop trusting AI model providers, even if they offer assurances of safety and security with their agentic products.

但最近，微软首席执行官萨提亚·纳德拉（Satya Nadella）一直在警告称，AI 敌人可能正是企业已经部署并付费使用的那些智能体，它们就像流氓员工一样，也就是所谓的“内部威胁”。本月初，纳德拉在其个人博客中警告称，是时候停止信任 AI 模型提供商了，即使他们为其智能体产品提供了安全保障。

He wants enterprises to implement separate systems, not controlled by the AI labs, to watch and control the agents that those AI labs’ products launch. “We are deploying these complex agentic systems and models, with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!” he wrote. “We simply can’t outsource responsibility for what intelligence does on our behalf. A model provider’s assurances do not relieve us of that responsibility. … In other words, we need to separate the supply of intelligence from the authority over it.”

他希望企业能够实施不受 AI 实验室控制的独立系统，以监视和控制这些实验室产品所启动的智能体。“我们正在部署这些复杂的智能体系统和模型，它们可以访问我们最敏感的数据，并有权代表我们采取关键任务行动！”他写道，“我们绝不能将智能代表我们所做行为的责任外包出去。模型提供商的保证并不能免除我们的责任……换句话说，我们需要将智能的供应与对智能的控制权分离开来。”

When announcing Guardian Agents, Microsoft’s security business chief Hayete Gallot was even more straightforward. “Unfortunately, frontier models, regardless of their origin, cannot be trusted,” she wrote, adding that this isn’t because they are malicious or easily hacked, but because they are unpredictable. They have been trained to achieve tasks, not follow “organizational norms or judgment.” On top of that, they are often given the keys to the kingdom, with access to data and tools that could do serious harm.

在宣布“守护者智能体”时，微软安全业务主管 Hayete Gallot 的表态更为直接。“不幸的是，前沿模型无论其来源如何，都不可信，”她写道，并补充说这并非因为它们是恶意的或容易被黑，而是因为它们具有不可预测性。它们受训的目标是完成任务，而不是遵循“组织规范或判断”。此外，它们往往被赋予了核心权限，能够访问可能造成严重损害的数据和工具。

Microsoft’s solution is both surprising and self-serving. The surprising side is that Microsoft is famously close to OpenAI and Anthropic, having invested in both. OpenAI is also one of its biggest cloud customers, and Microsoft has its own AI lab developing models. So, Microsoft is sowing distrust against some of its own business interests. The self-serving side is that Microsoft wants enterprises to buy agent security products from it, the cloud provider, rather than from Anthropic and OpenAI.

微软的解决方案既令人惊讶又带有私心。令人惊讶之处在于，微软与 OpenAI 和 Anthropic 关系密切，且对两者均有投资。OpenAI 也是其最大的云客户之一，微软自己也有开发模型的 AI 实验室。因此，微软实际上是在对其自身的部分商业利益播下不信任的种子。带有私心的一面则是，微软希望企业从它（云服务提供商）那里购买智能体安全产品，而不是从 Anthropic 和 OpenAI 那里购买。

But conflicting interests aside, its logic is sound. The software that monitors agent behavior should be separate from the suppliers of agents, just as auditors should be independent of the organizations they audit. Or as Nadella puts it: “We need to separate the supply of intelligence from the authority over it.”

但撇开利益冲突不谈，其逻辑是合理的。监控智能体行为的软件应该与智能体供应商分离，就像审计师应该独立于他们所审计的组织一样。正如纳德拉所言：“我们需要将智能的供应与对智能的控制权分离开来。”

Microsoft is now seizing that business opportunity in a variety of ways. In addition to Guardian Agents, it has added an agent detection feature to its Microsoft Defender product that can find up to 60 types of agents that employees bring into the workplace, including Claude Code, Codex, and OpenClaw. It has also added features that limit agents’ access to data and systems, and dynamically adjust their agent permissions based on the task. That includes requiring human approval if an agent appears to be making risky decisions.

微软目前正通过多种方式抓住这一商业机会。除了“守护者智能体”之外，它还在其 Microsoft Defender 产品中增加了智能体检测功能，可以发现员工带入工作场所的多达 60 种智能体，包括 Claude Code、Codex 和 OpenClaw。它还增加了限制智能体访问数据和系统的功能，并根据任务动态调整智能体的权限。这包括在智能体似乎做出高风险决策时要求人工审批。

This idea of agents as untrustworthy insiders is taking hold across the broader industry, too. Microsoft was an inaugural member of Nvidia’s new Open Agent Safety Platform, which garnered the support of more than 100 other tech companies. The platform isolates agents in secure environments and can monitor and stop behavior through hardware, functioning something like a kill switch. This follows Microsoft’s announcement of the new Surface Laptop Ultra PCs built for agents, which run on Nvidia chips and have agent security and controls baked in.

将智能体视为不可信的内部人员这一理念也在整个行业中扎根。微软是英伟达新的“开放智能体安全平台”（Open Agent Safety Platform）的创始成员，该平台获得了 100 多家其他科技公司的支持。该平台将智能体隔离在安全环境中，并能通过硬件监控和停止其行为，功能类似于“终止开关”。在此之前，微软还宣布了专为智能体打造的全新 Surface Laptop Ultra PC，这些电脑运行在英伟达芯片上，并内置了智能体安全和控制功能。