Meta’s Muse AI Assistant Rolled Out With a Serious Security Flaw
Meta’s Muse AI Assistant Rolled Out With a Serious Security Flaw
Meta 的 Muse AI 助手发布即曝严重安全漏洞
Meta founder and CEO Mark Zuckerberg has gone to great lengths to hype the security of its new AI assistant, Muse, claiming it is “built from the ground up for privacy and security.” A zero-day vulnerability that gives locally run apps and terminal commands complete control of the agent raises serious doubts. Further raising questions, Amazon on Sunday began blocking Muse from its site.
Meta 创始人兼首席执行官马克·扎克伯格曾不遗余力地宣传其新款 AI 助手 Muse 的安全性，声称它是“从底层开始就为隐私和安全而构建的”。然而，一个允许本地运行的应用程序和终端命令完全控制该助手的零日漏洞引发了严重质疑。此外，亚马逊已于周日开始在其网站上封禁 Muse，这进一步加剧了人们的疑虑。
Meta introduced Muse a few weeks ago. The assistant “books appointments, fills out forms, and handles customer service,” “proactively takes tasks off your plate,” and can “make purchases, generate images, create documents, and connect with your favorite apps and services.” The macOS app (curiously, there’s no Windows version) also works with a user’s WhatsApp, email, calendar, and social media accounts. When a task requires a tool that doesn’t exist, Muse creates one on the fly.
Meta 在几周前推出了 Muse。该助手可以“预约、填写表格并处理客户服务”、“主动分担你的任务”，还能“进行购物、生成图像、创建文档，并连接到你最喜爱的应用程序和服务”。这款 macOS 应用程序（奇怪的是没有 Windows 版本）还可以与用户的 WhatsApp、电子邮件、日历和社交媒体账户协同工作。当任务需要某种尚不存在的工具时，Muse 会即时创建一个。
Meta Doth Hype Muse Security Too Much
Meta 对 Muse 安全性的吹捧过头了
Of course, for Muse to do any of these things, users must first give it access to their accounts. This includes authenticating the assistant to each service and, because the app runs on macOS, giving it permissions to a broad range of operating system-restricted device resources, like writing files to disk, accessing the mic and camera, and monitoring location and calendars. Apple has spent years developing these defenses to prevent installed apps or commands entered into the terminal from accessing these resources, clearly because the company considers them a security threat. Muse completely undoes these default measures.
当然，为了让 Muse 完成这些任务，用户必须首先授予它访问其账户的权限。这包括对每个服务进行身份验证，并且由于该应用运行在 macOS 上，还需要授予它访问各种受操作系统限制的设备资源的权限，例如向磁盘写入文件、访问麦克风和摄像头，以及监控位置和日历。苹果公司多年来一直致力于开发这些防御措施，以防止已安装的应用程序或在终端中输入的命令访问这些资源，显然是因为苹果认为这些行为构成安全威胁。而 Muse 完全绕过了这些默认的安全措施。
The zero-day allowed any app or terminal command to gain access to the token that authenticates users to their Muse account. Meta developers designed the assistant so that any locally installed app or executed code, regardless of the macOS permissions it has, can change a long list of undocumented settings. Most of them are fairly innocuous, such as controlling dark mode. One setting, however, was anything but innocuous. It allowed processes to change the end point where transcription occurs. Normally, it’s a server address operated by Meta. Attackers could have exploited this flaw by changing the location to their own end point. If that happened, the attackers would have had the token that gives complete control over the Muse account.
该零日漏洞允许任何应用程序或终端命令获取用于验证用户 Muse 账户的令牌。Meta 的开发人员在设计该助手时，使得任何本地安装的应用程序或执行的代码，无论其拥有何种 macOS 权限，都可以更改一长串未公开的设置。其中大多数设置相当无害，例如控制深色模式。然而，其中一个设置却绝非无害——它允许进程更改语音转录的端点。通常，这是一个由 Meta 运营的服务器地址。攻击者可以利用这一漏洞，将该位置更改为他们自己的端点。如果发生这种情况，攻击者将获得能够完全控制 Muse 账户的令牌。
“We can manipulate the agent and leverage its privileges to do whatever we want,” Patrick Wardle, the macOS security expert who discovered the zero-day, told Ars ahead of the hotfix. “So instead of us having to write a very comprehensive Mac malware stealer, we can just leverage the AI assistant itself.” Wardle said he has developed several proof-of-concept attacks that do things like writing malicious files to disk and snapping pictures, in many cases with no indication to even an alert user.
“我们可以操纵这个助手，利用它的权限做任何我们想做的事，”在热修复补丁发布前，发现该零日漏洞的 macOS 安全专家 Patrick Wardle 对 Ars 说道。“所以我们不需要编写复杂的 Mac 恶意窃取软件，直接利用这个 AI 助手本身就可以了。”Wardle 表示，他已经开发了几种概念验证攻击，例如向磁盘写入恶意文件和拍摄照片，在许多情况下，即使是警觉的用户也无法察觉。
More than 12 hours after this post went live, Meta said it released a hotfix that patched the 0-day.
在这篇文章发布 12 小时后，Meta 表示已发布热修复补丁，修复了该零日漏洞。
Meta has published two posts in as many weeks documenting the design decisions that went into ensuring an assistant with such extraordinary access to user data and resources is secure and private. The posts come amid revelations that internal testing of models from Anthropic and Google has resulted in security breaches of external, third-party networks that the engineers involved never intended to target. In traditional human-only hacking, these actions could likely result in the filing of criminal charges. The Meta posts are likely mindful of the resulting blowback and the calls to slow down AI development in response.
Meta 在两周内发布了两篇文章，记录了其为确保这样一个拥有极高用户数据和资源访问权限的助手具备安全性和隐私性所做的设计决策。这些文章发布之际，正值有消息披露 Anthropic 和 Google 的模型内部测试导致了外部第三方网络的安全漏洞，而相关工程师从未打算针对这些网络。在传统的纯人工黑客攻击中，这些行为很可能会导致刑事指控。Meta 发布这些文章很可能是为了应对由此产生的负面影响以及要求放缓 AI 开发的呼声。
Wardle said that Meta developers made several design decisions that made his exploit possible. One is the choice for Muse dictation to occur in the cloud, where Meta can log it. macOS has long provided a simple means for apps to handle dictation and transcription in processes that stay securely on the device. Had the developers chosen this safer alternative, the attack wouldn’t have been possible.
Wardle 指出，Meta 开发人员做出的几个设计决策使得他的攻击成为可能。其中之一是选择将 Muse 的听写功能放在云端处理，这意味着 Meta 可以记录这些数据。macOS 长期以来一直为应用程序提供了一种简单的方法，可以在设备上安全地处理听写和转录。如果开发人员选择了这种更安全的替代方案，那么这次攻击将无法实现。
Another flawed decision is for any app to control all of the undocumented settings. It’s likely Meta intended for apps working with Muse to control UI settings, and for understandable reasons. The ability for any app or command to control an end point where sensitive user speech is processed is an entirely different matter. Together, the design decisions raise questions about just how much effort developers put into designing and testing the security and privacy of the new assistant.
另一个错误的决定是允许任何应用程序控制所有未公开的设置。Meta 的初衷可能是为了让与 Muse 协作的应用程序能够控制 UI 设置，这在情理之中。但允许任何应用程序或命令控制处理用户敏感语音的端点，则是完全不同的性质。这些设计决策共同引发了人们的质疑：开发人员在设计和测试这款新助手的安全性和隐私性方面到底投入了多少精力？
“To me, the bar is infinitely higher in terms of the security of these apps. They don’t have to be perfect, but when you take a look at Muse, it’s like they didn’t, in my opinion, think about security, which is really worrisome,” Wardle said. “At the very least, they should be thinking about security from the very start, and they are just not.”
“对我来说，这些应用程序的安全门槛要高得多。它们不必完美，但当你审视 Muse 时，在我看来，他们根本没有考虑过安全性，这非常令人担忧，”Wardle 说。“至少，他们应该从一开始就考虑安全性，但他们显然没有这样做。”
Roughly 12 hours before Wardle disclosed the zero-day, Amazon started blocking people from using Muse to shop on the site. Users who tried received a message saying Muse was an “unauthorized AI agent [that] violates Amazon’s Conditions of Use.”
在 Wardle 披露该零日漏洞前约 12 小时，亚马逊开始阻止用户使用 Muse 在其网站上购物。尝试使用的用户会收到一条消息，称 Muse 是一个“未经授权的 AI 代理，违反了亚马逊的使用条款”。
“We think it’s fairly straightforward that third-party applications that offer to make purchases on behalf of customers from other businesses should operate openly and respect service provider decisions about whether or not to participate,” Amazon said in an emailed statement. “This helps ensure a safe, secure, and reliable customer experience, and it is how others operate including food delivery apps and the restaurants they take orders for, delivery services apps and the stores they shop from, and online travel agencies and the airlines they book tickets with for customers. Agentic third-party applications such as Muse have the same obligations, and we’ve requested that Meta remove Amazon from the experience.”
“我们认为，第三方应用程序如果提供代表客户从其他企业进行购买的服务，就应该公开透明，并尊重服务提供商关于是否参与的决定，这一点显而易见，”亚马逊在电子邮件声明中表示。“这有助于确保安全、可靠的客户体验。其他行业也是这样运作的，包括外卖应用与餐厅、配送服务应用与商店，以及在线旅行社与航空公司。像 Muse 这样的代理型第三方应用程序也负有同样的义务，我们已要求 Meta 将亚马逊从其体验中移除。”
A Single ClickFix Is All It Takes
只需一次点击即可修复
There are several ways for attacks to work. One is for an attacker’s server to act as a proxy that’s placed between the Muse user and Meta end point. Once the user enters the voice prompt, the attacker’s server adds a prompt…
攻击有多种实现方式。其中一种是攻击者的服务器充当代理，置于 Muse 用户和 Meta 端点之间。一旦用户输入语音提示，攻击者的服务器就会添加一个提示……