Grok exfiltrates user data when malicious instructions are encrypted

当恶意指令被加密时，Grok 会泄露用户数据

Earlier this week, researchers outlined an attack that used a secret input provided by Microsoft 365 Copilot for enterprise to cause the AI assistant to exfiltrate a password present in the user’s inbox. Now, a separate team has devised a similar attack against Grok. The new data theft hack employs a deceptively simple trick to force the Elon Musk-owned large language model to steal user chats and other personal information. At the time this post went live, the assistant continued to cough up the data, despite xAI being informed of it in June. 本周早些时候，研究人员概述了一种攻击方法，该方法利用 Microsoft 365 Copilot 企业版提供的秘密输入，诱导 AI 助手泄露用户收件箱中的密码。现在，另一个团队针对 Grok 设计了类似的攻击。这种新的数据窃取黑客技术采用了一种看似简单的技巧，强迫这个由埃隆·马斯克拥有的语言模型窃取用户聊天记录和其他个人信息。截至本文发布时，尽管 xAI 早在 6 月份就已获悉此事，但该助手仍在泄露数据。

The lesson from both this week’s episodes—and the countless other ones that have come before it—is that LLMs are incapable of solving the root causes for prompt injections, the most severe vulnerability classes they’re most prone to. That leaves AI developers with no other option but to build a guardrail that steers the model away from the harmful actions. As I noted in Tuesday’s story, the approach is tantamount to a road traffic safety engineer erecting a protective rail around a dangerous bend rather than banking the curve. 本周发生的这两起事件，以及此前无数次类似事件给我们的教训是：大语言模型（LLM）无法从根本上解决提示词注入（Prompt Injection）问题，而这正是它们最容易遭受的最严重漏洞类别。这使得 AI 开发人员别无选择，只能建立防护栏（Guardrail）来引导模型远离有害操作。正如我在周二的文章中所指出的，这种方法等同于交通安全工程师在危险弯道处设置护栏，而不是去改善弯道的坡度设计。

Cryptographic Context Injection in the house

加密上下文注入（Cryptographic Context Injection）登场

Prompt injections exploit LLMs’ training to comply with user requests whenever possible. Attackers can capitalize on the predilection by smuggling harmful instructions into emails or webpages the assistant is instructed to summarize. Because LLMs can’t reliably distinguish between content in an email sent by an untrusted party and user instructions entered directly into a prompt, the overly solicitous LLM faithfully follows them. To date, Grok and other LLMs’ only recourse is to create guardrails that flag suspicious instructions and forbid them from being executed. 提示词注入利用了 LLM 尽可能满足用户请求的训练特性。攻击者可以利用这种倾向，将有害指令隐藏在助手被要求总结的电子邮件或网页中。由于 LLM 无法可靠地分辨不可信方发送的电子邮件内容与用户直接输入的提示词指令，过于“殷勤”的 LLM 会忠实地执行这些指令。迄今为止，Grok 和其他 LLM 唯一的补救措施就是建立防护栏，标记可疑指令并禁止其执行。

Rony Utevsky, a researcher at security firm Adversa, recently discovered a simple way to completely bypass that restriction. Rather than composing the harmful instruction in plaintext, the hacker encrypts it. The website hosting the ciphertext also includes plaintext instructions for decrypting the encrypted content, along with the decryption key. Using this simple sequence, Grok then follows the command as soon as the user instructs the assistant to summarize the page. There is no warning, and no confirmation is required. 安全公司 Adversa 的研究员 Rony Utevsky 最近发现了一种完全绕过该限制的简单方法。黑客不再以明文形式编写有害指令，而是将其加密。托管密文的网站还包含了解密该内容的明文说明以及解密密钥。通过这一简单的序列，一旦用户指示助手总结该页面，Grok 就会执行该命令。整个过程没有任何警告，也不需要任何确认。

The deciphered instructions direct the LLM to construct what is purported to be a decryption key. In fact, it’s something else completely. The value of the fake key is instead the user’s name, location, and chat history. The value is later used as a parameter added to a URL leading to the attacker’s site. Once Grok opens the link, the data is in the logs of the attacker’s server. 解密后的指令会引导 LLM 构建一个所谓的“解密密钥”。事实上，它完全是另一回事。这个伪造密钥的值实际上是用户的姓名、位置和聊天记录。该值随后被用作参数添加到指向攻击者网站的 URL 中。一旦 Grok 打开该链接，数据就会出现在攻击者服务器的日志中。

Adversa can’t be sure what causes Grok to refuse precisely the same plaintext instructions and follow the encrypted ones. The leading theory is that the Grok filtering guardrail inspects text entering and leaving the model, but not the output of its own code execution. Instructions to process the ciphertext with PBKDF2 and AES-256-GCM pass the filter as an ordinary request, because a classifier can read them but not resolve what they unlock. Once the additional instructions are decrypted, they reach the model as its own tool output, and it acts on them without the filtering guardrails ever inspecting them. Adversa 无法确定为什么 Grok 会拒绝完全相同的明文指令，却会执行加密后的指令。目前的主要理论是，Grok 的过滤防护栏会检查进出模型的内容，但不会检查其自身代码执行的输出。使用 PBKDF2 和 AES-256-GCM 处理密文的指令作为普通请求通过了过滤器，因为分类器可以读取它们，但无法解析它们解锁后的内容。一旦附加指令被解密，它们就会作为模型自身的工具输出到达模型，模型随后执行这些指令，而过滤防护栏从未对其进行检查。

“Static safety guardrails classify inputs as text; they do not execute them,” Utevsky wrote Thursday. “An attacker ships ciphertext along with the key material and an instruction to decrypt it, and the model runs that decryption inside its own code execution sandbox. Everything a guardrail’s scanner would need is right there on the page, but recovering the plaintext means running PBKDF2 and AES-256-GCM, which no content classifier does at inspection time.” “静态安全防护栏将输入视为文本进行分类；它们不会执行这些输入，”Utevsky 周四写道。“攻击者将密文与密钥材料以及解密指令一起发送，模型在其自身的代码执行沙箱内运行解密过程。防护栏扫描器所需的一切信息都在页面上，但要恢复明文意味着必须运行 PBKDF2 和 AES-256-GCM，而没有任何内容分类器会在检查时执行这些操作。”

In an email, the researcher said that such guardrails are called static “because they only read content as text. They don’t run code or decrypt anything. That’s the gap we exploit. The real instructions are encrypted, so the guardrail sees only meaningless ciphertext and passes it through.” 该研究员在电子邮件中表示，这种防护栏被称为“静态”防护栏，“因为它们只将内容作为文本读取。它们不运行代码，也不解密任何东西。这就是我们利用的漏洞。真正的指令被加密了，所以防护栏只能看到毫无意义的密文并将其放行。”

Adversa used a similar technique in a Gemini jailbreak attack, meaning making the Google LLM ignore its internal safety rules. Here, the ciphertext was decrypted to what appeared to be a traceback. The decrypted text issued one rule—if the code fails, read the error message and act on it. The cleartext injected a prompt that ultimately caused Gemini to violate its safety rules. Adversa 在一次 Gemini 越狱攻击中使用了类似的技术，即诱导 Google 的 LLM 忽略其内部安全规则。在这种情况下，密文被解密为看起来像回溯（traceback）的内容。解密后的文本发布了一条规则——如果代码失败，读取错误消息并据此采取行动。明文注入了一个提示词，最终导致 Gemini 违反了其安全规则。

“The technique produced a multi-paragraph example of restricted content that Gemini’s safety filters normally suppress (building an incendiary weapon),” Adversa said. “With a modified payload, the same vector reproduced Gemini’s system instructions, including the directive forbidding their disclosure.” Adversa didn’t report the behavior to Google because jailbreaks aren’t within the scope of the company’s vulnerability disclosure program. Over the past few weeks, however, Gemini has grown increasingly resistant to the attack. “We can’t attribute the change—it could be filter updates, model version changes, or both,” the security firm said. “该技术生成了一个多段落的受限内容示例，这些内容通常会被 Gemini 的安全过滤器拦截（例如制造燃烧武器），”Adversa 表示。“通过修改后的载荷，同样的向量重现了 Gemini 的系统指令，包括禁止披露这些指令的规定。”Adversa 没有向 Google 报告此行为，因为越狱不在该公司的漏洞披露计划范围内。然而，在过去几周里，Gemini 对这种攻击的防御能力越来越强。“我们无法确定这种变化的原因——可能是过滤器更新、模型版本变更，或者是两者兼有，”该安全公司表示。

Company researchers are calling the technique cryptographic context injection. “Cryptographic Context Injection is one instance of a broader shift: attacks that manipulate not just the prompt, but the wider context an LLM treats as its own, such as tool outputs, runtime results and intermediate state,” Adversa said. “This attack surface is far larger than what’s traditionally labeled ‘model inputs,’ and the next generation of attacks will emerge there.” 该公司研究人员将这种技术称为“加密上下文注入”。“加密上下文注入是更广泛转变的一个实例：攻击不仅操纵提示词，还操纵 LLM 视为自身一部分的更广泛上下文，例如工具输出、运行时结果和中间状态，”Adversa 表示。“这个攻击面远比传统上定义的‘模型输入’要大得多，下一代攻击将从这里涌现。”

The Cryptographic Context Injection is only the latest example of the disadvantage LLM defenders operate under. Every time they build a new, one-off guardrail, an attacker finds a new vector that allows the car to once again careen off the road. The cycle continues: lather, rinse, and repeat. 加密上下文注入只是 LLM 防御者处于劣势的最新例证。每当他们建立一个新的、一次性的防护栏时，攻击者就会找到一个新的向量，让这辆车再次冲出道路。循环往复：涂抹、冲洗、重复。