Once popular for attacking AI, ASCII smuggling is embraced by spammers

Once popular for attacking AI, ASCII smuggling is embraced by spammers

曾经用于攻击 AI 的“ASCII 走私”技术,如今被垃圾邮件发送者所采用

A clever technique used to hide malicious prompts in attacks on AI agents has been adopted by spammers to evade filters on email platforms that are designed to flag unwanted messages used in mass campaigns. The technique is broadly known as ASCII smuggling. 一种用于在攻击 AI 智能体时隐藏恶意提示词的巧妙技术,已被垃圾邮件发送者采用,旨在绕过电子邮件平台为拦截大规模垃圾邮件而设计的过滤器。这种技术被统称为“ASCII 走私”(ASCII smuggling)。

It gained attention two years ago as a means of making a class of AI attack known as prompt injections more stealthy. Malicious instructions embedded in emails or other untrusted content to be processed by an LLM aren’t written in ordinary text. Instead, they’re rendered by a special range of Unicode tags. For example, the tag point U+E0041 mirrors “A,” and U+E0061 mirrors “a.” 该技术在两年前引起关注,当时它被用作一种使“提示词注入”(prompt injections)类 AI 攻击更具隐蔽性的手段。嵌入在电子邮件或其他不可信内容中并由大语言模型(LLM)处理的恶意指令,并非以普通文本形式书写,而是通过一系列特殊的 Unicode 标签进行渲染。例如,标签 U+E0041 对应“A”,而 U+E0061 对应“a”。

No longer just for obscuring prompt injections

不再仅仅用于掩盖提示词注入

The block of 128 tags mimics a portion of the American Standard Code for Information Interchange almost perfectly, with one major difference: the characters they encode are readable by computers but, by design, are almost completely invisible to humans. By expressing the malicious prompts in these tags, LLMs detect the instructions, but people reading the email never see them. 这 128 个标签组成的区块几乎完美地模拟了美国信息交换标准代码(ASCII)的一部分,但有一个主要区别:它们编码的字符虽然计算机可读,但按照设计,对人类而言几乎完全不可见。通过使用这些标签来表达恶意提示词,LLM 可以检测到指令,但阅读邮件的人却完全看不到它们。

Earlier this year, Microsoft started seeing a massive increase in spam messages that used the technique. Beginning on one day in early February, the number of ASCII smuggling signatures detected by Microsoft Defender for Office spiked from roughly 21,000 per day to more than 1.3 million. Within four days, signature detections jumped to 2.5 million. The deluge persisted for months and then fell off sharply in mid-May. 今年早些时候,微软开始发现使用该技术的垃圾邮件数量激增。从 2 月初的某一天开始,Microsoft Defender for Office 检测到的 ASCII 走私特征码数量从每天约 2.1 万条飙升至超过 130 万条。四天内,特征码检测量跃升至 250 万条。这种激增持续了数月,随后在 5 月中旬急剧下降。

“Because tag characters are invisible to humans but exist at the text-processing level, the same property that makes them useful for smuggling instructions into a model also makes them useful for obfuscating keywords before a detector evaluates them,” Microsoft explained Thursday. “The intent is inverted, but the mechanism is similar, and a user’s suspicions are not raised.” “由于标签字符对人类不可见,但存在于文本处理层面,这种使其能够向模型‘走私’指令的特性,同样也使其能够用于在检测器评估之前混淆关键词,”微软周四解释道。“目的虽然相反,但机制是相似的,且不会引起用户的怀疑。”

Spammers are embedding Unicode in an attempt to evade filters that search for text, such as dollar amounts and the words “credit” and “term” that are commonly found in their mass emails. By sprinkling the invisible text into the middle of the word “funding,” for example, filters may read the words “fun” and “ding” instead. The receiver, meanwhile, sees the word “funding.” 垃圾邮件发送者嵌入 Unicode 是为了规避那些搜索特定文本的过滤器,例如其群发邮件中常见的金额数字以及“credit”(信用)和“term”(条款)等词汇。例如,通过在“funding”(资金)一词中间插入不可见文本,过滤器可能会将其识别为“fun”和“ding”。与此同时,收件人看到的依然是“funding”。

Using special text to camouflage certain trigger words isn’t new. Spammers have used zero-width spaces and non-breaking spaces for decades to achieve similar results. The characters can thwart searches matching a literal string and alter the byte sequence that regex filters hunt for. The spammers likely adopted the hidden Unicode tags because some spam filters had yet to be programmed to detect them. 使用特殊文本来伪装某些触发词并非新鲜事。几十年来,垃圾邮件发送者一直使用零宽空格和不换行空格来达到类似的效果。这些字符可以阻碍对字面字符串的匹配搜索,并改变正则表达式过滤器所寻找的字节序列。垃圾邮件发送者采用隐藏的 Unicode 标签,很可能是因为一些垃圾邮件过滤器尚未被编程以检测它们。

A bigger likely reason for its use is to counteract the advantages made possible by machine learning (ML) and natural language processing (NL) LLMs for use in spam detection. Microsoft explained: 其被使用的另一个更重要的原因,很可能是为了对抗机器学习(ML)和自然语言处理(NLP)大模型在垃圾邮件检测中带来的优势。微软解释道:

“The bigger prize for the attacker, though, is not preventing the literal string matches; it is the ML- and NLP-based models that increasingly drive modern spam and phishing classification. Unless a filtering system takes a picture of a message and does OCR extraction over the visual image, it may miss this type of attack. A standard email classifier may not reason over whole words exactly as a human sees them; for efficiency, they can first split text into tokens or sub-word pieces. A clean lure term such as funding may be represented as a familiar token or a familiar sequence of sub-tokens. Insert an invisible U+E0020 into the middle, however, and the tokenizer may no longer see that same familiar unit. It might split the text into fun, an unexpected tag character, and ding; it might emit rare or unknown sub-tokens; or, if normalization runs first, it simply removes the U+E0020 character, leaving funding.” “然而,攻击者更大的目标并非阻止字面字符串匹配,而是针对那些日益驱动现代垃圾邮件和网络钓鱼分类的机器学习及 NLP 模型。除非过滤系统对邮件进行截图并对视觉图像进行 OCR 提取,否则它可能会漏掉此类攻击。标准的电子邮件分类器可能不会像人类那样完整地理解单词;为了效率,它们通常先将文本拆分为标记(token)或子词片段。一个清晰的诱饵词(如 funding)可能被表示为一个熟悉的标记或熟悉的子标记序列。然而,如果在中间插入一个不可见的 U+E0020,分词器可能就无法再识别出那个熟悉的单元。它可能会将文本拆分为‘fun’、一个意外的标签字符和‘ding’;它可能会输出罕见或未知的子标记;或者,如果先进行标准化处理,它可能会直接移除 U+E0020 字符,从而还原出‘funding’。”

Thursday’s post provided guidance on ways developers can program filters to better account for ASCII smuggling in spam. 周四发布的文章为开发人员提供了指导,说明如何编写过滤器以更好地应对垃圾邮件中的 ASCII 走私行为。