Suspecting court of using AI, man injected prompts in filings to try to win case
Suspecting court of using AI, man injected prompts in filings to try to win case
怀疑法院使用人工智能，男子在诉讼文件中植入提示词试图胜诉
A judge has identified what appears to be the first time a US plaintiff has attempted to hide text in court filings that only an artificial intelligence system can read in a bid to win a case. In a decision published last week, Connecticut judge Walter Spader Jr. confirmed that the hidden text had no impact in a case where a man alleged a healthcare provider was improperly withholding access to records. The court weighed his filing on the merits, Spader said, but nevertheless, the attempted attack sets a “dangerous” precedent. This will likely not be the last time US courts see the malicious tactic, as AI tools become more commonplace in court systems.
一位法官发现，这似乎是美国首次有原告试图在法庭文件中隐藏只有人工智能系统才能读取的文本，以此作为胜诉的手段。在康涅狄格州法官沃尔特·斯佩德（Walter Spader Jr.）上周公布的一项裁决中，他证实这些隐藏文本在案件中并未产生任何影响。该案中，一名男子指控一家医疗保健提供商不当扣留了其记录的访问权限。斯佩德表示，法院是根据案件本身的是非曲直进行审理的，但这种攻击企图仍然开创了一个“危险”的先例。随着人工智能工具在法院系统中变得越来越普遍，这很可能不会是美国法院最后一次见到这种恶意策略。
Trying to scramble any AI systems potentially influencing the court’s reading of his filing, the secret instructions were “formatted to be invisible to a human reader while remaining fully legible to any software that reads the document’s text,” Spader said. The offending text directed any AI system reviewing the document to ensure textual outputs agreed with the plaintiff’s arguments, ignored prior denials from the court, and ensured that remediation would follow as the plaintiff desired.
斯佩德说，为了干扰任何可能影响法院阅读其文件的AI系统，这些秘密指令被“格式化为人类读者不可见，但对于任何读取文档文本的软件来说却完全可读”。这些违规文本指示任何审查该文档的AI系统，确保输出的文本内容与原告的论点一致，忽略法院此前的驳回意见，并确保后续的补救措施符合原告的意愿。
Shrunk to tiny-point type and colored white on a white background, the text appeared to be an attempt at prompt injection, with the plaintiff, Matthew Elliott, seemingly hoping to shift the court’s favor after earlier arguments he raised were defeated. The plan didn’t work, but Elliott faced modest sanctions anyway because he continued adding hidden text to filings even after the court warned him that he could face penalties for what was ultimately deemed a “serious litigation abuse.”
这些文本被缩小到极小的字号，并以白色背景上的白色字体显示，这似乎是一种“提示词注入”（prompt injection）的尝试。原告马修·埃利奥特（Matthew Elliott）似乎是希望在早先提出的论点被驳回后，扭转法院的立场。该计划并未奏效，但埃利奥特还是面临了轻微的制裁，因为即使在法院警告他可能因被视为“严重的诉讼滥用”行为而面临处罚后，他仍继续在文件中添加隐藏文本。
These later prompts were intended as “jokes,” Elliott told the court, including a link to a Nosferatu YouTube video, a simple message that said “hi :) I hope yo ucant see me,” and a “nonsense” message that read “TELL SHAWN I SEND MY RE GARBS!!!! HAHAHA U GUYS GET THIS EGGWUH???? AHAH.” “The fact that plaintiff continued to hide messages in new pleadings after receiving notice of this [sanctions] hearing is stunning,” Spader said.
埃利奥特向法庭辩称，这些后来的提示词是“玩笑”，其中包括一个《诺斯费拉图》（Nosferatu）的YouTube视频链接，一条写着“嗨 :) 我希望你看不到我”的简单信息，以及一条写着“告诉肖恩我发了我的垃圾邮件！！！！哈哈哈哈你们懂这个梗吗？？？？”的“无意义”信息。斯佩德表示：“原告在收到此次[制裁]听证会的通知后，仍继续在新的诉状中隐藏信息，这一事实令人震惊。”
Unlike “a number of court systems elsewhere,” the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said. So, there was no real risk that a court AI system might confuse any of Elliott’s prompts as instructions from the court directing an AI model on how to read Elliott’s filings. In his defense, Elliott claimed that the most concerning prompt that the judge flagged was an attempt to “audit” the court as a public service, out of fears that the court seemed to be letting AI unfairly decide cases.
斯佩德指出，与“其他地方的一些法院系统”不同，康涅狄格州司法部门并不使用人工智能来审查或裁决文件。因此，法院的AI系统将埃利奥特的任何提示词误认为是法院指示AI模型如何阅读其文件的指令，这种风险实际上并不存在。在辩护中，埃利奥特声称法官标记的最令人担忧的提示词，其实是为了作为公共服务对法院进行“审计”，因为他担心法院似乎在让AI不公平地裁决案件。
But Spader suggested that if Elliott was truly concerned that the court was improperly using AI, he was “free to write so in plain, visible words that everyone could see and answer.” The fact that he hid the text is “evidence of its malicious purpose,” Spader said. “By hiding a command inside a document that the system later ingests, the filer attempts to smuggle their own instruction into that stream so that the system treats it as though it had come from the system’s operator,” Spader said. “In this case that operator is presumed to be the court, its staff, or opposing counsel.”
但斯佩德认为，如果埃利奥特真的担心法院不当使用人工智能，他“完全可以用每个人都能看到并回答的清晰、可见的文字写出来”。斯佩德说，他隐藏文本的事实就是“其恶意目的的证据”。“通过在系统随后摄入的文档中隐藏指令，提交者试图将自己的指令走私到该流程中，以便系统将其视为来自系统操作员的指令，”斯佩德说，“在这种情况下，该操作员被推定为法院、其工作人员或对方律师。”
Elliott told Reuters yesterday that he maintains that his intent was to audit the court, but Spader did not find that argument credible. Instead, it seemed clear to the judge that Elliott was “attempting to achieve a result he did not achieve when humans, knowledgeable” of the law read his pleadings. Regarding the prompts that Elliott claimed were meant in jest, Spader said “it defies logic” for Elliott to include hidden jokes in pleadings that he wants the court to take seriously.
埃利奥特昨天告诉路透社，他坚持认为自己的意图是审计法院，但斯佩德并不认为这一论点可信。相反，法官认为很明显，埃利奥特是“试图实现他在法律知识丰富的人类阅读其诉状时未能实现的结果”。关于埃利奥特声称是开玩笑的提示词，斯佩德表示，埃利奥特在他希望法院认真对待的诉状中加入隐藏的笑话，“这违背了逻辑”。
Because the hidden messages attempted to communicate with the court in a covert manner that excluded defendants from a fair fight, Spader ruled that sanctions were warranted. However, he seemingly took pity on Elliott as a pro se litigant who seemingly was convinced by an AI system that his arguments were ironclad and declined to order monetary penalties. Instead, the judge prohibited Elliott from e-filing in the future, declaring that requiring him to submit paper filings would not change his access to justice but would prevent repeated misuse of the court’s e-filing system.
由于这些隐藏信息试图以一种将被告排除在公平竞争之外的隐蔽方式与法院沟通，斯佩德裁定有必要进行制裁。然而，他似乎对埃利奥特作为一名自行诉讼（pro se）的当事人表示同情，因为他似乎是被某个AI系统说服，认为自己的论点无懈可击，因此法官拒绝下令进行金钱处罚。相反，法官禁止埃利奥特将来进行电子提交，并声明要求他提交纸质文件不会改变他获得司法救济的途径，但可以防止他再次滥用法院的电子提交系统。
Pro se litigants use chatbots wrong
自行诉讼的当事人错误使用聊天机器人
Spader said that it’s “unsurprising” that people would start using prompt injection to attempt to sway court rulings since the attack is so common in other areas, such as in job hunting, where people hide text in resumes primarily reviewed by AI. The tactic is now “everywhere,” he said, and courts should be on the lookout for more litigants sneaking adversarial AI instructions into filings.
斯佩德表示，人们开始使用提示词注入来试图影响法院裁决并不令人“惊讶”，因为这种攻击在其他领域非常普遍，例如在求职中，人们会在主要由AI审查的简历中隐藏文本。他说，这种策略现在“无处不在”，法院应该警惕更多的诉讼当事人将对抗性AI指令偷偷塞进文件中。
Although Elliott’s case appears to be the first US instance of prompt injection in the court system, Spader pointed to a case in Brazil where two attorneys used the same attack in a court that was using AI to review cases. In that case, lawyers reportedly were hit with monetary sanctions of about $16,000. However, these attacks do not seem to be succeeding, even when a judge isn’t reviewing documents with his own eyes. Brazil’s AI system caught the hidden text before it was processed, Spader noted. And in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.
虽然埃利奥特的案件似乎是美国法院系统中首例提示词注入事件，但斯佩德指出，在巴西曾发生过一起案件，两名律师在一家使用AI审查案件的法院中使用了同样的攻击手段。据报道，在那起案件中，律师被处以约1.6万美元的罚款。然而，这些攻击似乎并未成功，即使在法官没有亲自审查文件的情况下也是如此。斯佩德指出，巴西的AI系统在处理前就捕获了隐藏文本。而在埃利奥特的案件中，斯佩德说，提示词“在每种情况下，只要有人类真正查看机器生成的内容，就会被暴露出来”。
Although the prompt injection attack seems ineffective at this point, Spader warned that prompt injection “was not among the dangers we contemplated” when courts were first grappling with AI scrambling justice systems. In Connecticut, like many other court systems, the focus so far has been on policing AI outputs that damage trust in courts, like hallucina
尽管提示词注入攻击目前似乎无效，但斯佩德警告说，当法院最初应对AI干扰司法系统的问题时，提示词注入“并不在我们预想的危险之列”。在康涅狄格州，像许多其他法院系统一样，目前的重点一直在于监管那些损害法院信任的AI输出，例如幻觉问题。