AI Is Getting Really Good at Messing With Cybercriminals
AI Is Getting Really Good at Messing With Cybercriminals
Save this storySave this storyWelcome to Kernel Panic!, a weekly newsletter by Lily Hay Newman and Matt Burgess from inside the new world of privacy and digital security. To receive this newsletter in your inbox each week, sign up here.
欢迎阅读《Kernel Panic!》，这是由 Lily Hay Newman 和 Matt Burgess 撰写的每周通讯，带您深入了解隐私和数字安全的新世界。如需每周在收件箱中接收此通讯，请点击此处订阅。
Even as AI supercharges digital scamming and cybercrime, it’s also being recruited in the fight to protect potential victims. Governments around the world have consistently struggled to address the global crisis of online crime, grappling with the limitations of attempting to crack down on criminals beyond their borders. This has inspired alternative efforts, like using automation to attempt to go after scammers en masse—perhaps by spamming the spammers to squander their resources. Now, AI platforms are turning these types of experiments into real tools in the fight against cybercrime.
尽管人工智能助长了数字诈骗和网络犯罪，但它也被招募来保护潜在的受害者。世界各国的政府在应对全球网络犯罪危机时一直举步维艰，难以克服打击跨境犯罪的局限性。这激发了一些替代性的尝试，例如利用自动化手段大规模针对诈骗者——或许是通过向诈骗者发送垃圾信息来消耗他们的资源。现在，人工智能平台正在将这些实验转化为打击网络犯罪的实际工具。
For the last two years, the Australian company Apate, named after the Greek goddess of deception, has been building a system that diverts phone scammers onto calls with AI bots that are trained to keep conversations going as long as possible without, of course, ever actually falling for the scam. The goal is to give scammers enough hope that they stay on the phone, while also collecting intelligence about the scams they’re running.
在过去两年里，澳大利亚公司 Apate（以希腊欺骗女神命名）一直在构建一个系统，将电话诈骗者转移到与人工智能机器人的通话中。这些机器人经过训练，能够尽可能长时间地维持对话，当然，它们绝不会真正上当受骗。其目标是给诈骗者足够的希望，让他们留在电话线上，同时收集有关他们所实施诈骗的情报。
“What we really like to think is that we’re building the perfect victims for scammers,” Dali Kaafar, the founder and CEO of Apate, tells us. “A minute that a scammer is talking to a bot or an agent is a minute where you’re probably saving hundreds, if not thousands of possible people being reached out to by that exact same scammer,” he says, referencing the ability for scammers to use automated dialing tools.
Apate 的创始人兼首席执行官 Dali Kaafar 告诉我们：“我们真正想的是，我们正在为诈骗者打造完美的受害者。”他提到诈骗者使用自动拨号工具的能力时说：“诈骗者与机器人或代理人交谈的每一分钟，都可能意味着你拯救了数百甚至数千名可能被同一个诈骗者联系的潜在受害者。”
Kaafar says Apate’s platform, which is used by banks and supported by telecom companies, has around 350,000 bots. They not only pick up calls, but also infiltrate scam chat groups online and respond to text messages with similar goals of frustrating scammers while collecting intelligence from them. He says the company has collected more than 250,000 pieces of information about fraudsters in real-time, from scam URLs to money mule accounts and bank details.
Kaafar 表示，Apate 的平台已被银行使用并得到电信公司的支持，拥有约 35 万个机器人。它们不仅接听电话，还渗透到在线诈骗聊天群中，并回复短信，其目的同样是挫败诈骗者，同时从他们那里收集情报。他说，该公司已经实时收集了超过 25 万条关于诈骗者的信息，从诈骗网址到洗钱账户和银行详细信息，应有尽有。
The bots have different personalities, language skills, and profiles, Kaafar says, so scammers hopefully don’t detect that they’re conversing with an AI. Like real people, “sometimes [the bots] do have WhatsApp, sometimes they don’t. Sometimes they pick up the phone, sometimes they just actually hang up on the scammer saying, ‘I’ll come back to you later,’” he says.
Kaafar 说，这些机器人拥有不同的个性、语言技能和个人资料，因此希望诈骗者不会察觉到他们是在与人工智能交谈。他说，就像真人一样，“有时（机器人）有 WhatsApp，有时没有。有时它们会接电话，有时它们真的会挂断诈骗者的电话，说‘我稍后再联系你’。”
But are they convincing? Kernel Panic tested the tool using a demo version where users play the role of the scammer and talk to one of Apate’s AI “victims.” The personas are designed to express a healthy amount of skepticism but leave enough openings for scammers to continue to try to win them over, a dynamic that we could immediately feel and found intensely frustrating.
但它们有说服力吗？《Kernel Panic》使用演示版本测试了该工具，用户在其中扮演诈骗者的角色，与 Apate 的人工智能“受害者”之一进行交谈。这些角色被设计为表现出适度的怀疑，但又留下了足够的空间让诈骗者继续试图说服他们，这种互动动态我们能立刻感受到，并觉得非常令人沮丧。
Scammers are likely familiar with this type of skeptical interaction, and Kaafar says that Apate calls regularly go on for more than two hours. In our testing, the Apate system was engaged and responsive, and the timing of the conversation felt natural, even with both of us tag-teaming the AI as a friend, “Lucy,” and her financial adviser “Mickey.” We did not succeed at getting the Apate AI system to invest in our cryptocurrency opportunity even after six minutes of diligent effort. (The returns are truly incredible, though, we swear.)
诈骗者可能很熟悉这种怀疑式的互动，Kaafar 说 Apate 的通话通常会持续两个多小时。在我们的测试中，Apate 系统参与度高且反应灵敏，对话的时机感觉很自然，即使我们两人轮番上阵，分别扮演朋友“Lucy”和她的财务顾问“Mickey”。尽管我们努力了六分钟，但还是没能让 Apate 的人工智能系统投资我们的加密货币项目。（不过我们发誓，回报率确实令人难以置信。）
For now, even with Apate’s massive swarm of bots on the case, it’s only a matter of time before you get your next scam call or text. Cybercriminals initiate billions of messages and calls each year, with the most sophisticated operations running physical, industrial scale scamming sites. And while professional scambaiters and initiatives to infiltrate scam operations have done critical deterrent work for years, law enforcement and research efforts haven’t been able to stop the expansion of digital scamming overall.
目前，即使有 Apate 的海量机器人参与其中，你收到下一次诈骗电话或短信也只是时间问题。网络犯罪分子每年发起数十亿条信息和电话，其中最复杂的行动甚至运营着物理性的工业规模诈骗站点。虽然专业的“诱骗者”（scambaiters）和渗透诈骗行动的举措多年来发挥了关键的威慑作用，但执法部门和研究工作仍无法阻止数字诈骗的整体扩张。
But AI could create a new era for anti-scam efforts, not just scammers themselves. Better intelligence sharing between cops, social media companies, banks, and other industries is desperately needed and could potentially be aided by AI-driven monitoring and data analysis. Increasingly, too, innovative efforts have been looking to disrupt cybercriminals by exploiting their “psychological vulnerabilities.” In some cases, these psyops have included directly trolling those behind ransomware attacks. As the focus on disrupting scammers and cybercriminals expands, the use of generative AI as a way to keep them preoccupied, wasting resources, and not launching other attacks appears to be increasingly effective.
但人工智能可能会为反诈骗工作开创一个新时代，而不仅仅是为诈骗者本身。警察、社交媒体公司、银行和其他行业之间迫切需要更好的情报共享，而人工智能驱动的监控和数据分析可能会对此有所帮助。此外，越来越多的创新努力正试图通过利用网络犯罪分子的“心理弱点”来干扰他们。在某些情况下，这些心理战包括直接戏弄勒索软件攻击的幕后黑手。随着干扰诈骗者和网络犯罪分子的重点不断扩大，利用生成式人工智能来让他们忙于应付、浪费资源，从而无法发动其他攻击，似乎正变得越来越有效。
Take honeypots, for instance. For years, companies and security researchers have deployed false virtual machines to attract hackers, waste their time, gather information, and understand their secret hacking techniques. Mark Vero, a doctoral researcher at the department of computer science at ETH Zurich, says open source honeypot providers have increasingly been incorporating LLMs into their systems to make them appear more realistic.
以蜜罐为例。多年来，公司和安全研究人员一直部署虚假的虚拟机来吸引黑客，浪费他们的时间，收集信息并了解他们的秘密黑客技术。苏黎世联邦理工学院计算机科学系的博士研究员 Mark Vero 表示，开源蜜罐提供商越来越多地将大语言模型（LLM）整合到他们的系统中，以使它们看起来更真实。
In recent research, Vero and colleagues found an LLM-powered honey pot would keep AI agents attacking their system “significantly longer” than a honey pot with more predictable behavior. “The agentic attackers are much more convinced by the LLM simulated honeypots and they also mark them as actual honeypots at a much lower rate,” he says. “If these systems are built well enough, then I think it’s quite advantageous for defenders.”
在最近的研究中，Vero 和他的同事发现，由大语言模型驱动的蜜罐能让攻击其系统的人工智能代理停留的时间“显著更长”，相比之下，行为更可预测的蜜罐则不然。他说：“代理攻击者更容易被大语言模型模拟的蜜罐所迷惑，他们将这些蜜罐标记为真实蜜罐的概率也更低。如果这些系统构建得足够好，那么我认为这对防御者来说是非常有利的。”
Spotted something we should include next week? Let us know at [email protected]. And stay safe out there.
发现了我们下周应该收录的内容吗？请通过 [email protected] 告诉我们。在外请注意安全。