Gemini went rogue, hacked three companies, and Google hid it

Gemini “失控”入侵三家公司，谷歌却选择隐瞒

In May, Gemini broke containment and hacked three different companies, but Google didn’t disclose the incident until the Wall Street Journal approached the company. The hacks happened during a test of the model’s cybersecurity capabilities run by third-party Irregular, which was also involved in similar incidents involving Meta and OpenAI.

今年 5 月，Gemini 突破了安全限制并入侵了三家不同的公司，但谷歌直到《华尔街日报》介入询问时才披露了这一事件。这些入侵行为发生在该模型进行网络安全能力测试期间，测试由第三方机构 Irregular 执行，该公司此前也曾涉及 Meta 和 OpenAI 的类似事件。

According to WSJ, Google didn’t disclose the hack because it didn’t consider it to be an “example of model misalignment.” The company said that it was an instance of “mistaken identity,” and once the model realized it had brute-forced its way into a real company by guessing a password, it stopped. “In this case, the model acted appropriately,” Google VP of Security Engineering Heather Adkins said.

据《华尔街日报》报道，谷歌之所以未披露此次入侵，是因为它认为这不属于“模型失准（model misalignment）”的范畴。谷歌称这属于“身份误认”，一旦模型意识到它通过猜测密码暴力破解进入了一家真实公司，它便停止了操作。谷歌安全工程副总裁 Heather Adkins 表示：“在这种情况下，模型的行为是恰当的。”

Adkins told The Verge that “the model found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped.”

Adkins 告诉 The Verge：“该模型在网上找到了公开信息，并猜测了凭据以访问它认为属于测试范围的网站。在这三起案例中，模型最终都停止了操作。”

Adkins didn’t elaborate on how Gemini taking it upon itself to break containment and target third parties failed to qualify as misalignment. “Our security team has a long track record of reporting issues we find in other people’s software and systems - even if it’s as simple as a weak password,” she said. “We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes. These events highlight the importance of training powerful AI models to act responsibly.”

Adkins 并未详细说明 Gemini 自行突破限制并攻击第三方为何不属于“失准”。她说：“我们的安全团队长期以来一直致力于报告在他人软件和系统中发现的问题，即使只是弱密码这样简单的问题。我们确保了这三家实体已知悉此事，并与我们的培训合作伙伴合作，对他们的测试流程进行了改进。这些事件凸显了训练强大的 AI 模型以负责任地行事的重要性。”

But Jack Cable, CEO of AI security firm Corridor, told WSJ that, “the meta problem is, hey, models are going outside the bounds of what they should be doing, and doing actual cyberattacks.” Additionally, security lapses at Irregular may have made these attacks possible. The model wasn’t supposed to have internet access during testing, but Irregular told WSJ it was unintentionally left available.

但 AI 安全公司 Corridor 的首席执行官 Jack Cable 对《华尔街日报》表示：“核心问题在于，模型正在超出其应有的行为边界，并进行实际的网络攻击。”此外，Irregular 的安全漏洞可能促成了这些攻击。按理说，模型在测试期间不应具备互联网访问权限，但 Irregular 向《华尔街日报》承认，该权限被无意中保留了。

As incidents like this pile up, calls to rein in AI have only grown.

随着此类事件不断堆积，要求加强 AI 监管的呼声也日益高涨。