Discord admits AI moderation bug wrongfully banned users over harmless images

Discord 承认 AI 审核漏洞导致用户因无害图片被错误封禁

Discord has acknowledged that a bug in its AI moderation system mistakenly banned more than 8,000 users over the past two months, after harmless images—including spreadsheets, chessboards, game textures, as well as white and gray transparent backgrounds—were incorrectly flagged as harmful content. The company confirmed that the issue had been affecting accounts since May, with an additional 200 users banned over the weekend before its team identified and fixed the problem. All affected accounts are currently in the process of being restored.

Discord 承认，其 AI 审核系统中的一个漏洞在过去两个月内错误地封禁了超过 8,000 名用户。这些用户上传的无害图片（包括电子表格、棋盘、游戏纹理以及白灰相间的透明背景）被错误地标记为有害内容。该公司证实，该问题自 5 月起就开始影响用户账户，在团队发现并修复问题之前，上周末又有 200 名用户被封禁。目前，所有受影响的账户正在恢复过程中。

The incident highlights one of the growing challenges surrounding AI-assisted moderation as many platforms increasingly rely on automated systems to identify illegal or abusive material at scale. In a detailed thread on X, Discord explained that its automated safety system works by matching uploaded content against databases of known harmful material. While the technology is designed to catch illegal content, the company acknowledged that it can sometimes generate false positives. A human moderator reviews the content, but a bug caused the system to immediately ban affected accounts. “We’re working on better safeguards so this can’t happen again,” the company wrote.

此次事件凸显了 AI 辅助审核所面临的日益严峻的挑战，因为许多平台正越来越依赖自动化系统来大规模识别非法或滥用内容。Discord 在 X 平台上发布的一系列详细推文中解释称，其自动化安全系统的工作原理是将上传的内容与已知有害内容的数据库进行比对。虽然该技术旨在拦截非法内容，但公司承认它有时会产生误报。通常情况下，人工审核员会复核内容，但由于系统漏洞，导致受影响账户被直接封禁。公司表示：“我们正在制定更好的保障措施，以防止此类事件再次发生。”

Across X and Reddit, users have claimed they had been permanently suspended simply for uploading images containing square grid patterns. Several users speculated that Discord’s AI moderation tools have become increasingly sensitive to grid-like patterns because they have previously been used in attempts to obscure or disguise NSFW and child exploitation content from automated detection systems.

在 X 和 Reddit 上，有用户声称他们仅仅因为上传了包含方形网格图案的图片就被永久封禁。一些用户推测，Discord 的 AI 审核工具对网格状图案变得越来越敏感，因为这些图案此前曾被用于试图掩盖或伪装色情及儿童剥削内容，以逃避自动化检测系统。

Affected users have been expressing frustration on social media, with some arguing that permanent account bans based solely on automated detection can have serious consequences, particularly for users who rely on Discord for work, gaming communities, or long-distance social connections. “Losing a Discord account to something as unfair as this can be extremely devastating and affect users severely, and every day millions of users are affected by false AI bans. This needs to be stopped,” one X user wrote.

受影响的用户在社交媒体上表达了不满，一些人认为仅凭自动化检测就进行永久封号会带来严重后果，尤其是对于那些依赖 Discord 进行工作、游戏社区交流或维持远程社交关系的用户而言。一位 X 用户写道：“因为这种不公平的原因失去 Discord 账号是毁灭性的，会严重影响用户。每天都有数百万用户受到 AI 误封的影响，这种情况必须停止。”

Discord isn’t alone in facing moderation troubles due to automated systems. Last year, users of Instagram and Facebook Groups reported widespread unexplained account suspensions that many believed were caused by AI moderation systems. Although users pointed to automation as the likely culprit, Meta never publicly confirmed whether AI errors were responsible. Now Meta’s Oversight Board is pushing for increased transparency. Tumblr last year also faced complaints from users who said their accounts had been mass-suspended without clear explanations.

Discord 并非唯一面临自动化系统审核问题的平台。去年，Instagram 和 Facebook 群组的用户报告了大规模且无法解释的账号封禁，许多人认为这是由 AI 审核系统造成的。尽管用户指出自动化系统是罪魁祸首，但 Meta 从未公开确认是否由 AI 错误导致。目前，Meta 的监督委员会正在推动提高透明度。Tumblr 去年也面临用户投诉，称其账号在没有明确解释的情况下被大规模封禁。