OSRS Wiki and RuneLite are increasingly under strain from low-effort AI development
OSRS Wiki and RuneLite are increasingly under strain from low-effort AI development
OSRS Wiki 和 RuneLite 正日益受到低质量 AI 开发的冲击
I spend an unusual amount of time making weird graphs about RuneScape, and this is the single craziest graph I’ve ever seen. The little bump in 2020 is Quest Helper, easily the most important and ambitious RuneLite plugin ever. The amount of Java code being submitted to RuneLite’s Plugin Hub has gone up about 100x since the beginning of the year (there are about 20x as many submissions, and the average submission is about 5x bigger). OSRS Wiki is also encountering graphs in day-to-day operations that have the same “holy shit” shape. If you use RuneLite and the wiki every day and haven’t noticed any of this, that’s mostly because a bunch of people on both projects have been working very hard to keep it from becoming a visible problem.
我花了很多时间制作关于《Runescape》的奇怪图表，而这是我见过最疯狂的一张。2020 年那个小小的凸起是“任务助手”（Quest Helper），它无疑是 RuneLite 历史上最重要、最宏大的插件。自今年年初以来，提交到 RuneLite 插件中心（Plugin Hub）的 Java 代码量增长了约 100 倍（提交数量增加了约 20 倍，且平均每个提交的规模扩大了约 5 倍）。OSRS Wiki 在日常运营中也遇到了同样令人“目瞪口呆”的增长曲线。如果你每天都在使用 RuneLite 和 Wiki 却没察觉到这些，那主要是因为两个项目的许多成员一直在努力工作，才没让这些问题显现出来。
3 months worth of code, every day
每天产生三个月的代码量
A 10,000-line plugin used to be an enormous project that was a months-long back and forth between the developer and the maintainers. Now somebody can spend 15 minutes asking ChatGPT to build them something, submit roughly as much code as the entire Plugin Hub used to process in a month, and hand it to a volunteer reviewer. Has RuneLite gotten 100x better since the year started? No. The median new plugin used to end up with a couple thousand installs. It’s now 28. All of that code still goes through the Plugin Hub review process, including the stuff that ends up with almost no users. Many of these AI-heavy plugins contain impressive hallucinations — one with 2,500 active installs just made up a bunch of Sailing ships that don’t exist, and another popular one for a recent quest basically fed the wiki’s quest guide into Claude and turned it into a step-by-step “helper” that was complete nonsense on the boss fight.
过去，一个 1 万行的插件是一个巨大的工程，需要开发者和维护者之间长达数月的反复沟通。现在，某人只需花 15 分钟让 ChatGPT 写点东西，提交的代码量就相当于插件中心过去一个月的处理总量，然后直接甩给志愿者审核员。RuneLite 自年初以来变好 100 倍了吗？并没有。过去，新插件的中位数安装量通常有几千次，现在只有 28 次。所有这些代码仍需经过插件中心的审核流程，包括那些几乎没人使用的插件。许多这类 AI 生成的插件包含令人震惊的“幻觉”——其中一个拥有 2500 次活跃安装的插件凭空捏造了一堆不存在的航海船只；另一个针对近期任务的热门插件，本质上是将 Wiki 的任务指南喂给 Claude，结果生成了一个在 Boss 战中完全胡言乱语的“助手”。
The obvious question to a layperson is, why not “just” use AI to review the code? The reviewers already do, partly. It helps a bit. But it’s not a replacement for a human with years of experience who knows what to look for, especially when some of the code is security-sensitive. Or maybe why not just immediately reject code that is AI-generated? This is perhaps not obvious to the average person, but it’s an extremely hard problem to, in every case, automatically distinguish AI code from handwritten code. If we had a classifier for this, then these AI models would just start optimizing to beat that classifier. That’s not a winnable battle.
对于外行来说，显而易见的问题是：为什么不“直接”用 AI 来审核代码呢？审核员们已经在部分使用了。这确实有帮助，但它无法取代拥有多年经验、知道重点在哪里的专业人员，尤其是当某些代码涉及安全敏感信息时。或者，为什么不直接拒绝所有 AI 生成的代码？这对普通人来说可能不明显，但在所有情况下自动区分 AI 代码和人工编写的代码是一个极难的问题。如果我们有一个分类器，这些 AI 模型就会开始针对该分类器进行优化。这是一场赢不了的战争。
The wiki is now everybody else’s backend
Wiki 现在成了所有人的后端
I’ve already written about AI scrapers making it kinda suck to run wikis, and we still spend a lot of time swatting those away. Increasingly, though, it’s software written with AI that decides some part of the wiki ecosystem is its backend. One recent very popular AI-developed RuneLite plugin hotlinked its images from the wiki, then fetched them again on every single client frame — up to about 50 times a second, per image — and retried forever when it got a non-200 response. This wasn’t something anyone meant to happen, nor was it a DDOS, nor did it impact the wiki’s uptime; once we figured out the images were infinite-loop hitting the wiki, the developer fixed it. By then it had used about 700,000 GB of wiki bandwidth (a normal full day is about 4,000 GB). It also used up 500 GB of some poor guy’s home internet connection.
我之前写过 AI 爬虫让运营 Wiki 变得很糟糕，我们至今仍花费大量时间去拦截它们。然而，越来越多由 AI 编写的软件将 Wiki 生态系统的某些部分当作了自己的后端。最近，一个非常流行的 AI 开发的 RuneLite 插件直接热链接了 Wiki 的图片，然后在客户端的每一帧都重新获取一次——每张图片每秒请求约 50 次——并且在收到非 200 响应时会无限重试。这并非有人蓄意为之，也不是 DDoS 攻击，也没有影响 Wiki 的正常运行；一旦我们发现这些图片在无限循环地请求 Wiki，开发者就修复了它。但到那时，它已经消耗了约 70 万 GB 的 Wiki 带宽（正常情况下全天约为 4000 GB）。它还耗尽了某个倒霉蛋 500 GB 的家庭宽带流量。
And by “the wiki” here, I mean a lot more than articles on runescape.wiki. The same systems and people serve images, realtime prices, WikiSync, drop data, and a bunch of APIs and public data that nearly all RuneScape tools depend on. We’re reaching a point where there’s 2 or 3 wiki infrastructure incidents per week due to badly written plugins and sites. We’ve had vibe-coded sites that choose a bizarrely expensive way to grab images from the wiki, and we often don’t even have a way to contact anyone, because no human was involved in that decision. Plugins expensively query the wiki’s drop-rate API every time a monster enters someone’s render range. We keep running into generated tools pulling prices, images, WikiSync data and other stuff from the same public systems in ways that are borderline unreasonable for one user and genuinely problematic for 10,000 users.
这里所说的“Wiki”不仅仅指 runescape.wiki 上的文章。同样的系统和人员还负责提供图片、实时价格、WikiSync、掉落数据，以及几乎所有《Runescape》工具所依赖的一系列 API 和公共数据。我们现在每周都会因为编写糟糕的插件和网站而发生 2 到 3 起 Wiki 基础设施事故。我们遇到过一些“凭感觉写出来”的网站，它们以极其昂贵的方式从 Wiki 获取图片，而我们甚至无法联系到任何人，因为在做出这些决策的过程中根本没有人类参与。插件在怪物进入玩家渲染范围的每一次，都会高频地查询 Wiki 的掉落率 API。我们不断发现生成的工具以一种对单个用户来说近乎不合理、对一万个用户来说则真正成问题的方式，从这些公共系统中拉取价格、图片和 WikiSync 数据。
Most readers never notice any of this. Logged-out traffic is heavily cached, and we have an insanely cracked technical team that works their asses off to keep 400 million pageviews a month boring. When something does spike, somebody on the team usually has to trace it back to the caller, find the developer, and figure out how to make it stop — which is a lot harder when the person who made the thing has never looked at the code.
大多数读者从未察觉到这些。未登录流量被大量缓存，我们拥有一支极其强悍的技术团队，他们拼尽全力让每月 4 亿次的页面浏览量保持平稳。当流量出现峰值时，团队成员通常必须追溯到调用源，找到开发者，并设法让其停止——而当制造这些东西的人从未看过代码时，这会变得困难得多。
It used to be harder to make something you don’t understand
过去，制造你不理解的东西要难得多
There have always been bad programmers and beginners. But until pretty recently, if you had enough technical skill to build a RuneLite plugin or a website capable of accidentally sending a billion requests somewhere, you had usually picked up enough adjacent knowledge to know that sending a billion requests somewhere might be a problem for whoever’s on the other side. Not everyone picked that up, obviously, but often enough that it acted as a useful filter. AI development has vaporized that filter, so now we frequently have to spend an afternoon understanding code that no human besides us(!) has ever looked at. Every hour spent doing that is an hour not spent on something else. That’s probably the part of this that bothers me the most — there are a lot of cool things Adam and I would be building if we weren’t spending so much of our time dealing with this stuff.
糟糕的程序员和初学者一直存在。但在不久前，如果你有足够的技术能力去构建一个 RuneLite 插件或一个能意外向某处发送十亿次请求的网站，你通常已经掌握了足够的周边知识，知道向某处发送十亿次请求可能会给对方带来麻烦。显然，并非每个人都能意识到这一点，但这种情况发生的频率足以作为一个有效的过滤器。AI 开发蒸发了这个过滤器，所以现在我们经常要花一下午的时间去理解那些除了我们（！）之外从未有人看过的代码。每花一小时做这些事，就意味着少了一小时去做别的事。这可能是我最困扰的地方——如果我们不用花这么多时间处理这些烂摊子，Adam 和我本可以构建许多很酷的东西。
There’s no simple answer
没有简单的答案
This is a stupidly complicated situation, and I think anyone who has a one-sentence solution is badly misunderstanding the ecosystem.
这是一个极其复杂的情况，我认为任何试图用一句话给出解决方案的人，都严重误解了这个生态系统。