AI Made Bugs Cheap to Find

AI Made Bugs Cheap to Find

AI 让发现漏洞变得廉价

The most important AI security story right now is not that models can find bugs. It is that models can find more bugs than humans can responsibly process. That is the part that changes how builders should think about software. 目前最重要的 AI 安全新闻并非“模型可以发现漏洞”,而是“模型发现漏洞的速度超过了人类的处理能力”。这一点彻底改变了开发者看待软件的方式。

For years, security work was constrained by discovery. Could someone find the vulnerability? Could they reproduce it? Could they build an exploit? Could a small team afford enough expert review to catch the important issues before attackers did? Now that bottleneck is moving. 多年来,安全工作的瓶颈一直在于“发现”。是否有人能找到漏洞?能否复现?能否构建攻击载荷?小团队是否有足够的专家资源在攻击者之前发现关键问题?现在,这个瓶颈正在转移。

Anthropic’s recent Project Glasswing update is the clearest signal yet. The company says Claude Mythos Preview and its partners found more than 10,000 high- or critical-severity vulnerabilities across major software systems. In open source alone, Anthropic says it scanned more than 1,000 projects and surfaced thousands of serious findings, with human triage becoming the slow part. Anthropic 最近的 Project Glasswing 更新是最明确的信号。该公司表示,Claude Mythos Preview 及其合作伙伴在各大软件系统中发现了超过 10,000 个高危或严重漏洞。仅在开源领域,Anthropic 就扫描了超过 1,000 个项目,并挖掘出数千个严重问题,而人工分类筛选反而成了效率瓶颈。

You do not have to take every number at face value to see the shape of the shift. AI is making vulnerability discovery cheaper. That sounds like good news, and it is. But it also means every software team is about to face a harder question: What happens when the scanner is faster than the organization? 你无需纠结于每一个数字,就能看出这种转变的趋势。AI 正在降低发现漏洞的成本。这听起来是好消息,事实也确实如此。但这也意味着每个软件团队都将面临一个更严峻的问题:当扫描器的速度超过了组织的处理速度时,会发生什么?

The Patch Window Is the Product Now

补丁窗口期即产品本身

Security used to have a familiar rhythm. A bug was found. A report was filed. A team reproduced it. Someone argued about severity. Someone wrote a patch. Users eventually upgraded. That process was never fast enough, but it mostly matched the speed of human discovery. 安全工作过去有一种熟悉的节奏:发现漏洞,提交报告,团队复现,争论严重程度,编写补丁,用户最终升级。这个过程虽然从未足够快,但基本与人类发现漏洞的速度相匹配。

AI breaks that balance. If models can search codebases, reason about exploit paths, generate reports, and repeat that work across thousands of projects, then finding bugs stops being the scarce skill. The scarce skill becomes the system around the finding: Can you tell which reports are real? Can you prioritize the ones that actually matter? Can you patch without breaking production? Can you ship fixes before attackers learn the same thing? Can you keep maintainers from drowning in low-quality reports? That is the new security stack. Not just detection. Response capacity. AI 打破了这种平衡。如果模型能够搜索代码库、推导攻击路径、生成报告,并在数千个项目中重复这些工作,那么“发现漏洞”就不再是稀缺技能。稀缺的技能变成了围绕漏洞的处理系统:你能分辨哪些报告是真实的吗?你能优先处理真正重要的问题吗?你能修补漏洞而不破坏生产环境吗?你能赶在攻击者发现漏洞前发布修复吗?你能防止维护者被低质量报告淹没吗?这就是新的安全栈——不仅仅是检测,更是响应能力。

A vulnerability that sits untriaged for three weeks is not meaningfully safer because an AI found it. In some cases, it is riskier, because the same class of model may soon make the path to exploitation easier for everyone else. The patch window is not an operational detail anymore. It is part of the product. 一个被搁置三周未处理的漏洞,并不会因为是 AI 发现的就变得更安全。在某些情况下,它反而更危险,因为同类模型很快也会让其他人更容易找到攻击路径。补丁窗口期不再仅仅是一个运营细节,它已成为产品的一部分。

AI Does Not Remove Security Work

AI 并不能消除安全工作

There is a lazy version of the story where AI agents make security easy. Run the model. Get the report. Apply the patch. Done. That is not how real systems work. Real systems are full of tradeoffs. The obvious fix can break an integration. The technically correct fix can create a migration problem. The most severe-looking vulnerability might be unreachable in production, while the boring one in a forgotten admin path is exposed to the internet. 有一种懒惰的观点认为 AI 代理让安全变得简单:运行模型,获取报告,应用补丁,搞定。现实系统并非如此运作。现实系统充满了权衡:显而易见的修复可能会破坏集成;技术上正确的修复可能会导致迁移问题;看起来最严重的漏洞在生产环境中可能根本无法触达,而一个被遗忘的后台路径中平平无奇的漏洞却暴露在互联网上。

AI can help find and explain these problems. It can write a first patch. It can generate a regression test. It can compare similar code paths and look for variants. But someone still has to own the decision. That is the same pattern I keep seeing across AI-assisted building. The manual work shrinks, but the judgment work expands. The operator has to decide what deserves attention, what can wait, what risk is acceptable, and what needs a deeper human review. Security is just where this becomes impossible to ignore. AI 可以帮助发现并解释这些问题,可以编写初步补丁,生成回归测试,比较相似代码路径并寻找变体。但最终仍需有人负责决策。我在 AI 辅助开发中不断看到同样的模式:手动工作减少了,但判断工作增加了。操作者必须决定什么值得关注,什么可以等待,什么风险是可以接受的,以及什么需要更深入的人工审查。安全领域只是让这一点变得无法忽视。

The Dangerous Middle

危险的过渡期

There is a period we are entering that feels especially unstable. Eventually, AI should make software much safer. Every serious codebase should have agents continuously searching for vulnerabilities, proposing patches, generating tests, and checking whether fixes actually landed. That world is better than the one we have now. But the transition is messy. The discovery side is improving faster than the response side. That creates a gap. More findings, more reports, more possible attack paths, more pressure on teams that already do not have enough security capacity. 我们正在进入一个感觉特别不稳定的时期。最终,AI 应该会让软件变得更安全。每个严肃的代码库都应该有代理持续搜索漏洞、提出补丁、生成测试并检查修复是否生效。那个世界比我们现在所处的更好,但过渡期是混乱的。发现侧的进步速度快于响应侧,这造成了缺口:更多的发现、更多的报告、更多的潜在攻击路径,给那些本就安全能力不足的团队带来了更大的压力。

This is especially painful for open source. A large company can assign security engineers, rotate incident response, and fund dedicated tooling. A maintainer with a popular library might be doing all of this after work, for free, while also reviewing feature requests and answering issue comments. Dumping hundreds of AI-generated reports into that maintainer’s inbox does not automatically make the ecosystem safer. It might make it worse unless the reports are high quality, reproducible, prioritized, and paired with patches that are easy to review. AI security only works if it respects the human throughput on the other side. 这对开源项目尤其痛苦。大公司可以指派安全工程师、轮换应急响应并资助专用工具。而一个流行库的维护者可能是在下班后免费做这一切,同时还要审查功能请求并回答问题。将数百份 AI 生成的报告塞进维护者的收件箱并不会自动让生态系统更安全。除非这些报告高质量、可复现、有优先级且附带易于审查的补丁,否则情况反而会恶化。AI 安全只有在尊重人类处理能力的前提下才能发挥作用。

What Builders Should Change Now

开发者现在应该做出哪些改变

If you are building with AI agents, this is not just a cybersecurity industry story. It changes the default operating model for anyone shipping software. The old advice was “move fast and break things.” The AI-era version needs an asterisk: Move fast, but build a system that can notice what broke. That means security cannot be a quarterly cleanup pass. It has to live inside the same loop as product work. 如果你正在使用 AI 代理进行开发,这不仅仅是网络安全行业的故事,它改变了所有软件发布者的默认运营模式。旧的建议是“快速行动,打破常规”。AI 时代的版本需要加个星号:快速行动,但要建立一个能察觉到哪里被打破的系统。这意味着安全不能只是每季度的清理工作,它必须存在于产品开发的循环之中。

  1. Treat every AI-generated change as reviewable work AI code should not feel like magic output. It should feel like a pull request from a very fast junior-to-mid-level engineer who sometimes has excellent instincts and sometimes misses the reason the system is shaped the way it is. Review it. Ask what changed. Ask what assumptions it made. Ask what surfaces it touched. If a change affects auth, payments, permissions, file handling, secrets, user data, or external integrations, slow down. Fast does not mean casual.

  2. 将每一次 AI 生成的变更视为可审查的工作 AI 代码不应被视为神奇的产出,而应被视为来自一位速度极快的初中级工程师的 Pull Request——他有时直觉敏锐,有时却忽略了系统设计的初衷。审查它。询问变更了什么,做了什么假设,触及了哪些接口。如果变更涉及身份验证、支付、权限、文件处理、密钥、用户数据或外部集成,请放慢速度。快并不意味着草率。

  3. Make tests prove the risky behavior AI is good at producing tests that increase coverage and bad at knowing which behavior deserves proof unless you tell it. For security-sensitive changes, generic tests are not enough. Ask for tests that prove the boundary: A user cannot access another user’s data; A disabled feature cannot be invoked through an API path; A webhook cannot be replayed without detection; A malformed upload…

  4. 让测试去验证风险行为 AI 擅长生成提高覆盖率的测试,但除非你明确告知,否则它不知道哪些行为值得验证。对于安全敏感的变更,通用测试是不够的。要求测试去验证边界条件:用户无法访问其他用户的数据;禁用的功能无法通过 API 路径调用;Webhook 无法在未被检测的情况下重放;畸形上传……