Software is slow because of bad design, not slow languages

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Many people are saying that AI will make software faster by rewriting it in faster languages, but this isn’t generally true. Most programs cannot be meaningfully sped up by rewriting them in C or Rust. How can this be true? Some languages are orders-of-magnitude faster than others: for instance, C can be fifty times faster than Python. Won’t rewriting my Python program in C thus make it fifty times faster? No, it won’t. Software performance is usually dominated by I/O, not the speed of the language. Where language speed matters, most slow languages already have popular libraries that allow them to unlock C-like speed. In general, performance is a matter of design, not of language choice.

许多人认为人工智能可以通过用更快的语言重写软件来提高其速度，但这通常是不正确的。大多数程序无法通过用 C 或 Rust 重写来获得显著的性能提升。这怎么可能呢？有些语言比其他语言快几个数量级：例如，C 语言可能比 Python 快五十倍。那么，用 C 语言重写我的 Python 程序难道不会使其速度提升五十倍吗？不，并不会。软件性能通常由 I/O（输入/输出）决定，而不是由语言速度决定。在语言速度确实重要的地方，大多数慢速语言已经拥有了流行的库，允许它们解锁类似 C 语言的速度。总的来说，性能是设计的问题，而不是语言选择的问题。

Most real programs are I/O-bound Consider a simple CRUD website with a database. For any given request, 50% of the request time might be spent querying the database: to fetch the current user, the record being asked for, any relevant user memberships or settings, and so on. Rewriting that website in C can only speed up that other 50%, which puts a hard ceiling on the improvements you can make. At best, you’ll make requests twice as fast (probably closer to 1.5x in practice). That’s not bad, but it’s nowhere near an order-of-magnitude speedup. In other words, most real programs are “I/O-bound”: they spend most of their time waiting for data to be input or output somewhere, as opposed to “CPU-bound” programs that spend most of their time actively doing things. I/O-bound programs don’t just have to rely on a database. Often programs have to wait on some third-party API, or on writing or reading data to disk. You could speed the code in curl up by 100x and you probably wouldn’t even notice, because curl spends all of its time waiting on the network.

大多数实际程序都是“I/O 密集型”的。考虑一个带有数据库的简单 CRUD 网站。对于任何给定的请求，50% 的请求时间可能花在查询数据库上：获取当前用户、请求的记录、任何相关的用户成员资格或设置等等。用 C 语言重写该网站只能加速剩下的那 50%，这为你所能做的改进设定了一个硬性上限。充其量，你会使请求速度提高一倍（实际上可能更接近 1.5 倍）。这还不错，但远未达到数量级的速度提升。换句话说，大多数实际程序都是“I/O 密集型”的：它们大部分时间都在等待数据在某处输入或输出，这与大部分时间都在积极处理任务的“CPU 密集型”程序不同。I/O 密集型程序不仅仅依赖于数据库。程序通常必须等待某些第三方 API，或者等待向磁盘写入或读取数据。你可以将 curl 中的代码速度提高 100 倍，但你可能根本察觉不到，因为 curl 的所有时间都花在等待网络上。

Slow languages already delegate work to fast languages What about work that is obviously not I/O-bound? For instance, wikis have to turn stored markdown into renderable HTML. Wouldn’t rewriting that in a faster language cause a massive speedup? It would! That’s why applications that do that use libraries like cmark-gfm that call out to fast languages for markdown rendering. In general, most slow languages can write very fast programs by invoking this kind of library. Python is famously a slow language, but it’s a competitive choice for CPU-bound workloads like machine learning and mathematics, because it has a healthy set of libraries that let it offload these workloads to fast C code. If a Python program spends 80% of its time doing matrix multiplications on a GPU, you are very unlikely to be able to speed it up by rewriting it in C, because those matrix multiplications are already happening in tightly-optimized C or assembly code.

慢速语言已经将工作委托给了快速语言。那么那些显然不是 I/O 密集型的工作呢？例如，维基（wikis）必须将存储的 Markdown 转换为可渲染的 HTML。用更快的语言重写它难道不会带来巨大的速度提升吗？确实会！这就是为什么执行此类操作的应用程序会使用像 cmark-gfm 这样的库，调用快速语言进行 Markdown 渲染。总的来说，大多数慢速语言可以通过调用这类库来编写非常快的程序。Python 以慢著称，但对于机器学习和数学等 CPU 密集型工作负载来说，它是一个具有竞争力的选择，因为它拥有一套完善的库，可以将这些工作负载卸载到快速的 C 代码中。如果一个 Python 程序 80% 的时间都在 GPU 上进行矩阵乘法，那么你几乎不可能通过用 C 语言重写来加速它，因为这些矩阵乘法已经在经过严格优化的 C 或汇编代码中完成了。

Of course, some programs do in fact do a lot of CPU-bound work in slow languages. If a program has not had much engineering attention, there will probably be a lot of low-hanging fruit. But these programs are often unimportant: if they were important, someone would have gone in and optimized them already, before AI made it (relatively) easy. That doesn’t mean it’s impossible to find a useful program to rewrite. Actual Computer recently rewrote tiktoken in assembly, providing a significant speedup. However, you’re not going to revolutionize some field of computing like this. Even fast tokenization is only useful on the margins, because time spent in tokenization is a fraction of the time spent in inference.

当然，有些程序确实在慢速语言中执行了大量的 CPU 密集型工作。如果一个程序没有得到太多的工程关注，可能存在许多容易实现的优化点。但这些程序通常并不重要：如果它们很重要，在人工智能使其变得（相对）容易之前，早就有人进去优化它们了。这并不意味着不可能找到一个值得重写的有用程序。Actual Computer 最近用汇编语言重写了 tiktoken，提供了显著的速度提升。然而，你不会通过这种方式彻底改变某个计算领域。即使是快速的标记化（tokenization）也只是在边缘上有用，因为标记化所花费的时间只是推理所花费时间的一小部分。

Speeding up software typically involves a redesign Despite all this, I think most software is far slower than it could be. I’m optimistic about the potential (with or without AI) to significantly speed up commonly-used software. But I think most of these speedups will come from redesigning the program, not rewriting it in a faster language. In a 2018 post, Bryan Cantrill described trying to rewrite a C program into Rust. It was difficult because the carefully-optimized design of the C program did not play well with the Rust borrow checker. Giving up, he simply wrote the program in Rust in the naive way, expecting a 20% slowdown or worse. Instead, it was 30% faster. The reasons boil down to Rust’s internal preference for a B-tree over a binary search tree, which happens to be a more cache-efficient data structure. So even in a largely CPU-bound workload, in the hands of someone very skilled at writing fast C, performance is dominated by design, not by language speed.

加速软件通常涉及重新设计。尽管如此，我认为大多数软件的速度远低于其应有的水平。我对（无论是否有 AI）显著加速常用软件的潜力持乐观态度。但我认为，这些速度提升中的大部分将来自重新设计程序，而不是用更快的语言重写它。在 2018 年的一篇文章中，Bryan Cantrill 描述了尝试将一个 C 程序重写为 Rust 的过程。这很困难，因为 C 程序经过精心优化的设计与 Rust 的借用检查器（borrow checker）并不兼容。他放弃了，只是以简单的方式用 Rust 重写了该程序，预期会有 20% 或更糟的减速。结果，它反而快了 30%。原因归结为 Rust 内部更倾向于使用 B 树而不是二叉搜索树，而 B 树恰好是一种缓存效率更高的数据结构。因此，即使在很大程度上是 CPU 密集型的工作负载中，在擅长编写快速 C 代码的人手中，性能也由设计决定，而不是由语言速度决定。

This matches my own experience in higher-level languages. When I think of the significant speedups I’ve managed, they’re almost exclusively redesigns of the software to do less work: Caching a slow operation (typically an external API call) Performing slow operations asynchronously, either in a background job or out of the request/response lifecycle Performing operations in parallel instead of sequentially (sometimes breaking up large operations in order to do this) Replacing a set of SQL queries with a single SQL query (fixing N+1 problems) Replacing a slow SQL query with a fast SQL query Skipping unnecessary slow operations (e.g. in a paginated view, only fetch ancillary data from entities that are on the current page) All of these strategies are completely unrelated to how fast the language is. In fact, writing in a slow higher-level language can make it much easier to optimize the code, because you can see at a glance what the program is doing. If (for some reason) I had first rewritten the web server in assembly, I would have a much harder time noticing the N+1 query, or the fact that we could move an API call into a job, and so on.

这与我在高级语言方面的经验相符。当我想到我所取得的显著速度提升时，它们几乎全是软件的重新设计，以减少工作量：缓存慢速操作（通常是外部 API 调用）；异步执行慢速操作，要么在后台作业中，要么在请求/响应生命周期之外；并行执行操作而不是顺序执行（有时为了做到这一点会拆分大型操作）；用单个 SQL 查询替换一组 SQL 查询（修复 N+1 问题）；用快速 SQL 查询替换慢速 SQL 查询；跳过不必要的慢速操作（例如在分页视图中，仅获取当前页面上实体的辅助数据）。所有这些策略都与语言的速度完全无关。事实上，使用慢速高级语言编写代码可以更容易地优化代码，因为你可以一眼看出程序在做什么。如果（出于某种原因）我首先用汇编语言重写了 Web 服务器，我将很难注意到 N+1 查询，或者我们本可以将 API 调用移至后台作业等事实。

Conclusion Right now everyone is asking their AI agents to rewrite programs in Rust or C or assembly. This is fun, but it’s not a serious attempt at optimization. The set of programs that can be meaningfully sped up by this kind of rewrite is very narrow. If you want to make everything faster with AI — and you should! — you ought instead to be asking your agent to profile programs, identify hotspots, and suggest ways to redesign what

结论：现在每个人都在要求他们的人工智能代理用 Rust、C 或汇编语言重写程序。这很有趣，但这不是一种严肃的优化尝试。可以通过这种重写方式获得显著加速的程序集非常有限。如果你想用人工智能让一切变得更快——你应该这样做！——你反而应该要求你的代理去分析程序、识别热点，并建议重新设计的方法。