There continue to be reasons for software to be slow
There continue to be reasons for software to be slow
软件依然会缓慢的原因
Dan Luu recently published a blog post “There’s no reason for software to be slow anymore” which talks about how various kinds of things are cheaper to do nowadays by virtue of having access to LLMs such as building specialized solutions (e.g. JITs, indexes for search-like problems) as well as workload-specific optimizations. Dan Luu 最近发表了一篇博文《软件不再有理由变慢了》（There’s no reason for software to be slow anymore），文中探讨了由于大语言模型（LLM）的普及，构建专门的解决方案（如 JIT、针对搜索类问题的索引）以及针对特定工作负载的优化变得更加廉价。
We’re not quite at the point where we want to write everything in assembly, but some variant of what Nolan Lawson said about testing, “you can choose how many bugs you want now,” which I less eloquently noted here, is becoming more true for performance. I believe that this statement as written is well-intentioned but incorrect, in much of the same way in which the statement made by Lawson is well-intentioned but incorrect, and in the same way that formal methods advocates arguing that an increasingly larger fraction of software will be formally verified are well-intentioned but incorrect. 我们还没到需要用汇编语言编写一切的地步，但 Nolan Lawson 关于测试所说的“你现在可以选择想要多少 Bug”的某种变体（我在文中也曾以不太优雅的方式提到过），在性能领域正变得越来越真实。我认为这一论断虽然出发点是好的，但并不准确，就像 Lawson 的观点以及那些主张“越来越多的软件将被形式化验证”的形式化方法倡导者们的观点一样，出发点虽好，却并不正确。
To be clear, I’m very much in favor of better testing, use of formal methods, as well as performance work! I’ve done work along these lines at multiple jobs, including my current one! It’s just that I don’t agree with these predictions about the future. In essence, the argument that’s been offered in all of these places goes something like: Desirable property X used to cost $A over the budget $B. Pre-LLMs, the reason people didn’t aim for X was it was over budget. Post-LLMs, getting X costs $A/N < $B because N >> 1. If these premises hold, then people will now spend $A/N for X. 需要明确的是，我非常支持更好的测试、形式化方法以及性能优化工作！我在多份工作中都做过这些，包括现在这份！我只是不同意这些关于未来的预测。本质上，上述所有论点大致都是这样的：理想的属性 X 过去需要花费 $A，超出了预算 $B。在 LLM 出现之前，人们不追求 X 是因为它超出了预算。在 LLM 出现之后，获得 X 的成本变成了 $A/N < $B（因为 N >> 1）。如果这些前提成立，那么人们现在就会花费 $A/N 来获得 X。
On the face of it, if you’ve personally found LLMs useful at improving property X, the argument seems sensible. You might be thinking “this also assumes that people are rational economic actors with perfect knowledge.” Yes, that’s correct. Most of this post will assume that and show how things can go wrong even with such a strong assumption. In practice, yes people are not rational and don’t have perfect knowledge. I’m ignoring that for this post because there’s already a lot of writing on that topic. But it only works in practice if the premises hold. I agree that there are situations where these premises hold. Will certain highly experienced people with deep domain expertise (like the ones cited in Luu’s post) do a bunch more optimizations, or work on teams which ship many more optimizations than before? Yeah, I think that’s definitely going to happen. However, based on what I’ve seen so far, the situations in which the premises hold are far outweighed by the situations in which they do not hold. 表面上看，如果你亲身体验过 LLM 在提升属性 X 方面的作用，这个论点似乎很有道理。你可能会想：“这还假设了人们是拥有完美知识的理性经济人。”没错，确实如此。本文大部分内容将基于这一假设，并展示即使在如此强的假设下，事情仍可能出错。在实践中，人们确实不是理性的，也没有完美知识。我在这篇文章中忽略了这一点，因为关于该主题已经有很多论述了。但只有在前提成立的情况下，这种逻辑才在实践中有效。我承认在某些情况下这些前提是成立的。那些拥有深厚领域专业知识的资深人士（如 Luu 文中提到的那些）是否会进行更多的优化，或者在比以前交付更多优化的团队中工作？是的，我认为这肯定会发生。然而，根据我目前的观察，前提成立的情况远少于前提不成立的情况。
In this post, I’m going to give examples of situations in which I’ve seen these premises not holding. The tolerance for ‘not X’ goes up (Or: “the desirability for X goes down”). One of the differences with the advent of LLMs is that the work you would do synchronously now potentially needs to be done asynchronously, due to latency of agentic loop iterations. As a concrete example of this, I’ve been working on improving git performance for our monorepo at work recently. If you took the performance numbers we see today on a good day, and you gave them to me from 2022, and told me that people find the same numbers acceptable, I would likely have given you a very skeptical or confused look. 在本文中，我将举例说明这些前提不成立的情况。对“非 X”的容忍度提高了（或者说：“对 X 的需求降低了”）。LLM 出现带来的差异之一是，由于智能体循环迭代的延迟，原本可以同步完成的工作现在可能需要异步完成。举个具体的例子，我最近一直在为公司的大型代码库（monorepo）改进 git 性能。如果你拿我们今天在理想状态下的性能数据，回到 2022 年给我看，并告诉我人们觉得这些数据是可以接受的，我可能会用非常怀疑或困惑的眼光看着你。
As another example, the latency for LLM-based auto-complete used to be much higher than standard IDE auto-complete when it was introduced. This changed later as Cursor and other editors introduced smaller, specialized models for faster completions. Around that time, if you saw videos of developers live-coding, you’d notice them having small pauses waiting for the LLM suggestions. But historically, one of the reasons auto-completion was purportedly prized was the “instant” feedback! This point also applies to things like compilation speed, link times, time to run tests etc. In general, people’s tolerances for synchronous work and asynchronous work are quite different. 再举一个例子，基于 LLM 的自动补全在刚推出时，其延迟远高于标准的 IDE 自动补全。后来随着 Cursor 等编辑器引入了更小、更专业的模型以实现更快的补全，这种情况有所改变。在那段时间，如果你看开发者直播写代码的视频，会注意到他们经常会有短暂的停顿来等待 LLM 的建议。但从历史上看，自动补全之所以被推崇，原因之一就是“即时”反馈！这一点同样适用于编译速度、链接时间、测试运行时间等。总的来说，人们对同步工作和异步工作的容忍度是非常不同的。
If you’re a performance-minded person, it can be hard to accept that people are actually fine with putting up with worse performance in software, especially if you already believe that the performance of said software is “too slow.” It can be doubly-frustrating if the same people are willing to put up with worse performance specifically in exchange for more features, especially if you already believe that the said software is “too bloated.” 如果你是一个注重性能的人，很难接受人们实际上愿意忍受软件性能变差，特别是当你已经认为该软件“太慢”的时候。如果这些人为了换取更多功能而专门忍受更差的性能，这会让人加倍沮丧，尤其是当你已经认为该软件“太臃肿”的时候。
The budget was zero from the start. Outside of well-paying tech companies that treat developers well, granting them a fair amount of autonomy, it’s common in many companies for the software function to be perceived as a “cost center” instead of a “profit center”. There may not even be a CI process – it may be entirely reliant on manual QA. Getting budget approvals might take ages. And yet, the business might be doing well! For example, the company might have a government-granted monopoly. Or it might have some other form of power. If the environment is entirely focused on keeping costs low, it likely requires a fair bit of effort to convince a manager of the return on investment (RoI) of working on performance. It’s plausible that this effort is better spent elsewhere. 预算从一开始就是零。除了那些待遇优厚、给予开发者相当大自主权的科技公司外，在许多公司中，软件部门通常被视为“成本中心”而非“利润中心”。甚至可能根本没有 CI 流程，完全依赖人工 QA。获得预算审批可能需要很长时间。然而，业务可能依然做得很好！例如，公司可能拥有政府授予的垄断地位，或者拥有其他形式的权力。如果环境完全专注于降低成本，那么要说服经理相信性能优化工作的投资回报率（RoI）可能需要付出相当大的努力。这种努力花在其他地方可能更划算。
The budget got reduced post-LLMs. Say the budget started out at non-zero. For example, you might’ve already been spending about 1 week on performance every quarter. Even so, there’s an implicit assumption that the budget $B for obtaining the property X is unchanged post-LLMs. This assumption often fails to hold. If you browse the r/experienceddevs subreddit, it’s not uncommon to see engineers talking about how, over the past year, timelines for projects are getting squeezed tighter, because management expects things to take much less time. LLM 出现后预算反而减少了。假设预算最初不是零，例如你可能每季度已经花费大约一周的时间在性能优化上。即便如此，这里隐含了一个假设，即获得属性 X 的预算 $B 在 LLM 出现后保持不变。这个假设往往无法成立。如果你浏览 r/experienceddevs 子版块，经常会看到工程师们讨论在过去一年里，项目时间表是如何变得越来越紧的，因为管理层期望事情能更快完成。