Beyond Scaling Laws: Why “Thinking Longer” Is a Systems Problem, Not a Prompting Trick

超越缩放定律：为什么“深度思考”是系统工程问题，而非提示词技巧

For five years, the answer to “how do we make the model better” was always the same: bigger model, more data, wait for the loss curve to bend. Scaling laws made AI progress feel almost boring — predictable, like compound interest. That knob still works. But it’s not the interesting one anymore. 过去五年里，“如何让模型变得更好”的答案始终如一：更大的模型、更多的数据，等待损失曲线下降。缩放定律（Scaling laws）让 AI 的进步显得有些乏味——就像复利一样可预测。虽然这个调节旋钮依然有效，但它已不再是核心焦点。

The real question teams are fighting with now is: given a model you already have, how much compute should it spend per question? Not per training run. Per question. And the answer turns out to break almost every assumption your inference stack was built on. 现在各团队真正纠结的问题是：在已有模型的基础上，针对每一个问题应该投入多少算力？不是针对训练过程，而是针对每一个具体问题。事实证明，这个问题的答案打破了你现有推理架构几乎所有的假设。

The single forward pass was always the weak link. Here’s the thing nobody says out loud enough: a model’s first answer is often just a noisy sample. The right answer was in there somewhere — it just wasn’t the loudest one. You don’t fix that with more pretraining tokens. You fix it by sampling more, checking the work, or letting the model take a second pass. That’s the whole idea behind test-time compute. 单次前向传播（Single forward pass）一直是薄弱环节。有一点很少有人大声说出来：模型的第一次回答往往只是一个带有噪声的样本。正确答案其实就在那里，只是它不是概率最高的那一个。你无法通过增加预训练 Token 来解决这个问题，只能通过多次采样、核对工作或让模型进行二次思考来解决。这就是“测试时计算”（Test-time compute）的核心理念。

And there are three ways to spend that extra compute, and they are not interchangeable: 投入这些额外算力有三种方式，且它们不可相互替代：

1. Let it ramble longer. Cheap. Also the reason some models talk themselves out of the right answer after 3,000 tokens of unnecessary detours.

2. 让它长篇大论。成本低，但这也是为什么有些模型在经过 3000 个 Token 的无效绕路后，反而推翻了正确答案的原因。

3. Generate N answers, vote. Parallel, simple, plugs into infra you already have. Also: returns drop off fast. Past a certain N you’re burning GPU-seconds for crumbs.

4. 生成 N 个答案并投票。这种方式并行且简单，可以接入现有基础设施。但缺点是收益递减极快，超过一定数量后，你就是在浪费 GPU 时间来换取微不足道的提升。

5. Search a tree of partial reasoning steps, prune the bad branches early with a verifier. The most compute-efficient option by far. Also the one that quietly breaks your serving stack.

6. 在推理步骤树中进行搜索，并利用验证器尽早剪掉错误分支。这是目前算力效率最高的方式，但它也会悄无声息地摧毁你的服务架构。

Most real systems end up doing all three — cheap by default, escalating only when a confidence check says “this one’s hard.” That escalation decision is its own piece of architecture. Most teams bolt it on as an afterthought. It shouldn’t be one. 大多数实际系统最终会结合这三种方式——默认采用低成本方案，仅在置信度检查显示“此题较难”时才升级算力。这种升级决策本身就是架构的一部分，但大多数团队只是将其作为事后补丁。这不应该被视为补丁。

Your serving stack was not built for this. Continuous batching, paged KV-cache, speculative decoding — all of that infra assumes requests are independent, similar in length, and finish in one pass. Test-time compute violates every one of those assumptions: 你的服务架构并非为此而生。连续批处理（Continuous batching）、分页 KV 缓存（Paged KV-cache）、投机采样（Speculative decoding）——所有这些基础设施都假设请求是独立的、长度相似的，并且能在一次传递中完成。而“测试时计算”违背了所有这些假设：

• A search request spawns children that share a parent’s cache. Miss that, and you’re recomputing the same prefix hundreds of times per query.

• 搜索请求会产生共享父级缓存的子请求。如果处理不好，你会在每次查询中重复计算相同的前缀数百次。

• Length goes wildly bimodal — easy queries finish fast, hard ones branch into dozens of long threads. Tune your scheduler for the median and you’ll starve the long tail or let it eat your throughput.

• 长度呈现极端的双峰分布——简单查询快速结束，困难查询则分支成数十个长线程。如果你的调度器只针对中位数进行优化，要么会导致长尾任务饥饿，要么会拖垮整体吞吐量。

• One “request” now touches multiple models: generator, verifier, maybe a router. That’s a distributed pipeline, not a single call.

• 一个“请求”现在涉及多个模型：生成器、验证器，可能还有路由器。这是一个分布式流水线，而非单一调用。

If you’ve built speculative decoding before, forking KV-cache across search branches will feel familiar. It’s the same “shared prefix, divergent suffix” problem — just stretched over a much longer horizon, with a lot more at stake if you get the cache management wrong. 如果你曾经构建过投机采样系统，那么在搜索分支间分叉 KV 缓存的概念会让你感到熟悉。这本质上是同一个“共享前缀，发散后缀”的问题——只是被拉长到了更远的维度，一旦缓存管理出错，代价将大得多。

The actual shift: Scaling laws told us how to spend money before a model ever met a user. Test-time compute asks the harder question: how do you spend money per question, in real time, without quietly teaching your search process to fool its own judge? That’s not a prompting trick. It’s inference architecture with an alignment problem sitting inside it — and it deserves the same rigor we’ve spent five years pouring into pretraining. 真正的转变在于：缩放定律告诉我们在模型接触用户之前如何花钱。而“测试时计算”提出了一个更难的问题：如何在实时环境下，针对每个问题合理分配算力，且不让搜索过程学会欺骗自己的判别器？这绝非提示词技巧，而是一个内嵌了对齐问题的推理架构——它值得我们投入过去五年在预训练领域所付出的同等严谨度。