Models Are Getting Dumber on Purpose

模型正在“故意”变笨

2026-08-17 | Walter van der Giessen

Reasoning scores keep climbing while per-token compute keeps dropping. GLM-5.2 scores 99.2% on AIME 2026 with about 40 billion parameters active per token. Qwen3.5 scores 91.3% with 17 billion active. DeepSeek V4-Flash runs 13 billion active. For scale, GPT-4 was rumored to run around 280 billion active parameters in 2023, and it could barely solve an AIME problem. 推理得分不断攀升，而每个 token 的计算成本却在持续下降。GLM-5.2 在 AIME 2026 测试中取得了 99.2% 的分数，每个 token 激活参数约为 400 亿；Qwen3.5 以 170 亿激活参数取得了 91.3% 的分数；DeepSeek V4-Flash 则仅需 130 亿激活参数。作为对比，传闻 2023 年的 GPT-4 激活参数约为 2800 亿，却几乎无法解决 AIME 的数学题。

At the small end, Qwen3.5 9B fits in 6GB of VRAM quantized and roughly doubles the score of the next best model under 10B parameters on Artificial Analysis’s intelligence index. If you only looked at math and code benchmarks, you’d conclude that models are getting smarter per parameter at an absurd rate. They are, on those benchmarks. 在小模型领域，量化后的 Qwen3.5 9B 仅需 6GB 显存，其在 Artificial Analysis 智能指数上的得分几乎是 10B 参数以下第二名模型的两倍。如果你只看数学和代码基准测试，你会得出结论：模型在单位参数上的智能提升速度快得离谱。在这些基准测试上，事实确实如此。

Ask the same models a plain factual question and the picture flips. On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions. The small models barely register. Artificial Analysis measures Qwen3.5 4B and 9B at hallucination rates of 80 to 82% on its knowledge benchmark, which means that when they don’t know a fact, which is most of the time, they make one up. Ask the 9B for the birth year of a minor 19th-century mathematician and you get a confident, plausible, wrong answer. 但如果向这些模型提出简单的常识性问题，情况就完全反转了。在禁止使用工具的事实检索基准测试 SimpleQA 上，目前的领先者 Gemini 2.5 Pro 得分仅为 53%，这意味着即便花大价钱买到的最强检索能力，依然有一半的问题答不上来。小模型则几乎完全不可用。Artificial Analysis 的知识基准测试显示，Qwen3.5 4B 和 9B 的幻觉率高达 80% 到 82%，这意味着当它们不知道某个事实时（这种情况占大多数），它们会直接编造一个。如果你问 9B 模型一位 19 世纪小众数学家的出生年份，你会得到一个自信、听起来合理但完全错误的答案。

The parameter count didn’t drop for free. Labs are trading world knowledge for reasoning skill, and the trade is deliberate. 参数数量的下降并非没有代价。实验室正在用“世界知识”交换“推理能力”，这种权衡是刻意为之的。

What the parameters were for

参数原本的用途

Facts take space. Research on knowledge capacity (the “Physics of Language Models” series has the cleanest measurements) puts it on the order of two bits of factual knowledge per parameter. If you want a model that knows the birth year of every minor Wikipedia figure, the population of every Dutch municipality, and the argument order of every function in every npm package, you pay for that in weights, and it’s a big part of why frontier models grew to trillions of parameters. 事实需要占用空间。关于知识容量的研究（“语言模型物理学”系列有最清晰的测量数据）表明，每个参数大约能存储两比特（bits）的事实知识。如果你想要一个模型记住维基百科上每一个小人物的出生年份、荷兰每一个市镇的人口，以及每一个 npm 包中每个函数的参数顺序，你就必须在权重中为此买单，这也是前沿模型参数量增长到万亿级别的主要原因。

Reasoning compresses much better than facts do, because it’s a relatively small set of procedures applied over and over: break the problem into parts, track intermediate state, check your own work, backtrack when a step fails. Distillation and reinforcement learning on verifiable tasks turn out to transfer those procedures into small models remarkably well. Phi-4 is 14 billion parameters, trained heavily on synthetic textbook-style data, and it’s good at math and bad at trivia, which tells you exactly what its training data contained. That mix used to look like a limitation of the synthetic-data approach. It now looks like the design goal. 推理比事实更容易压缩，因为它是一套相对较小的、被反复应用的程序：将问题拆解、跟踪中间状态、检查自己的工作、在步骤失败时回溯。事实证明，通过蒸馏和在可验证任务上的强化学习，可以将这些程序非常出色地迁移到小模型中。Phi-4 拥有 140 亿参数，主要在合成的教科书式数据上进行训练，它擅长数学但不擅长琐事，这恰恰反映了其训练数据的构成。这种组合过去被视为合成数据方法的局限性，现在看来，这正是设计目标。

The knowledge that survives the trade has a shape

权衡后留下的知识是有“形状”的

These models are generalists: they know a little about nearly everything and almost nothing in depth. Ask one about PostgreSQL and it knows what it is, what it’s good at, and roughly how MVCC works, but ask which version added a specific planner feature and you’re back to invented facts. That’s the right layer to keep in weights, because breadth is what lets a model understand what a question is about, know what to look up, and judge whether a source is plausible. The depth is cheap to retrieve and expensive to store, so it’s the part that goes. 这些模型是通才：它们对几乎所有事情都略知一二，但几乎没有深入了解。问它们关于 PostgreSQL 的问题，它们知道它是什么、擅长什么以及 MVCC 大致如何工作；但如果你问是哪个版本增加了某个特定的规划器功能，它们又会开始编造事实。这正是权重中应该保留的知识层面，因为广度让模型能够理解问题的核心、知道去哪里查找信息，并判断来源是否可信。深度知识检索成本低但存储成本高，所以这部分被舍弃了。

Facts rot, procedures don’t

事实会过时，程序不会

A frontier training run takes months and costs hundreds of millions of dollars, and the moment it finishes, the facts inside it start going stale. Library APIs change, prices change, people change jobs, and half of what a 2024 model believed about the JavaScript ecosystem was outdated before the model shipped. Every fact you bake into weights has a shelf life, and the only way to refresh it is another training run. 一次前沿模型的训练需要数月时间，耗资数亿美元，而当训练完成的那一刻，其中的事实就开始过时了。库的 API 会变、价格会变、人员会跳槽，2024 年的模型对 JavaScript 生态系统的认知，有一半在模型发布前就已经过时了。你写入权重的每一个事实都有保质期，而更新它的唯一方法就是重新训练。

The procedures don’t rot. Algebra worked the same way in 1970 as it does now, and so does breaking a problem down or spotting a contradiction between two sources. A model that’s mostly procedure and only lightly loaded with facts doesn’t age the way a knowledge-heavy model does. Its training cutoff matters much less, because the current state of the world was never supposed to live in the weights in the first place. I think this is the best argument for the whole approach: it decouples the expensive, slow artifact (the trained model) from the thing that changes daily (what’s true). 程序不会过时。代数在 1970 年和今天的工作方式是一样的，拆解问题或发现两个来源之间的矛盾也是如此。一个以程序为主、事实负载较轻的模型，不会像知识密集型模型那样迅速老化。它的训练截止日期（cutoff）变得不再那么重要，因为世界的现状本就不该存储在权重里。我认为这是支持这种方法最有力的论据：它将昂贵且缓慢的产物（训练好的模型）与每天都在变化的事物（事实真相）解耦了。

The harness carries the knowledge

外部框架承载知识

If the model doesn’t know things, something else has to, and that something is the harness: retrieval over a knowledge base, tool calls, web search, a filesystem full of docs. I wrote earlier that Rust is a harness for agents, a source of cheap machine-checkable feedback. This is the same shape from the other side. The model contributes reasoning, and everything it reasons about gets supplied at runtime. 如果模型不知道某些事情，那就必须有其他东西来承担，这个东西就是“外部框架”（harness）：基于知识库的检索、工具调用、网络搜索、充满文档的文件系统。我之前写过，Rust 是智能体的框架，是一种廉价的、机器可检查的反馈来源。从另一面看，这本质是一样的。模型提供推理能力，而它推理所需的一切都在运行时提供。

You can already watch agents work this way. A coding agent doesn’t need to have memorized your dependency’s API surface, because it greps node_modules or reads the docs before calling anything, and its answer is grounded in the version you actually have installed rather than whichever version dominated the training data. The recall that used to be a fixed cost in every forward pass became an on-demand lookup. 你已经可以看到智能体是这样工作的了。一个编程智能体不需要背诵你依赖库的 API 接口，因为它在调用任何东西之前会先搜索 node_modules 或阅读文档，它的回答是基于你实际安装的版本，而不是训练数据中占主导地位的某个版本。过去在每次前向传播中作为固定成本的“记忆检索”，现在变成了按需查询。

A frontier model on your GPU

在你的 GPU 上运行前沿模型

Follow the trend a couple of years out and I think we get a model with frontier-quality reasoning, Fable-quality, that runs on a single consumer GPU. The compute half is nearly there. DeepSeek V4-Flash reasons with about 13 billion active parameters per token, well within consumer-GPU range. What doesn’t fit is the other 271 billion parameters sitting in its experts, and expert layers are mostly fact storage. That’s the part this whole trade makes optional. Strip the knowledge out and total size shrinks toward active size, and a 20 to 40B model at 4-bit quantization fits on the 24GB card that’s been sitting in gaming PCs since 2022. 顺着这个趋势再过几年，我认为我们将得到一个具备前沿推理质量（如 Fable 级别）的模型，且可以在单张消费级 GPU 上运行。计算部分已经快达标了。DeepSeek V4-Flash 每个 token 的推理激活参数约为 130 亿，完全在消费级 GPU 的能力范围内。装不下的部分是它专家层中剩余的 2710 亿参数，而专家层大多是事实存储。这正是这种权衡使其变得可选的部分。剥离掉知识，总参数量就会向激活参数量靠拢，一个 200 亿到 400 亿参数的模型在 4-bit 量化下，完全可以塞进 2022 年以来游戏 PC 中常见的 24GB 显存显卡里。

The catch is that it won’t know much. Ask it a bare factual question with no tools attached and the right behavior is to say it doesn’t know and go look it up. 代价是它不会知道太多东西。如果你在不提供任何工具的情况下问它一个纯粹的事实问题，它正确的反应应该是说它不知道，然后去查阅资料。