本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Neural networks are clean

The Short Version I think I have good news about the nature of neural networks. After many months studying neurons and circuits in image and language models, I’ve come to believe a network’s own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I’d feared. The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I’ll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.

简短版本 我认为关于神经网络的本质，我有一些好消息。在研究了图像和语言模型中的神经元与电路数月后，我开始相信网络自身的机制就是其清晰易读的构建模块，且机械可解释性的前景可能远比我担心的要好。此前让该领域大部分人（包括我）确信网络会随意打包概念的“多义性”和“叠加”的典型证据，其实是被误读了。那些看起来像是随意打包的现象，大多是一种我称之为“递归泛化”的模式，这并不那么令人担忧。这意味着我们可以直接研究网络，而无需先对其进行解构。神经网络本身就是清晰的。

There are at least two ways to approach mechanistic interpretability. The first is rapid normal science on the network itself: look at a network’s neurons and circuits, study how they relate, watch data flow through them, and run experiments at agentic scale to test explanations. The second starts from a list of hypothesized features and tries to extract them all, designing increasingly sophisticated reconstruction objectives and usually ignoring the original.

研究机械可解释性至少有两种途径。第一种是对网络本身进行快速的常规科学研究：观察网络的神经元和电路，研究它们之间的关系，观察数据如何在其中流动，并以代理规模运行实验来验证解释。第二种途径是从一系列假设的特征列表出发，试图将它们全部提取出来，设计日益复杂的重构目标，通常会忽略原始网络。

The field began with the first approach and fell back to the second because neural networks seemed to be tangled knots. We had canonical examples of polysemantic neurons, the broad name for a neuron that appears to have multiple unrelated jobs or that looks impossible to understand. The most famous one, in an image model, responded to both cat faces and the fronts of cars. And the last layer of the early field’s model organism, InceptionV1, seemed so full of arbitrary mixtures it wasn’t clear how to study it: almost none of the neurons were easily interpretable.

该领域最初采用第一种方法，但后来退回到第二种，因为神经网络看起来像是一团乱麻。我们有“多义神经元”的典型例子，这是一个宽泛的术语，指那些似乎有多个不相关工作或看起来无法理解的神经元。最著名的一个例子出现在一个图像模型中，它既对猫脸做出反应，也对汽车前部做出反应。而早期领域模型生物 InceptionV1 的最后一层似乎充满了随意的混合，以至于不清楚该如何研究它：几乎没有任何神经元是容易解释的。

The leading explanation for polysemanticity was superposition. This proposed a new learning principle: when a neural network runs out of neurons, it stops grouping related things together and instead begins folding unrelated things together, simply because they don’t interfere. The result is a kind of knot: the network could have folded in many almost equally good ways, so which things a neuron groups together tells you almost nothing. You have to unfold the knot before you can study anything. In practice that means training a second model – a decomposition such as a sparse autoencoder – to approximate the network as it would look unfolded and studying that instead. If the network really is knotted, there may be no alternative.

对于多义性的主要解释是“叠加”。这提出了一个新的学习原则：当神经网络的神经元用尽时，它会停止将相关事物归类在一起，转而开始将不相关的事物折叠在一起，仅仅因为它们不会相互干扰。结果形成了一种结：网络可以以许多几乎同样好的方式进行折叠，因此一个神经元将哪些事物归在一起几乎无法告诉你任何信息。你必须先解开这个结，才能研究任何东西。在实践中，这意味着训练第二个模型——例如稀疏自动编码器这样的分解模型——来近似网络解开后的样子，并转而研究它。如果网络确实是纠结的，可能别无他法。

One main result is that superposition was the wrong explanation for InceptionV1. Architectural problems made later layers overflow. Weak interpretability tools amplified these problems, making the late layers seem incoherent. Since then, architectural changes designed to help deep layers learn better have had the unintended effect of making them more interpretable. When we first studied InceptionV1, we found almost no interpretable neurons in its last layer. But with these ordinary improvements, and no change to model size or training data, 99-100% of neurons in that last layer are easily interpretable and the model is broadly legible the whole way through.

一个主要结论是，叠加并不是 InceptionV1 的正确解释。架构问题导致了后续层的溢出。薄弱的可解释性工具放大了这些问题，使得后续层看起来不连贯。从那时起，旨在帮助深层网络更好学习的架构改进，产生了一个意想不到的效果：使它们变得更具可解释性。当我们最初研究 InceptionV1 时，在它的最后一层几乎找不到可解释的神经元。但通过这些常规改进，在不改变模型规模或训练数据的情况下，最后一层中 99-100% 的神经元都变得易于解释，且整个模型在很大程度上是清晰可读的。

It’s the first non-trivial neural network I feel I mostly understand; you can browse it here. I trained versions of it across a 300× range in size, and it’s broadly legible at every scale. With less room, the network doesn’t fold unrelated things together; its neurons just cover coarser-grained concepts, like one car neuron covering many kinds of car. With more room, that neuron differentiates into many specific cars. So the problem wasn’t the presence of a second learning principle, it was just an ordinary learning failure of the first one, and it’s easy to fix.

这是第一个我觉得自己基本理解的非平凡神经网络；你可以在这里浏览它。我训练了跨度达 300 倍规模的多个版本，它在每个尺度上都大体可读。当空间较小时，网络并不会将不相关的事物折叠在一起；它的神经元只是涵盖了更粗粒度的概念，比如一个“汽车神经元”涵盖了多种汽车。当空间更大时，该神经元会分化出许多特定的汽车类型。因此，问题不在于存在第二个学习原则，而仅仅是第一个原则的普通学习失败，而且这很容易修复。

The canonical vision examples of polysemanticity and superposition were errors too. In 2019, Chris Olah and I found the circuit that was originally introduced as the superposition phenomenon in Zoom In. It showed a car feature appearing to spread into a set of unrelated dog neurons. It was the only one we showed, and the only one I ever saw. We read it as the network placing the car in a high-dimensional hiding place, for downstream neurons to read back out. But nothing reads that combination. It wasn’t hiding anything; it was ordinary generalization and composition. To my knowledge, no one has published a second circuit with the right shape.

多义性和叠加的典型视觉示例也是错误的。2019 年，Chris Olah 和我发现了最初在《Zoom In》中作为叠加现象引入的电路。它显示了一个汽车特征似乎扩散到了一组不相关的狗神经元中。这是我们展示的唯一一个例子，也是我见过的唯一一个。我们将其解读为网络将汽车放置在一个高维隐藏处，以便下游神经元读取。但没有任何东西会读取那种组合。它并没有隐藏任何东西；这只是普通的泛化和组合。据我所知，没有人发表过第二个具有这种形状的电路。

Modern language models are also far more interpretable than generally believed. Anthropic’s Towards Monosemanticity (2023), studying a small one-layer model, gave the median neuron a 0 on its rubric out of a possible 14, meaning the rater couldn’t form any hypothesis. The automated interpretability project I started at OpenAI (also 2023) found about 1.7% of neurons interpretable by our most generous measure.

现代语言模型也比普遍认为的要容易解释得多。Anthropic 的《迈向单义性》（2023 年）研究了一个小型单层模型，给出的中位数神经元在 14 分的评分标准中仅得 0 分，这意味着评估者无法形成任何假设。我在 OpenAI 发起的自动化可解释性项目（同样是 2023 年）发现，按照我们最宽松的衡量标准，大约只有 1.7% 的神经元是可解释的。

With better tools and modern models, the situation looks quite different. In preliminary audits of Qwen3-32B, we estimate that roughly 90% of neuron poles have one main job, often with small related offshoots; another ~7% have two related jobs; and only ~2-4% are not yet understood.

有了更好的工具和现代模型，情况看起来大不相同。在对 Qwen3-32B 的初步审计中，我们估计大约 90% 的神经元极点有一个主要工作，通常伴随着小的相关分支；另外约 7% 有两个相关工作；只有约 2-4% 尚未被理解。