Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

测试，然后路由：语言模型如何在不同模型和语言中执行上下文条件规则

Abstract: When a language model follows an in-context conditional rule such as “if P(x) then A else B,” does it assemble a runtime circuit with one module that tests the predicate and another that routes the answer?

摘要： 当语言模型遵循诸如“如果 P(x) 则 A，否则 B”之类的上下文条件规则时，它是否会组装一个运行时电路，其中一个模块负责测试谓词，另一个模块负责路由答案？

We probe this with activation patching under a four-donor design whose two swapped-rule donors make the condition and the answer word disagree, so each layer reveals which of the two it carries.

我们通过四供体设计下的激活修补（activation patching）对此进行了探测，其中两个交换规则的供体使得条件和答案词不一致，因此每一层都能揭示它携带的是两者中的哪一个。

Across three open models from two families and six languages sharing one fixed item bank, a mid-stack residual band carries the predicate’s truth value: patching it reroutes the answer with predicate-outcome flip near 1.0 and mapping flip near 0.0, meeting a strict pre-specified isolation criterion in 17 of 18 cells, and the same localization holds across five predicate families.

在来自两个模型家族的三个开源模型和共享一个固定项目库的六种语言中，中间层的残差带携带了谓词的真值：对其进行修补会重定向答案，谓词结果翻转率接近 1.0，映射翻转率接近 0.0，在 18 个单元中的 17 个满足了严格的预设隔离标准，并且这种定位在五个谓词家族中均成立。

The router shows the opposite profile. A learned subspace flips A and B near-perfectly within the trained pair yet transfers to a new pair at approximately 0 in every model, while in Gemma-3-4B (the only model probed cross-lingually) it transfers at approximately 0.98 to the same pair in other languages.

路由器的表现则截然相反。一个学习到的子空间在训练对内几乎完美地翻转了 A 和 B，但在每个模型中迁移到新对时效果几乎为 0；而在 Gemma-3-4B（唯一进行跨语言探测的模型）中，它在其他语言的相同对上迁移率约为 0.98。

Under every probe we ran, the router direction is token-bound and non-transferable (largely answer-readout in Gemma, pair-specific in Qwen) rather than an abstract routing module. Test is modular; under these probes, route is not.

在我们运行的每一次探测中，路由方向都是与 Token 绑定的且不可迁移的（在 Gemma 中主要是答案读取，在 Qwen 中是特定于对的），而不是一个抽象的路由模块。测试（Test）是模块化的；但在这些探测下，路由（Route）并非如此。