Capability from Access Structure, Not Scale: Lower Bounds and Pre-Registered Tests for Hybrid Sequence Models

Capability from Access Structure, Not Scale: Lower Bounds and Pre-Registered Tests for Hybrid Sequence Models

能力源于访问结构而非规模:混合序列模型的下界与预注册测试

The Platonic Representation Hypothesis (PRH) holds that as models scale, representations of heterogeneous networks converge toward a shared model of reality. We propose its sequel and boundary, the Capability Convergence Hypothesis (CCH): under a fixed per-token inference budget, representational convergence does not entail capability convergence. “柏拉图表征假说”(Platonic Representation Hypothesis, PRH)认为,随着模型规模的扩大,异构网络的表征会趋向于一个共享的现实模型。我们提出了该假说的续篇及其边界——“能力收敛假说”(Capability Convergence Hypothesis, CCH):在固定的单 Token 推理预算下,表征的收敛并不意味着能力的收敛。

Capability instead converges toward a class, the access-complete hybrid: any architecture holding both a compressive O(1)-state channel and a scalable verbatim-index channel. We anchor it on a witness task, the Newton’s-apple problem in an infinite stream, and name three resource walls: a Shannon wall barring any o(Nb)-state architecture, a horizon wall barring any fixed window, and a circuit wall barring fixed-depth attention-only composition (conditional on TC0 != NC1). 能力反而趋向于一类特定的架构——“访问完备型混合架构”(access-complete hybrid):即任何同时具备压缩式 O(1) 状态通道和可扩展逐字索引通道的架构。我们通过一个见证任务——无限流中的“牛顿苹果问题”——将其锚定,并定义了三道资源壁垒:禁止任何 o(Nb) 状态架构的“香农壁垒”(Shannon wall)、禁止任何固定窗口的“视界壁垒”(horizon wall),以及在 TC0 != NC1 条件下禁止固定深度纯注意力组合的“电路壁垒”(circuit wall)。

Under an explicit separability assumption a hybrid crosses all three by paying each wall’s price, so capability is strictly super-additive under composition. We separate what we prove from what we conjecture: the access-completeness principle rests on information-theoretic lower bounds and pre-registered experiments, while the field-level convergence trend is an economics-motivated conjecture. 在明确的可分性假设下,混合架构通过支付每道壁垒的代价跨越了这三者,因此在组合下,能力呈现严格的超加性。我们将证明部分与猜想部分区分开来:访问完备性原则建立在信息论下界和预注册实验的基础上,而领域层面的收敛趋势则是一个基于经济学动机的猜想。

We report the first pre-registered small-scale tests under criteria frozen before the data: the predicted scissors gap is measured (exact-retrieval error 0.994 vs. 0.000 once a 64-scalar state gains one global-attention layer), the state-tracking bifurcation lands at the registered boundary, and a conjunction witness shows an irreducibly two-channel solution; one prediction failed with its direction reversed and is reported as such. Representational convergence is given freely by scale; capability convergence must be purchased by access structure. 我们报告了首批在数据获取前即锁定标准的预注册小规模测试结果:测量到了预期的“剪刀差”(当 64 标量状态增加一个全局注意力层后,精确检索误差从 0.994 降至 0.000),状态跟踪的分叉点落在了预注册的边界上,且合取见证显示了不可约的双通道解;其中一项预测失败,其方向与预期相反,我们对此进行了如实报告。表征的收敛由规模免费提供,而能力的收敛必须通过访问结构来购买。