3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language Models

Computer Science > Computation and Language arXiv:2610.36178 (cs) [Submitted on 26 Oct 2026] Title: 3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language Models Authors: Samar Ansari.

计算机科学 > 计算与语言 arXiv:2610.36178 (cs) [提交于 2026 年 10 月 26 日] 标题：3, 7, 14, 17, 42：指令微调在语言模型中安装并强化了数字选择启发式算法 作者：Samar Ansari。

Abstract: We measure, without sampling error, the probability a language model assigns to each integer from 1 to N when asked for a random number. For a fixed prompt the model defines one probability vector, which we compute by summing over each integer’s scored spellings rather than estimate by sampling, removing the noise that limits earlier studies [1, 4].

摘要：我们测量了语言模型在被要求提供随机数时，分配给从 1 到 N 每个整数的概率，且不存在采样误差。对于固定的提示词，模型定义了一个概率向量，我们通过对每个整数的评分拼写求和来计算该向量，而不是通过采样进行估计，从而消除了限制早期研究的噪声 [1, 4]。

We evaluate 18 open-weight model lines at five ranges up to N = 100, under 29 decoding configurations, on two inference stacks; 15 of them have matched base and instruction-tuned checkpoints that both give usable answers. Instruction tuning makes the distribution less uniform in all 15 pairs. It installs one heuristic, round-number avoidance, and sharpens another, endpoint avoidance.

我们在两个推理栈上，在最高 N = 100 的五个范围内，对 18 个开放权重模型系列进行了评估，共使用了 29 种解码配置；其中 15 个模型拥有匹配的基础检查点和指令微调检查点，且两者都能给出可用的答案。在所有 15 对模型中，指令微调都使分布变得不再那么均匀。它安装了一种启发式算法——规避圆整数字，并强化了另一种——规避端点数字。

Base checkpoints do not avoid round numbers once the endpoints are removed from those sets; tuned checkpoints avoid them, multiples of 10 almost completely. Base checkpoints return the endpoints 1 and N at 0.30 to 0.60 of the interior rate; tuned checkpoints almost never return them.

一旦从集合中移除端点，基础检查点就不会规避圆整数字；而微调后的检查点则会规避它们，几乎完全规避了 10 的倍数。基础检查点返回端点 1 和 N 的频率是内部数字频率的 0.30 到 0.60 倍；而微调后的检查点几乎从不返回这些端点。

A preference for odd numbers is consistent across tuned checkpoints only at N = 100, where base checkpoints also show it weakly. Endpoint avoidance and the odd preference fit the account Kubovy and Psotka [6] proposed for human number choice; round-number avoidance is not part of that account, and the midpoint avoidance it predicts is essentially absent, with one slight exception at N = 100.

仅在 N = 100 时，微调后的检查点对奇数的偏好才表现出一致性，而基础检查点在此处也表现出微弱的偏好。规避端点和偏好奇数符合 Kubovy 和 Psotka [6] 为人类数字选择提出的解释；规避圆整数字并不属于该解释的一部分，且其预测的规避中点现象基本不存在，仅在 N = 100 时有一个微小的例外。

Tuning does not install a new favourite integer shared across models: above N = 10 no modal integer is shared by more than 42 percent of tuned conditions, and P (17) rises in 10 of 15 pairs and falls in 5. The concentration is present in the model’s own distribution: at temperature 1.0, nucleus truncation at top-p 0.95 adds a median of 0.035 to the total variation distance.

微调并没有在模型间安装一个共同的新偏好整数：在 N > 10 时，没有哪个众数整数被超过 42% 的微调条件所共享，且 P(17) 在 15 对中的 10 对里上升，在 5 对里下降。这种集中度存在于模型自身的分布中：在温度为 1.0 时，top-p 为 0.95 的核截断使总变差距离增加了 0.035 的中位数。