Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives

实体追踪能力在十亿参数以下的语言模型中涌现，并在自然叙事中超越人类表现

Abstract: Understanding language requires tracking entities across discourse - i.e., knowing where things are and how they change, even when not explicitly stated. Whether language models perform such tracking in a human-like fashion remains unclear, in part because existing evaluations rely on artificial tasks, far removed from natural language comprehension, and lack comparisons to humans.

摘要： 理解语言需要跨语篇追踪实体——即了解事物的位置及其变化，即使文中并未明确说明。语言模型是否以类似人类的方式进行这种追踪尚不明确，部分原因是现有的评估依赖于远离自然语言理解的人工任务，且缺乏与人类表现的对比。

Here, we evaluate entity tracking in both language models and humans (N = 48) using naturalistic narratives at multiple levels of complexity. In humans, we find that entity tracking degrades specifically with narrative complexity, not narrative length.

在此，我们使用不同复杂程度的自然叙事，对语言模型和人类（N = 48）的实体追踪能力进行了评估。在人类受试者中，我们发现实体追踪能力的下降主要与叙事的复杂性有关，而非叙事的长度。

In language models, we find that human-level entity tracking is already present at 410 million parameters - well below the multi-billion parameter, code-specialised models identified by prior work - and improves with scale, with contemporary models far exceeding human performance.

在语言模型方面，我们发现参数量仅为 4.1 亿的模型就已经具备了人类水平的实体追踪能力——这远低于先前研究中提到的数十亿参数、针对代码优化的模型——并且该能力随模型规模的扩大而提升，当代模型在这一指标上已远超人类表现。

Together, these results demonstrate that entity tracking, a core component of language understanding, emerges at model scales far smaller than previously thought.

综上所述，这些结果表明，作为语言理解核心组成部分的实体追踪能力，在远比此前预期更小的模型规模下即可涌现。

---

Paper Details:

• Authors: Karolina Drożdż, Micha Heilbron
• Submission Date: 4 Jun 2026
• Subject: Computation and Language (cs.CL)
• DOI: 10.48550/arXiv.2608.18083

论文详情：

• 作者： Karolina Drożdż, Micha Heilbron
• 提交日期： 2026 年 6 月 4 日
• 学科： 计算与语言 (cs.CL)
• DOI: 10.48550/arXiv.2608.18083