What happens when an LLM never sees material beyond fifth grade?

当大语言模型从未接触过五年级以上的知识，会发生什么？

Talk to LittleLearner The hosted 5B model, live in your browser. Open in a new tab ↗ if the chat doesn’t load below. A controlled sandbox for studying how models acquire knowledge. 与 LittleLearner 对话：在浏览器中直接体验托管的 5B 参数模型。如果下方聊天窗口未加载，请在新标签页中打开。这是一个用于研究模型如何获取知识的受控沙盒。

Modern LMs are trained on everything at once, so it is hard to tell whether a new skill was learned or merely elicited. We constrain the training distribution itself: an 88B-token corpus filtered to the U.S. elementary-school curriculum, with models trained from scratch on it and matched unfiltered controls. 现代语言模型通常在海量数据上进行训练，因此很难区分一项新技能究竟是“习得”的，还是仅仅被“激发”出来的。我们对训练分布本身进行了限制：构建了一个包含 880 亿 token 的语料库，并根据美国小学课程标准进行了过滤，随后在此基础上从零训练模型，并配备了未经筛选的对照组模型。

Dataset: LittleCurriculum

数据集：LittleCurriculum

An 88B-token corpus distilled from FineWeb-Edu through a five-stage filtering pipeline aligned with Common Core standards (K–5). Concepts, facts, and vocabulary taught above Grade 5 are explicitly excluded. 这是一个从 FineWeb-Edu 中提炼出的 880 亿 token 语料库，经过了与美国共同核心州立标准（K–5 年级）对齐的五阶段过滤流程。五年级以上教授的概念、事实和词汇均被明确排除在外。

Models: LittleLearner

模型：LittleLearner

Three scales (0.6B / 1.3B / 5B) trained from scratch on LittleCurriculum: chattable models with an interpretable knowledge boundary. Each ships with a matched Unfiltered control for clean comparison. 三种规模（0.6B / 1.3B / 5B）的模型均在 LittleCurriculum 上从零训练：这些模型具备可交互的聊天能力，且拥有清晰可解释的知识边界。每个模型都配有一个未经筛选的对照组，以便进行纯净的对比分析。

Findings: Elicitation, not acquisition

研究发现：是“激发”，而非“习得”

In our experiments, scaling, SFT+GRPO post-training, and in-context learning amplify what the curriculum taught, but none meaningfully improves out-of-scope performance, indicating that the pretraining filter sets the effective capability ceiling. 在我们的实验中，模型缩放（Scaling）、SFT+GRPO 后训练以及上下文学习（In-context learning）都增强了模型在课程范围内的能力，但没有一项能显著提升范围外的表现。这表明预训练阶段的过滤设置了模型能力的有效上限。

Model checkpoints

模型检查点

LittleLearner at three scales (0.6B / 1.3B / 5B), each with a matched Unfiltered control sharing its architecture, tokens, and recipe. LittleLearner 提供三种规模（0.6B / 1.3B / 5B），每个模型都有一个共享架构、token 和训练配方的未经筛选对照组。

Base: the pretrained model. Base（基础版）： 预训练模型。
GRPO: math specialists post-trained on MathCAMPS; responses may exhibit a tendency toward math-oriented output. GRPO： 在 MathCAMPS 上进行后训练的数学专家模型；其回答可能表现出偏向数学输出的倾向。
Chatty: variants tuned for general chat behavior. Chatty（聊天版）： 针对通用聊天行为进行微调的变体。

Capability stays inside the curriculum

能力被限制在课程范围内

Can standard interventions push a model past what its pretraining data taught it? With the boundary under experimental control, we can ask cleanly. In our experiments, each intervention amplifies in-scope ability; none of them meaningfully improves out-of-scope performance. 常规干预手段能否推动模型超越其预训练数据所教授的范围？由于边界处于实验控制之下，我们可以清晰地回答这个问题。在我们的实验中，每种干预手段都增强了范围内的能力，但没有一种能显著改善范围外的表现。

Scaling: Scaling model size improves performance within the model’s controlled knowledge exposure and extends modestly to problems along the same learning trajectory, but yields little improvement on problems requiring more advanced capabilities outside the exposure. 缩放（Scaling）： 增加模型规模可以提升其在受控知识范围内的表现，并能适度扩展到同一学习轨迹上的问题，但对于需要超出该范围的高级能力的问题，几乎没有提升。
Post-training: Post-training through GRPO significantly boosts in-scope K–5 capabilities, but fails to recover out-of-scope beyond-K–5 capabilities, even when training with out-of-scope data. 后训练（Post-training）： 通过 GRPO 进行的后训练显著提升了 K–5 范围内的能力，但即便使用范围外的数据进行训练，也无法恢复 K–5 范围之外的能力。
In-context learning: In-context learning with the prompts we test does not unlock new reasoning capabilities in beyond-K–5 for our trained 5B LittleLearner. 上下文学习（In-context learning）： 在我们测试的提示词下，上下文学习并未能为我们训练的 5B LittleLearner 解锁 K–5 范围之外的推理能力。

What will you teach it?

你想教它什么？

Because LittleLearner’s training exposure is explicitly specified, behavioral and representational changes can be related directly to the concepts you introduce. Three directions we’re excited about: 由于 LittleLearner 的训练内容是明确指定的，因此行为和表征的变化可以直接与你引入的概念相关联。我们对以下三个方向感到兴奋：

RL & discovery: Can RL create capability? The prior is restricted to K–5, so capabilities that emerge under RL can be attributed to the RL process itself. A tractable proxy for reward-driven discovery. 强化学习与发现： 强化学习能创造能力吗？由于先验知识被限制在 K–5，因此在强化学习下涌现的能力可以归因于强化学习过程本身。这是奖励驱动发现的一个易于处理的代理研究。
Continual learning: Watch a concept being learned. Introduce negative numbers and measure sample efficiency, retention, and interference. Or probe behavior near the boundary: does it answer, abstain, or hallucinate? 持续学习： 观察概念的学习过程。引入负数并测量样本效率、保留率和干扰。或者探测边界附近的行为：它是回答、拒绝还是产生幻觉？
Educational science: Machine vs. child learners. Specified exposure enables controlled human-model comparison. Do models and children need similar exposure to learn fractions, or make similar errors on word problems? 教育科学： 机器与儿童学习者。明确的知识暴露使得受控的人机对比成为可能。模型和儿童在学习分数时是否需要相似的接触量？或者在应用题上是否会犯类似的错误？

＋ Your turn: Bring your own question

＋轮到你了：提出你的问题

A known boundary turns your idea into a clean experiment! 一个已知的边界可以将你的想法转化为一个纯净的实验！