Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
Computer Science > Artificial Intelligence arXiv:2610.28698 (cs) [Submitted on 20 Oct 2026] Title: Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds Authors: Kuan Zhang, Yukun Chen, Zhihao Yang, Yue Su, Xiangnan Wu, Zirong Chen, Run Luo, Jinkun Hou, Tao Tan, Yinhe Zheng, Yiming Li.
计算机科学 > 人工智能 arXiv:2610.28698 (cs) [提交于 2026 年 10 月 20 日] 标题：游戏中的规模化：多样化虚拟世界中具身智能体的持续预训练。作者：Kuan Zhang, Yukun Chen, Zhihao Yang, Yue Su, Xiangnan Wu, Zirong Chen, Run Luo, Jinkun Hou, Tao Tan, Yinhe Zheng, Yiming Li。
Abstract: Scaling data, diversity and model capacity has improved embodied learning in physical environments, where diverse experiences remain grounded in shared physical laws. Whether these gains extend to virtual worlds with different visual styles, dynamics, controls, and objectives remains unclear. We investigate this question through a controlled study of continued pre-training (CPT) across three axes: data volume, diversity, and model size.
摘要：扩展数据、多样性和模型容量改善了物理环境中的具身学习，在这些环境中，不同的体验仍然基于共同的物理定律。这些收益是否能扩展到具有不同视觉风格、动力学、控制方式和目标的虚拟世界中，目前尚不清楚。我们通过对数据量、多样性和模型大小这三个维度的持续预训练（CPT）进行对照研究，探讨了这一问题。
Our 120 training configurations span four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five mixtures ranging from Minecraft-only data to diverse multi-world corpora. We assess performance and generalization using three-level validation losses and gameplay benchmarks from in-corpus worlds to unseen worlds.
我们的 120 种训练配置涵盖了从 0.8B 到 27B 的四种模型规模、从 50 到 1,600 小时的六种数据预算，以及从仅限 Minecraft 的数据到多样化多世界语料库的五种混合方式。我们使用三级验证损失以及从语料库内世界到未见世界的游戏基准来评估性能和泛化能力。
Our results show: (1) more data improves in-corpus performance, diversity improves transfer to related worlds with limited gains in distant worlds, and model size improves data efficiency but generalizes worse to distant worlds under single-world training; (2) we observe an early interface phase with broad cross-world benefits and a later world phase with gains narrowed to in-corpus and related worlds, and fit an empirical scaling law relating validation loss to data volume, diversity and model size; and (3) our models are competitive with Minecraft specialists and frontier models in 3D worlds, and perform best in real-time settings.
我们的结果表明：（1）更多的数据提高了语料库内的性能，多样性改善了向相关世界的迁移，但在遥远世界中的收益有限；模型规模提高了数据效率，但在单世界训练下对遥远世界的泛化能力较差；（2）我们观察到一个具有广泛跨世界收益的早期接口阶段，以及一个收益缩小到语料库内和相关世界的后期世界阶段，并拟合了一个将验证损失与数据量、多样性和模型大小相关联的经验缩放定律；（3）我们的模型在 3D 世界中与 Minecraft 专家模型和前沿模型具有竞争力，并且在实时设置中表现最佳。
However, the limits of imitation and persistent gaps in distant worlds suggest that building generalist embodied agents requires richer training stages beyond continued pre-training.
然而，模仿的局限性和在遥远世界中持续存在的差距表明，构建通用具身智能体需要超越持续预训练的更丰富的训练阶段。