WorldCast: Exploring Intelligent Visual Generation in World State Prediction

查看原文 / View Original 编辑 / Edit

Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world.

人类通过解读世界的多模态观测结果来预测其演变方式。近期的多模态生成系统展现出了类似的能力：它们能够整合多模态背景，并合成描绘世界变化的视觉输出。

To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references.

为了刻画这一新兴趋势，我们将模型能力划分为四个层级，并发现尽管模型取得了进展，但更高级的任务仍相对缺乏探索。为了弥补这一差距，我们探讨了模型在多步世界状态预测方面的表现，这是一项高级任务，要求模型根据多模态观测和多样化的参考信息，预测连续的未来世界状态。

Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models.

此类任务要求模型具备交织的理解与生成能力，以及广泛的世界知识，从而将人类动作序列、物理变换和抽象视觉推理转化为连贯的视觉输出。我们推出了 WorldCast，这是一个包含八个类别、320 个手动设计案例的基准测试，并评估了 12 个代表性模型。

The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models.

实验得出了三个关键见解：(a) 前沿模型的进步正在迅速推动智能世界建模的边界，GPT-Image-2.5 系统得分 72.8，并在现实世界任务中展示了早期的视觉状态预测能力；(b) WorldCast 揭示了模型之间存在巨大差距：即使是 Nano Banana Pro 也仅达到 42.8 分，而最好的开源模型得分仅为 20.1，在处理复杂背景、交织参考和推理密集型任务方面存在不足；(c) 强大的生成模型有效地利用了其底层语言模型的能力。

These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.