Frozen in Time: How a Physics Foundation Model Forgets Under Sim-to-Real Adaptation

Computer Science > Machine Learning arXiv:2610.40857 (cs) [Submitted on 29 Oct 2026] Title: Frozen in Time: How a Physics Foundation Model Forgets Under Sim-to-Real Adaptation Authors: Haaris Mian, Miles Cranmer, Alessandro Favero, Payel Mukhopadhyay.

计算机科学 > 机器学习 arXiv:2610.40857 (cs) [提交于 2026 年 10 月 29 日] 标题：冻结在时间里：物理基础模型在模拟到现实适应过程中的遗忘机制 作者：Haaris Mian, Miles Cranmer, Alessandro Favero, Payel Mukhopadhyay。

Abstract: Scientific foundation models (SciFMs) have achieved remarkable performance on a wide range of PDEs, including the ability to efficiently adapt to unseen data distributions with additional finetuning. As with all neural architectures, finetuning such models comes with the risk of catastrophic forgetting, whereby adapting to new tasks degrades prior knowledge of the model.

摘要：科学基础模型（SciFMs）在广泛的偏微分方程（PDEs）上取得了显著表现，包括通过额外微调高效适应未见数据分布的能力。与所有神经架构一样，对此类模型进行微调伴随着灾难性遗忘的风险，即适应新任务会削弱模型原有的知识。

We study catastrophic forgetting in Walrus-1.3B, a SciFM for continuum dynamics, over the course of a two-stage sim-to-real adaptation: first finetuning on simulated flow around a cylinder and then adapting to experimental data of the same system. We find that finetuning on the simulations rapidly degrades rollout performance on the original pretraining datasets, with one-step errors increasing up to 35x after just 20 epochs.

我们研究了连续介质动力学科学基础模型 Walrus-1.3B 在两阶段“模拟到现实”（sim-to-real）适应过程中的灾难性遗忘：首先在圆柱绕流模拟数据上进行微调，随后适应同一系统的实验数据。我们发现，在模拟数据上进行微调会迅速降低模型在原始预训练数据集上的展开性能，仅 20 个 epoch 后，单步误差就增加了高达 35 倍。

Subsequent adaptation to real data further erodes performance on the simulated task. We characterize the potential drivers of forgetting, such as dataset dependence and task representation similarity. We then show that replaying as little as 1% simulated data during real-data finetuning recovers most of the lost simulation performance while retaining effective adaptation to the experimental dynamics.

随后的现实数据适应进一步侵蚀了模型在模拟任务上的性能。我们刻画了导致遗忘的潜在驱动因素，例如数据集依赖性和任务表示相似性。随后我们证明，在现实数据微调过程中重放仅 1% 的模拟数据，即可恢复大部分丢失的模拟性能，同时保持对实验动力学的有效适应。

Our results identify catastrophic forgetting as a practical obstacle to sim-to-real adaptation of SciFMs and show that lightweight replay strategies significantly improve retention.

我们的研究结果指出，灾难性遗忘是科学基础模型在“模拟到现实”适应过程中的一个实际障碍，并表明轻量级的重放策略能显著改善知识保留。