Saga: an in silico ontology for a multimodal patient world model

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.33054 (cs) [Submitted on 19 Oct 2026] Title: Saga: an in silico ontology for a multimodal patient world model Authors: Youcheng Li (Peking University, Isoplex Lab), Fa Tian (Isoplex Lab), Binbin Su (Peking Union Medical College, Isoplex Lab).

计算机科学 > 计算机视觉与模式识别 arXiv:2610.33054 (cs) [2026年10月19日提交] 标题：Saga：一种用于多模态患者世界模型的计算机模拟本体论 作者：Youcheng Li（北京大学，Isoplex实验室）、Fa Tian（Isoplex实验室）、Binbin Su（北京协和医学院，Isoplex实验室）。

Abstract: Simulating an individual’s disease course requires connecting patient history to future events, clinical timing and disease morphology. Longitudinal prediction and multimodal representations exploit this information, but organizing future readouts around a common patient state and advancing trajectories without real intermediate observations remain central challenges.

摘要：模拟个人的疾病进程需要将患者病史与未来事件、临床时间安排和疾病形态联系起来。纵向预测和多模态表征利用了这些信息，但在缺乏真实中间观测的情况下，围绕共同的患者状态组织未来读数并推进轨迹仍然是核心挑战。

Here we introduce Saga, a multimodal patient world model that learns a shared state from patient histories to predict clinical events and generate multimodal images. Generated events and times are appended to the history and re-encoded to advance prediction.

在此，我们介绍了Saga，这是一种多模态患者世界模型，它通过学习患者病史中的共享状态来预测临床事件并生成多模态图像。生成的事件和时间被附加到病史中并重新编码，以推进预测。

We studied 461,370 patients with 12 cancer types across 112 centres. Modality-specific paired evaluations linked future images to observed follow-up. With historical images held fixed, removing patient conditioning reduced event F1 from 0.716 to 0.591 and increased lesion-burden error from 11.8% to 21.3%.

我们研究了来自112个中心的461,370名患有12种癌症的患者。特定模态的配对评估将未来图像与观察到的随访结果联系起来。在保持历史图像不变的情况下，移除患者条件限制导致事件F1值从0.716下降至0.591，病灶负荷误差从11.8%增加至21.3%。

In free rollouts, eight-step event F1 was 0.672, compared with 0.553 for a longitudinal Transformer evaluated from the same origins, and timing error was lower by 8.7 days. In the internal test set, one-year progression AUROC was 0.882, with values of 0.871 and 0.856 in temporal and geographical external evaluations.

在自由展开测试中，八步事件F1为0.672，而从相同起点评估的纵向Transformer为0.553，且时间误差降低了8.7天。在内部测试集中，一年进展AUROC为0.882，在时间和地理外部评估中分别为0.871和0.856。

These results show that shared patient states connect future phenotypes across observational scales with event-driven trajectory progression, sustaining comparative predictive advantages during rollout and discrimination in external clinical evaluation. Saga makes patient-level disease simulation a multimodal prediction task that can be tested against observed follow-up.

这些结果表明，共享的患者状态通过事件驱动的轨迹演进，在观测尺度上连接了未来的表型，在展开过程中保持了比较预测优势，并在外部临床评估中保持了辨别力。Saga将患者层面的疾病模拟转化为一项可以通过观察到的随访结果进行测试的多模态预测任务。