SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model

查看原文 / View Original 编辑 / Edit

Computer Science > Computer Vision and Pattern Recognition arXiv:2611.05443 (cs) [Submitted on 3 Nov 2026] Title: SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model Authors: Saurabh Koju, Love Panta, Deepak Thapa, Rudra P. K. Poudel, Prajwal Ghimire, Angus Watson, Yash Raj Shrestha, Prashnna Kumar Gyawali, Binod Bhattarai.

计算机科学 > 计算机视觉与模式识别 arXiv:2611.05443 (cs) [提交于 2026 年 11 月 3 日] 标题：SurgTransfer：用于手术世界模型中动作迁移的多步潜在动作建模作者：Saurabh Koju, Love Panta, Deepak Thapa, Rudra P. K. Poudel, Prajwal Ghimire, Angus Watson, Yash Raj Shrestha, Prashnna Kumar Gyawali, Binod Bhattarai。

Abstract: Surgical world models have the potential to provide intra-operative guidance, accelerate risk-free training and facilitate transfer of expert surgical maneuvers. However, the use of world models for transferring surgical expertise has been largely unexplored. Moreover, recent approaches in this domain have mainly relied on ground truth kinematics actions or expert annotations for specific tasks, limiting their applicability beyond the narrow range of tasks with ground-truth annotations.

摘要：手术世界模型有潜力提供术中指导、加速无风险培训并促进专家手术操作的迁移。然而，利用世界模型来迁移手术专业知识的研究在很大程度上尚未被探索。此外，该领域近期的研究方法主要依赖于特定任务的地面真值运动学动作或专家标注，这限制了它们在具有地面真值标注的狭窄任务范围之外的适用性。

In contrast, existing works in the natural domain have shown promising results in learning latent actions solely from unlabeled videos and transferring the learned actions across contexts. We observe that this paradigm is short-sighted for the surgical domain, failing to capture the hierarchical and goal-directed structure of surgical procedures. Thus, we propose a novel multi-step action learning framework to learn context-invariant latent actions that are easily transferable across various surgical contexts.

相比之下，自然领域现有的研究成果表明，仅从无标签视频中学习潜在动作并在不同情境下迁移这些学习到的动作已取得了令人鼓舞的结果。我们观察到，这种范式对于手术领域而言是短视的，因为它无法捕捉手术过程的分层和目标导向结构。因此，我们提出了一种新颖的多步动作学习框架，旨在学习可在各种手术情境间轻松迁移的上下文无关潜在动作。

We perform extensive experiments on the publicly available datasets, SurgToolLoc-2022 and PitVis-2023, verifying our approach with both quantitative metrics and qualitative evaluation by surgical experts. We have published our code and the implementation details at this https URL.

我们在公开数据集 SurgToolLoc-2022 和 PitVis-2023 上进行了广泛的实验，通过定量指标和外科专家的定性评估验证了我们的方法。我们已将代码和实现细节发布在此 HTTPS 链接中。