Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases

动态决策学习：用于罕见病异常定位的测试时演化方法

Abstract: Clinical abnormality grounding for rare diseases is often hindered by data scarcity, making supervised fine-tuning impractical and single-pass inference highly unstable. We propose Dynamic Decision Learning (DDL), a framework that enables frozen large vision-language models (LVLMs) to refine their decisions across both language and visual spaces by optimizing instructions and consolidating predictions under visual perturbations.

摘要： 罕见病的临床异常定位往往受限于数据稀缺，导致监督微调难以实施，且单次推理过程极不稳定。我们提出了动态决策学习（Dynamic Decision Learning, DDL）框架，该框架使冻结的大型视觉-语言模型（LVLMs）能够通过优化指令并在视觉扰动下整合预测结果，从而在语言和视觉空间中不断优化其决策。

This process improves localization quality and produces a consensus-based reliability score that quantifies model confidence. Results on brain imaging benchmarks, including a rare-disease dataset with 281 pathology types across models ranging from 3B to 72B parameters, show that DDL improves mAP@75 by up to 105% on rare-disease cases and outperforms adaptation baselines and supervised fine-tuning.

这一过程提升了定位质量，并生成了一个基于共识的可靠性分数，用于量化模型的置信度。在脑成像基准测试（包括包含 281 种病理类型的罕见病数据集）上的实验结果表明，在参数规模从 3B 到 72B 的模型中，DDL 在罕见病案例上的 mAP@75 指标提升高达 105%，且表现优于现有的适应性基准方法和监督微调方法。

Furthermore, DDL demonstrates stronger calibration between reliability scores and localization accuracy under severe distribution shifts and increasing task difficulty. Code is available at: this https URL.

此外，DDL 在面对严重的分布偏移和不断增加的任务难度时，在可靠性分数与定位准确度之间展现出了更强的校准能力。代码链接：点击此处访问。