Towards a Deterministic Math Solver for Clinical Language Models

面向临床语言模型的确定性数学求解器

Abstract: Large language models are unreliable at arithmetic, which is a problem for clinical calculators where a single numerical error changes the recommendation. The standard response is to hardcode each calculator as a validated function, one at a time. We test an alternative: the model does not calculate. Instead, it writes case-specific Python that a restricted local executor runs as a deterministic solver, and the model’s task reduces to deciding how to use it.

摘要： 大型语言模型在算术运算方面表现不可靠，这对临床计算器而言是一个严重问题，因为单一的数值错误就可能改变临床建议。目前的标准应对方案是为每个计算器编写经过验证的硬编码函数。我们测试了一种替代方案：模型本身不进行计算，而是编写针对特定病例的 Python 代码，由受限的本地执行器作为确定性求解器运行，模型的任务简化为决定如何使用该求解器。

We evaluate this Program-Solve interface on MedCalc-Bench Verified (1,100 cases, 55 calculators) against direct model arithmetic and a hand-written 22-calculator library, using Qwen2.5-7B and Qwen2.5-32B-AWQ, after auditing the benchmark’s formulas against current clinical guidelines and flagging 16 of 55 with version, use or coefficient concerns.

我们在 MedCalc-Bench Verified（包含 1,100 个病例和 55 个计算器）上评估了这种“程序-求解”（Program-Solve）接口，并将其与模型直接算术运算以及一个包含 22 个计算器的手工编写库进行了对比。实验使用了 Qwen2.5-7B 和 Qwen2.5-32B-AWQ 模型，并在评估前根据当前临床指南对基准测试中的公式进行了审计，标记了 55 个公式中存在版本、用途或系数问题的 16 个公式。

With formulas and gold variables supplied and both routes reading the whole note, handing off to the solver is not a reliable advantage at 7B (75.31% against 72.02%, a paired +3.29 points with a 95% calculator-cluster interval of [-3.49, 10.38]) but is one at 32B (90.53% against 83.47%, +7.05 [0.47, 14.60], clear of zero).

在提供公式和标准变量且两种路径均读取完整病历的情况下，对于 7B 模型，交给求解器处理并没有带来可靠的优势（准确率为 75.31% 对比 72.02%，配对提升 3.29 个百分点，95% 计算器聚类区间为 [-3.49, 10.38]）；但对于 32B 模型，这种方法具有明显优势（90.53% 对比 83.47%，提升 7.05 个百分点 [0.47, 14.60]，区间不包含零）。

The hand-written library is exact on its 440 supported cases but abstains elsewhere (40.0% overall). Adding an executor thus helps some open-weight models more than others even under matched formula, variable and note access, and is not a substitute for verified formulas or reliable variable extraction either way.

手工编写的库在其支持的 440 个病例上表现精确，但在其他情况下则无法处理（总体覆盖率为 40.0%）。因此，即使在公式、变量和病历访问权限相同的情况下，引入执行器对某些开源权重模型的帮助也大于其他模型，且无论如何，这都不能替代经过验证的公式或可靠的变量提取。