The Clinician’s Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing

临床医生的否决权：在自主 AI 处方中驾驭信任、责任与不确定性

Abstract: Autonomous AI systems are transitioning from advisory to autonomous roles for medication prescriptions. Recent United States bill H.R. 238 and Utah’s prescription-renewal pilot both authorize AI to prescribe medications in an agentic capacity. While some regulatory guidelines suggest aggregate model performance metrics for clearance, they do not require i) calibrated per-prediction confidence for action-gated thresholds, ii) differentiated communication of uncertainty arising from model ignorance (epistemic) versus genuine clinical ambiguity (aleatoric), and iii) inferential transparency at the moment of decision that allows for liability allocation.

摘要： 自主 AI 系统正在从药物处方的咨询角色向自主角色转变。美国最近的 H.R. 238 法案和犹他州的处方续签试点项目均授权 AI 以代理身份开具处方。尽管一些监管指南建议使用总体模型性能指标作为审批依据，但它们并未要求：i) 为行动门控阈值提供校准后的单次预测置信度；ii) 区分由模型无知（认知不确定性）与真实临床模糊性（偶然不确定性）引起的不确定性；以及 iii) 在决策时刻提供允许进行责任分配的推理透明度。

Here, we present a regulatory and technical argument (tested with a survey of 136 U.S. prescribing clinicians) positioning these as minimum architectural requirements for safe autonomous prescribing. Our results suggest prescribing clinicians i) would not permit autonomous prescribing without a calibrated confidence-based escalation mechanism, ii) preferred a competing-options summary when uncertainty was aleatoric but shifted to abstention when uncertainty was epistemic, and iii) were only willing to accept additional liability when inferential transparency enabled a substantive judgment under acknowledged uncertainty.

在此，我们提出了一项监管和技术论证（通过对 136 名美国处方临床医生的调查进行了测试），将上述要求定位为安全自主处方的最低架构要求。我们的研究结果表明，处方临床医生：i) 如果没有基于置信度的校准升级机制，将不会允许自主处方；ii) 在存在偶然不确定性时倾向于查看竞争选项摘要，但在存在认知不确定性时则倾向于弃权；iii) 只有在推理透明度使其能够在已知不确定性的情况下做出实质性判断时，才愿意承担额外责任。

These findings indicate our recommended architectural features would encourage higher rates of clinician adoption, largely through collapsing much of what “autonomy” conventionally means. A system meeting these requirements would function less as an autonomous agent, and more as a heavily supervised decision-support tool. As legislation and state pilots proceed, our technical argument backed by clinician perspectives provides opportunities for regulation to constrain the degree of autonomy ethically granted to AI in prescribing while aligning liability with the institutional actors who control system design and deployment.

这些发现表明，我们推荐的架构特征将通过大幅削弱“自主性”的传统定义，从而提高临床医生的采用率。一个满足这些要求的系统，其功能将不再像一个自主代理，而更像是一个受到严格监管的决策支持工具。随着立法和州级试点项目的推进，我们由临床医生观点支持的技术论证为监管提供了契机，即在限制 AI 处方中被赋予的伦理自主权的同时，将责任与控制系统设计和部署的机构主体相挂钩。