Human-in-the-Loop Large Language Model Framework for Identification of Cutaneous Immune-Related Adverse Events
Human-in-the-Loop Large Language Model Framework for Identification of Cutaneous Immune-Related Adverse Events
用于识别皮肤免疫相关不良事件的“人在回路”大语言模型框架
Abstract: This study evaluated a retrieval-augmented, multi-agent large language model (LLM)-driven, human-in-the-loop framework for detecting cutaneous immune-related adverse events (cirAEs) from clinical notes. 摘要： 本研究评估了一种基于检索增强、多智能体大语言模型（LLM）驱动的“人在回路”（human-in-the-loop）框架，旨在从临床记录中检测皮肤免疫相关不良事件（cirAEs）。
Compared with unassisted manual review, the LLM-assisted workflow improved accuracy (F1 = 0.88 vs 0.77), inter-rater agreement measured by Cohen’s kappa (kappa = 0.82 vs 0.50), and reduced average review time by approximately half. 与人工独立审查相比，LLM 辅助的工作流程提高了准确性（F1 分数从 0.77 提升至 0.88），改善了以 Cohen’s kappa 衡量的评估者间一致性（kappa 值从 0.50 提升至 0.82），并将平均审查时间缩短了约一半。
This framework pilots how LLMs can be applied to identify immune-related toxicities across organ systems and, more broadly, enable accurate, scalable, and transparent adverse event data extraction. 该框架为如何应用大语言模型识别各器官系统的免疫相关毒性提供了试点，并更广泛地实现了准确、可扩展且透明的不良事件数据提取。
Paper Details:
- Authors: Charles Lu, Olivia Burke, Debby Cheng, Adam Kashlan, Caitlyn Duffy, Zeyun Lu, Lirit Fuksman, Jin Ning Tian, Andrew Sedlack, Priya Katyal, Eudora Lee, Ralina Karagenova, Chuck Lin, Kun-Hsing Yu, Nicole LeBoeuf, Alexander Gusev, Yevgeniy R. Semenov
- arXiv ID: 2607.20428
- Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Multiagent Systems (cs.MA)
- Submission Date: 9 May 2026
论文详情：
- 作者： Charles Lu, Olivia Burke, Debby Cheng, Adam Kashlan, Caitlyn Duffy, Zeyun Lu, Lirit Fuksman, Jin Ning Tian, Andrew Sedlack, Priya Katyal, Eudora Lee, Ralina Karagenova, Chuck Lin, Kun-Hsing Yu, Nicole LeBoeuf, Alexander Gusev, Yevgeniy R. Semenov
- arXiv 编号： 2607.20428
- 学科分类： 计算与语言 (cs.CL)；人机交互 (cs.HC)；多智能体系统 (cs.MA)
- 提交日期： 2026年5月9日