MedFabric and EtHER: A Data-Centric Framework for Word-Level Fabrication Generation and Detection in Medical LLMs
MedFabric and EtHER: A Data-Centric Framework for Word-Level Fabrication Generation and Detection in Medical LLMs
MedFabric 与 EtHER:用于医疗大模型词级伪造生成与检测的以数据为中心的框架
Large Language Models exhibit strong reasoning and semantic understanding capabilities but often hallucinate in domains that require expert knowledge, among which fabrications, the generation of factually incorrect yet fluent statements, pose the greatest risk in medical contexts. 大型语言模型(LLM)展现出了强大的推理和语义理解能力,但在需要专业知识的领域中经常出现幻觉。其中,“伪造”(即生成事实错误但语言流畅的陈述)在医疗背景下构成了最大的风险。
Existing medical hallucination datasets inadequately capture fabrication phenomena due to limited fabrication coverage, stylistic disparities between human and LLM-authored texts, and distributional drift during hallucinated sample synthesis. 现有的医疗幻觉数据集由于伪造覆盖范围有限、人类与大模型文本之间的风格差异,以及在合成幻觉样本过程中的分布偏移,无法充分捕捉伪造现象。
To address this, we propose a data-centric pipeline to generate realistic and word-level fabrications that preserve syntactic and stylistic fidelity while introducing subtle factual deviations, resulting in MedFabric. 为了解决这一问题,我们提出了一种以数据为中心的流水线,旨在生成逼真的词级伪造内容。该方法在引入细微事实偏差的同时,保持了句法和风格的忠实度,最终构建了 MedFabric 数据集。
Building upon this dataset, we introduce ETHER, a modular word-level fabrication detector integrating Text2Table Decomposition, Word Masking and Filling and Hybrid Sentence Pair Evaluation to enhance factual alignment. 基于该数据集,我们引入了 ETHER,这是一个模块化的词级伪造检测器。它集成了“文本转表格分解”(Text2Table Decomposition)、“词掩码与填充”(Word Masking and Filling)以及“混合句子对评估”(Hybrid Sentence Pair Evaluation),以增强事实对齐能力。
Empirical results demonstrate that MedFabric outperforms state-of-the-art detectors by over 15% on word-level fabrication benchmarks while maintaining consistent performance across structural similarities, offering a comprehensive framework for reliable and domain-specific factuality detection. 实证结果表明,MedFabric 在词级伪造基准测试中表现优于当前最先进的检测器,提升幅度超过 15%,同时在结构相似性方面保持了稳健的性能,为可靠且特定领域的真实性检测提供了一个全面的框架。