Can a Model Catch Its Own Hallucinations for Free?: Label-Free Doubt Signals Hold Their Own Against a Labelled Dataset for Abstention

模型能否免费捕捉自身的幻觉？：无标签的怀疑信号在弃权任务中表现不输于有标签数据集

---

Large language models state false facts as fluently as true ones, yet a model often “knows” internally when it is on shaky ground: the probability it assigns to its own answer tends to dip on the facts it gets wrong.

大型语言模型陈述错误事实时与陈述正确事实一样流畅，但模型在内部往往“知道”自己何时处于不确定的境地：它为自身答案分配的概率往往会在其出错的事实上出现下降。

The usual way to act on this, teaching a model to abstain rather than guess, requires a labelled dataset of right and wrong answers. We ask whether the model’s own confidence, which is free and needs no labels, can do that job instead.

通常处理这种情况的方法是教导模型选择“弃权”而非盲目猜测，但这需要一个包含正确和错误答案的标注数据集。我们探讨了模型自身的置信度（这是免费且无需标签的）是否可以替代这一工作。

We fine-tune each model (with LoRA) to answer when its frozen confidence is high and to say “I’m not sure” when it is low, using the signal alone and no correctness labels.

我们使用 LoRA 对每个模型进行微调，使其在冻结的置信度较高时回答问题，而在置信度较低时回答“我不确定”，整个过程仅使用该信号，无需任何正确性标签。

Across six open-weights models (1B-8B, two families) on short-form factual question answering, with correctness adjudicated by an independent judge model, this label-free recipe holds its own against label-supervised abstention-tuning: at matched coverage we find no statistically detectable difference between the two.

在针对六个开源权重模型（1B-8B，两个系列）进行的简短事实问答测试中，通过独立的评判模型来裁定正确性，这种无标签方案在表现上丝毫不逊色于有标签监督的弃权微调：在覆盖率匹配的情况下，我们发现两者之间没有统计学上的显著差异。

A control that drills hard examples instead of abstaining does not help, indicating the gain comes from calibration, not rote memorization. The signal’s one blind spot is confidently wrong facts, which it cannot flag.

一项针对困难样本进行强化训练而非弃权的对照实验显示并无帮助，这表明性能提升来自于校准，而非死记硬背。该信号唯一的盲点是那些“自信地错误”的事实，这是它无法标记出来的。

A model’s own doubt is thus a near-free substitute for a labelled dataset when teaching it when to abstain. Code and artifacts are available on request.

因此，当教导模型何时弃权时，模型自身的怀疑信号可以作为标注数据集的一种近乎免费的替代方案。代码和相关产物可按需提供。