Information Discernment in Large Language Models

大型语言模型中的信息辨别能力

Abstract: LLMs are increasingly used with external knowledge sources like the internet. Do they weigh information appropriately — updating more for reliable sources (source discernment) and more when claims bring priors closer to the truth (truth discernment)? We formalize this as information discernment and introduce Learn2Discern (L2D), an experimental framework and benchmark grounded in three normative axioms with interpretable metrics.

摘要： 大型语言模型（LLMs）正越来越多地与互联网等外部知识源结合使用。它们是否能适当地权衡信息——即针对可靠来源更新更多信息（来源辨别），以及在声明使先验知识更接近事实时更新更多信息（事实辨别）？我们将此形式化为“信息辨别”（Information Discernment），并引入了 Learn2Discern (L2D)，这是一个基于三个规范公理且具有可解释指标的实验框架与基准测试。

To establish external validity, a pre-registered, quota-matched user study (n=299) confirms that real LLM users endorse all three axioms and report that violations reduce their trust and usage intent.

为了确立外部有效性，一项预注册的、配额匹配的用户研究（n=299）证实，真实的大型语言模型用户认可这三个公理，并表示违背这些公理会降低他们的信任度和使用意愿。

Across 13 models and nearly 670K trials, we find consistent failures across both dimensions: models perform near chance on source and truth discernment, rely on source popularity twice as much as source reliability, and update roughly equally whether a claim improves or worsens their position relative to the ground truth.

通过对 13 个模型和近 67 万次试验的分析，我们发现模型在这两个维度上都存在持续的失败：模型在来源辨别和事实辨别上的表现接近随机水平；它们对来源流行度的依赖程度是来源可靠性的两倍；无论声明是改善还是恶化了其相对于事实的准确性，模型更新信息的幅度大致相同。

Models integrate external knowledge most effectively on datasets where their priors are already the most accurate. Newer and larger models improve truth discernment but not source discernment, a blind spot that model complexity does not address. We identify simple inference-time interventions that improve both forms of discernment. We release our dataset and survey as a testbed for a core alignment property that scales in importance as LLMs replace traditional search.

模型在自身先验知识已经最准确的数据集上，整合外部知识的效果最好。更新、更大的模型改善了事实辨别能力，但并未改善来源辨别能力，这是模型复杂性无法解决的盲点。我们确定了一些简单的推理时干预措施，可以改善这两种辨别能力。我们发布了数据集和调查问卷，作为一项核心对齐属性的测试平台，随着大型语言模型取代传统搜索，这一属性的重要性将日益凸显。