Confidence Calibration in Large Language Models
Confidence Calibration in Large Language Models
大型语言模型中的置信度校准
Abstract: We investigate the calibration of large language models’ (LLMs’) confidence across diverse tasks. The results of our preregistered study show that the current crop of LLMs are, like people, too sure they are right: confidence exceeds accuracy, on average. 摘要： 我们研究了大型语言模型（LLM）在不同任务中的置信度校准情况。我们预注册研究的结果表明，当前的一批大型语言模型与人类一样，往往过于自信：平均而言，它们的置信度超过了准确率。
Importantly, however, this tendency is moderated by a powerful hard-easy effect, wherein overconfidence is greatest on difficult tests; by contrast, easy tests actually show substantial underconfidence. We develop LifeEval, a test for evaluating model calibration across levels of difficulty. 然而，重要的是，这种倾向受到显著的“难易效应”（hard-easy effect）调节，即在困难测试中过度自信最为严重；相比之下，在简单测试中反而表现出明显的置信度不足。我们开发了 LifeEval，这是一个用于评估模型在不同难度水平下校准能力的测试集。
Paper Details:
- Authors: Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore
- arXiv ID: 2605.23909
- Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
- Submission Date: 3 Apr 2026
论文详情：
- 作者： Noam Michael, Daniel BenShushan, Jacob Bien, Don A. Moore
- arXiv ID： 2605.23909
- 学科分类： 人工智能 (cs.AI)；机器学习 (cs.LG)
- 提交日期： 2026年4月3日