Automatic Construction of Strong Simple Baselines
Computer Science > Machine Learning arXiv:2610.40700 (cs) [Submitted on 28 Oct 2026] Title:Automatic Construction of Strong Simple Baselines Authors:Caleb N. Ellington, Jiaqi Wang, Sohan Addagudi, Elijah Cole View a PDF of the paper titled Automatic Construction of Strong Simple Baselines, by Caleb N. Ellington and 3 other authors View PDF HTML (experimental)
计算机科学 > 机器学习 arXiv:2610.40700 (cs) [提交于 2026 年 10 月 28 日] 标题：自动构建强大的简单基准 作者：Caleb N. Ellington, Jiaqi Wang, Sohan Addagudi, Elijah Cole 查看标题为“自动构建强大的简单基准”的论文 PDF，作者为 Caleb N. Ellington 及其他 3 位作者 查看 PDF HTML（实验性）
Abstract:To rigorously establish the value of a novel deep learning method, it must be compared against appropriate baselines. As deep learning has become increasingly common in the physical and life sciences, there have been numerous cases in which apparent progress has turned out to be an artifact of omitted or missing baselines.
摘要：为了严格确立一种新型深度学习方法的价值，必须将其与适当的基准进行比较。随着深度学习在物理科学和生命科学中变得越来越普遍，出现了许多案例，其中明显的进展被证明是由于忽略或缺失基准而产生的假象。
In this work, we investigate whether this problem can be mitigated by using large language models to automate the construction of baselines. We assemble 36 supervised classification and regression tasks from peer-reviewed studies in Nature, Science, and Cell-family journals that investigate novel deep learning methods for scientific applications.
在这项工作中，我们研究了是否可以通过使用大型语言模型来自动化构建基准以缓解这一问题。我们从《自然》、《科学》和《细胞》系列期刊的同行评审研究中收集了 36 项监督分类和回归任务，这些研究探讨了用于科学应用的新型深度学习方法。
We then use a language model to construct simple baselines for each task. These automatically constructed baselines are strong, recovering a median of 95.5% (mean 85.7%) of the gap between a trivial predictor and the score of the deep learning model. The median simple baseline matches or exceeds the performance of the deep learning model in 12 studies.
然后，我们使用语言模型为每项任务构建简单的基准。这些自动构建的基准非常强大，恢复了平凡预测器与深度学习模型得分之间差距的中位数 95.5%（平均 85.7%）。在 12 项研究中，中位数简单基准达到或超过了深度学习模型的性能。
We further examine the variation across scientific domains and publication venues and we characterize the effects of test-time compute and access to different levels of study context. As human peer review has struggled to keep up with the rapidly increasing volume of research articles in recent years, our results suggest that automated baseline construction is a practical and scalable check on deep learning contributions in the physical and life sciences.
我们进一步研究了不同科学领域和出版场所之间的差异，并描述了测试时计算和获取不同程度研究背景的影响。由于近年来人类同行评审难以跟上研究论文数量的快速增长，我们的结果表明，自动基准构建是对物理和生命科学领域深度学习贡献的一种实用且可扩展的检查方式。