OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2610.37739 (cs) [Submitted on 27 Oct 2026] Title: OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery Authors: Jiarui Yao, Ruida Wang, Tong Zhang.

计算机科学 > 人工智能 arXiv:2610.37739 (cs) [提交于 2026 年 10 月 27 日] 标题：OpenFrontiers：开放研究问题集与科学再发现基准作者：Jiarui Yao, Ruida Wang, Tong Zhang。

Abstract: Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging.

摘要：大型语言模型（LLMs）在研究级问题解决方面展现出日益增长的潜力，但目前仍缺乏此类问题的综合集合，且在研究级问题上对其进行评估依然充满挑战。

We introduce OpenFrontiers, a cross-disciplinary resource that combines a structured corpus of open research problems with a retrospective benchmark of recently resolved problems in mathematics. The corpus records problem statements, context, progress, and other information for 12,151 problems.

我们引入了 OpenFrontiers，这是一个跨学科资源，结合了结构化的开放研究问题语料库与数学领域近期已解决问题的回顾性基准。该语料库记录了 12,151 个问题的陈述、背景、进展及其他信息。

The benchmark contains 200 problems, 110 resolved in 2026 and 90 still open, each with a Lean 4 statement checked by automated probes and two independent reviews; a subset HQ-50 further filters based on quality score, etc.

该基准包含 200 个问题，其中 110 个于 2026 年解决，90 个仍处于开放状态；每个问题均配有经自动探测器检查和两次独立评审的 Lean 4 陈述；HQ-50 子集则基于质量评分等标准进行了进一步筛选。

Our evaluation protocol grades formal proofs with two independent checkers in an isolated sandbox under closed-book, frozen-retrieval, and live-web conditions.

我们的评估协议在隔离沙箱中，通过两个独立检查器，在闭卷、冻结检索和实时网络条件下对形式化证明进行评分。

Evaluating seven LLM agents on the HQ-50 subset under a standardized budget, we find that closed-book rediscovery is rare, at most 7 of 40 resolved problems, whereas live-web success mostly reflects retrieval of published solutions.

在标准化预算下对 HQ-50 子集上的七个 LLM 智能体进行评估后，我们发现闭卷再发现的情况很少见（40 个已解决问题中最多只有 7 个），而实时网络环境下的成功大多反映了对已发表解决方案的检索。

By linking a broad research-problem corpus to a formally graded evaluation subset with documented solutions, OpenFrontiers provides a foundation for measuring research-level rediscovery and studying progress toward solving currently open scientific problems.

通过将广泛的研究问题语料库与带有记录解决方案的正式评分评估子集相连接，OpenFrontiers 为衡量研究级再发现以及研究解决当前开放科学问题的进展奠定了基础。