StochBench: A Domain-Specific Benchmark for Stochastic Processes in Lean
StochBench: A Domain-Specific Benchmark for Stochastic Processes in Lean
StochBench:Lean 中针对随机过程的领域特定基准测试
Leading benchmarks for formal theorem proving with large language models are small collections drawn from competition math, such as the IMO and Putnam, that poorly represent field-specific applications. 目前用于大语言模型形式化定理证明的主流基准测试,大多是源自国际数学奥林匹克(IMO)和普特南数学竞赛(Putnam)等竞赛数学的小型集合,这些题目难以代表特定领域的实际应用。
We introduce StochBench, a Lean 4 benchmark of 450 graduate stochastic-processes problems at varying abstraction levels, each paired with its natural-language source. 我们推出了 StochBench,这是一个基于 Lean 4 的基准测试,包含 450 道不同抽象程度的研究生水平随机过程题目,每道题都配有其自然语言来源。
Addressing a field underrepresented in Mathlib, it covers finite and countable Markov chains, renewal processes, random walks, martingales, stopping times, queues, Brownian motion, stochastic calculus, weak convergence, and Poisson and continuous-time Markov processes. 针对 Mathlib 中代表性不足的领域,该基准测试涵盖了有限和可数马尔可夫链、更新过程、随机游走、鞅、停时、排队论、布朗运动、随机微积分、弱收敛以及泊松过程和连续时间马尔可夫过程。
Our Opus 4.8-based agent achieves a 34.9% proof rate (157/450) under a 15-minute per-problem limit. 我们基于 Opus 4.8 的智能体在每题 15 分钟的限制下,实现了 34.9%(157/450)的证明成功率。
StochBench better represents domain-specific applied mathematics while remaining challenging for advanced provers. StochBench 更好地代表了领域特定的应用数学,同时对高级证明系统而言仍具有挑战性。