SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2610.12596 (cs) [Submitted on 8 Oct 2026] Title:SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire Authors:Scott C. Lowe, Sander W. Keemink View a PDF of the paper titled SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire, by Scott C. Lowe and 1 other authors View PDF HTML (experimental)

计算机科学 > 人工智能 arXiv:2610.12596 (cs) [提交于 2026 年 10 月 8 日] 标题：SpireBench：利用《杀戮尖塔》评估智能模型的长程战略博弈作者：Scott C. Lowe, Sander W. Keemink 查看论文 PDF，标题为《SpireBench：利用《杀戮尖塔》评估智能模型的长程战略博弈》，作者：Scott C. Lowe 及另一位作者查看 PDF HTML（实验性）

Abstract:Agentic large language models now tackle complex tasks, yet strategic reasoning over long horizons is under-studied: most agentic benchmarks are short and episodic, and performance on long tasks is hard to dissect into stored knowledge, reasoning and luck. We introduce SpireBench, in which models play the complete commercial deck-builder Slay the Spire (StS), where early decisions decide long-term success.

摘要：智能大语言模型目前能够处理复杂的任务，但针对长程视角的战略推理研究尚显不足：大多数智能基准测试都是短期的、片段式的，且长程任务的表现很难拆解为存储知识、推理能力和运气。我们引入了 SpireBench，在该基准中，模型需要游玩完整的商业卡牌构建游戏《杀戮尖塔》（StS），其中早期的决策决定了长期的成功。

Across 20 large language models (LLMs) at the game’s highest difficulty, frontier models beat heuristic baselines and match the average human player but remain far behind experts. This gap widens as the run lengthens: the strongest models win the first act as often as experts, then fall behind. What separates strong models from weak is how well they play each fight; all of them misjudge risk over a run, preserving health rather than spending it to grow stronger.

在游戏最高难度下，对 20 个大语言模型（LLM）进行测试发现，前沿模型击败了启发式基准并达到了普通人类玩家的水平，但仍远落后于专家。随着游戏进程的推进，这一差距进一步拉大：最强的模型在第一幕的胜率与专家相当，但随后便会落后。强模型与弱模型的区别在于它们处理每场战斗的表现；所有模型在整个游戏过程中都误判了风险，倾向于保留生命值而非通过消耗生命值来变得更强。

A comprehensive, self-consistent renaming of every game term shows that part of their performance rests on prior knowledge of the game, and models do not improve from the memories they write across runs, which are often wrong and overly cautious. Our harness is the first to give a model the complete original game, and we compare it with existing harnesses. Overall, SpireBench offers an unsaturated long-horizon benchmark with tools to locate where and why agents fail.

通过对所有游戏术语进行全面且自洽的重命名，研究表明模型的部分表现依赖于对游戏的先验知识；此外，模型无法从其在多次游戏过程中记录的记忆中获得改进，这些记忆往往是错误的且过于谨慎。我们的测试框架是首个将完整原始游戏提供给模型的工具，我们将其与现有框架进行了对比。总体而言，SpireBench 提供了一个尚未饱和的长程基准测试，并配备了用于定位智能体失败位置和原因的工具。