FinSkillBench: Evaluating AI Agents and Domain Skills for Investment Management

FinSkillBench：评估投资管理中的 AI 智能体与领域技能

Investment management is a high-stakes domain in which agentic AI systems must do more than generate plausible text. They must retrieve point-in-time data, assemble correct computational inputs, invoke specialized methods, and produce auditable structured outputs. 投资管理是一个高风险领域，在该领域中，智能体 AI 系统不仅需要生成看似合理的文本，还必须能够检索特定时间点的数据、整合正确的计算输入、调用专业方法，并产出可审计的结构化输出。

We introduce FinSkillBench, an evaluation suite designed to measure whether language model agents can effectively use financial domain skills to solve investment management tasks. The benchmark spans three domains, portfolio construction, risk management, and fundamental analysis, and includes 12 subtasks with 2,603 task episodes. Each episode provides point-in-time inputs, hidden ground truth, and a task-specific evaluation harness. 我们推出了 FinSkillBench，这是一套旨在衡量语言模型智能体能否有效利用金融领域技能来解决投资管理任务的评估套件。该基准测试涵盖了投资组合构建、风险管理和基本面分析三个领域，包含 12 个子任务和 2,603 个任务片段。每个片段都提供了特定时间点的输入、隐藏的真实结果以及任务特定的评估工具。

We compare three conditions: no skill, curated skill packages consisting of procedural documents and executable components, and self-generated skills in which the agent writes and reuses its own procedures within an episode. 我们对比了三种情况：无技能辅助、由程序文档和可执行组件组成的精选技能包，以及智能体在片段内自行编写并复用程序的自生成技能。

Across 9 models and a large-scale evaluation, curated skills consistently improve performance, raising mean scores from 0.366 to 0.528, with the largest gains in portfolio construction and risk management. In contrast, self-generated skills provide little benefit despite higher computational cost. 通过对 9 个模型进行大规模评估，结果显示精选技能包能持续提升性能，将平均得分从 0.366 提高到 0.528，其中在投资组合构建和风险管理方面的提升最为显著。相比之下，自生成技能尽管计算成本更高，却几乎没有带来额外收益。

An independent evaluation using a separate agent framework (Hermes Agent, 8 models, 5,280 episodes total) reproduces the directional pattern across all three domains, with the magnitude of skill effects varying by subtask and harness. These results show that in investment management agents, access to reliable procedural skills can be as important as model choice, while naive self-generation of skills is often ineffective. 使用独立智能体框架（Hermes Agent，8 个模型，共 5,280 个片段）进行的独立评估，在所有三个领域中重现了相同的趋势，尽管技能效果的幅度因子任务和评估工具而异。这些结果表明，对于投资管理智能体而言，获取可靠的程序化技能与模型选择同样重要，而盲目的技能自生成往往是无效的。

We release the benchmark, evaluation tools, curated skill packages, and full trajectories to support further research. 我们发布了该基准测试、评估工具、精选技能包以及完整的执行轨迹，以支持后续研究。