AOS-Bench: Can LLM Agents Manage Dynamic Workloads?

Computer Science > Artificial Intelligence arXiv:2610.33178 (cs) [Submitted on 23 Oct 2026 (v1), last revised 27 Oct 2026 (this version, v2)] Title: AOS-Bench: Can LLM Agents Manage Dynamic Workloads? Authors: Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang.

计算机科学 > 人工智能 arXiv:2610.33178 (cs) [提交于 2026 年 10 月 23 日 (v1)，最后修订于 2026 年 10 月 27 日 (此版本，v2)] 标题：AOS-Bench：大模型智能体能否管理动态工作负载？作者：Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang。

Abstract: Coding agents are moving from handling one request at a time to serving as persistent assistants to which users hand a stream of work: a build, a data-processing job and a bug fix may all be pending on the same machine at once. Such an agent must decide when to take on new tasks, how to delegate and overlap them, and how to stay within the machine’s CPU and memory. Existing benchmarks cannot measure this, because they evaluate each task in isolation.

摘要：编程智能体正在从一次处理一个请求转向担任持久助手，用户会向其提交一系列工作流：构建任务、数据处理作业和错误修复可能同时在同一台机器上等待处理。此类智能体必须决定何时接收新任务、如何委派和重叠任务，以及如何保持在机器的 CPU 和内存限制内。现有的基准测试无法衡量这一点，因为它们是在孤立状态下评估每个任务的。

We introduce AOS-Bench, which evaluates an agent system as a workload manager: verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference. Across five model—harness pairs, isolated task success does not carry over evenly. On heavy workloads every system loses 13 to 23 points.

我们引入了 AOS-Bench，它将智能体系统作为工作负载管理器进行评估：经过验证的任务随时间推移而到达，必须在共享的 CPU 和内存限制下完成，而同一任务在单次运行时的表现则作为各系统的参考基准。在五个模型与测试框架的组合中，孤立任务的成功率并不能均匀地转化为流式任务的成功率。在高负载情况下，每个系统的得分都会下降 13 到 23 分。

On medium workloads, systems span 11 points in isolated success but 26 points in stream success, and two systems 0.6 points apart in isolation deliver 54% and 68% of the same tasks. Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet more elaborate management does not necessarily help.

在中等负载下，各系统在孤立成功率上的差距为 11 分，但在流式成功率上差距扩大到 26 分；两个在孤立测试中仅相差 0.6 分的系统，在流式任务中分别完成了 54% 和 68% 的任务。执行策略干预显示，这种差距很大程度上源于任务接收机制：串行调度导致 73% 的高负载任务未被处理。然而，更复杂的管理并不一定有帮助。

A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads, and under every policy median CPU use stays below 30% of the provisioned cores while tasks wait. These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it.

在处理中等和高负载任务时，固定并发数为 2 的策略与最佳智能体管理策略的得分差距在 4 分以内；在所有策略下，当任务处于等待状态时，CPU 的中位数使用率均保持在预配置核心数的 30% 以下。这些结果表明，工作负载管理是智能体的一项独特能力，而 AOS-Bench 是衡量该能力的一种有效方法。