Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity

Harness 即语言：一种极简且极具表达力的智能体框架

Abstract: Modern language-model agents are built around the agent loop, where the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain workflows currently require additional engineering beyond the agent loop itself, such as memory systems and self-improving systems.

摘要： 现代语言模型智能体通常围绕“智能体循环”（agent loop）构建，即将大语言模型（LLM）置于一个提供工具集的环境中，通过交替进行工具调用和观察输出，使 LLM 能够完全控制工作流。然而，某些工作流目前需要智能体循环之外的额外工程设计，例如记忆系统和自我改进系统。

We built an LLM agent framework, JAZ, to explore the extent to which a minimal harness that is little more than the agent loop itself can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive invoke and provides a set of built-in hooks that allow the programmer to apply constraints and monitoring.

我们构建了一个名为 JAZ 的 LLM 智能体框架，旨在探索一个仅包含智能体循环本身的极简框架，能在多大程度上完成那些专门系统所设计的任务。JAZ 暴露了一个基于 LLM 的单一原语 invoke，并提供了一组内置钩子（hooks），允许程序员施加约束和监控。

Generalizing existing code-mode agent loops, invoke is the simplest loop that satisfies two defining properties: (1) the LLM can write arbitrary executable code that can include recursive invoke; (2) everything visible to the LLM — all inputs to invoke as well as its interaction history with the code environment — are variables in the code environment.

作为对现有代码模式智能体循环的泛化，invoke 是满足以下两个定义属性的最简单循环：(1) LLM 可以编写任意可执行代码，其中可以包含递归的 invoke；(2) LLM 可见的所有内容——包括 invoke 的所有输入以及它与代码环境的交互历史——都是代码环境中的变量。

We motivate our design from first principles, viewing invoke as a language primitive representing a function whose implementation is provided at runtime by an LLM every time it is called. To validate the design of our core invoke primitive, we evaluate invoke — with only prompting, no manually designed tools, harness, or external systems (e.g., memory or the file system) — on workflows traditionally implemented through specialized external harnesses.

我们从第一性原理出发阐述了我们的设计，将 invoke 视为一种语言原语，代表一个函数，其实现由 LLM 在每次调用时在运行时提供。为了验证核心 invoke 原语的设计，我们仅通过提示词（prompting），在没有手动设计的工具、框架或外部系统（如记忆或文件系统）的情况下，对传统上通过专用外部框架实现的工作流进行了评估。

On long-horizon workflows requiring recall beyond the context window, JAZ invoke outperforms Letta (MemGPT) by 8% at half its cost on the recall-heavy portion of StuLife. On continual self-improvement, JAZ invoke outperforms ACE by 4% at a lower cost on AppWorld.

在需要超出上下文窗口进行回溯的长程工作流中，JAZ invoke 在 StuLife 的重回溯部分表现优于 Letta (MemGPT) 8%，且成本仅为其一半。在持续自我改进方面，JAZ invoke 在 AppWorld 上以更低的成本表现优于 ACE 4%。