Put the Agent Inside the Workflow

将智能体置于工作流之中

A hybrid LLM application pattern that combines a predefined workflow with adaptive agent behavior. 一种结合了预定义工作流与自适应智能体行为的混合大模型（LLM）应用模式。

When building an LLM application, one of the first design decisions we have to make is: Workflow or agent? The workflow paradigm follows a sequence we define in advance. This makes the application very easy to understand, and gives us clear control over how information moves from one stage to the next. It works well when we know which operation should happen at each stage. 在构建大模型应用时，我们首先要做的设计决策之一是：选择工作流还是智能体？工作流范式遵循我们预先定义的顺序。这使得应用非常易于理解，并让我们能够清晰地控制信息如何在不同阶段之间流转。当我们明确每个阶段应该执行什么操作时，这种模式非常有效。

For more open-ended questions, however, the next useful action may depend on what the system discovers along the way. The agent paradigm, on the other hand, starts with a goal and decides which actions or tools to use along the way. This makes it more flexible when the solution path is uncertain. However, that flexibility also means we give up some control over how the solution unfolds. 然而，对于更开放的问题，下一步最有用的行动可能取决于系统在过程中发现的内容。另一方面，智能体范式从目标出发，并决定在过程中使用哪些行动或工具。当解决方案路径不确定时，这使其更具灵活性。但这种灵活性也意味着我们放弃了对解决方案展开过程的部分控制。

But why make this choice for the entire application? From what I see, in practice, many applications contain both kinds of work. Some stages might perform a known transformation, then the workflow paradigm is a good choice. Other stages might need to adapt based on intermediate results, which naturally calls for the agentic approach. For these applications, a hybrid workflow-agent pattern is a better fit. 但为什么非要为整个应用做出这种选择呢？据我观察，在实践中，许多应用同时包含这两种工作模式。某些阶段可能执行已知的转换，那么工作流范式是一个不错的选择。而其他阶段可能需要根据中间结果进行调整，这自然需要智能体方法。对于这些应用，混合工作流-智能体模式更为合适。

In this post, we’ll explore this hybrid pattern through a concrete case study. The overall solution path will remain fixed, while an agent is placed inside the stage where the path cannot be determined in advance. 在本文中，我们将通过一个具体的案例研究来探讨这种混合模式。整体解决方案路径将保持固定，而智能体将被放置在无法预先确定路径的阶段中。

1. Case Study: LLM-Assisted Hyperparameter Tuning

1. 案例研究：大模型辅助的超参数调优

For the case study, let’s build an LLM application that helps select algorithms and tune hyperparameters for classification problems. This application works like this: it first receives a labeled dataset and a plain-language modeling request. It then needs to run model experiments and recommend one of the tested configurations. Finally, it summarizes the result. Naturally, we can break this work into three stages. 在案例研究中，让我们构建一个大模型应用，帮助分类问题选择算法并调整超参数。该应用的工作方式如下：首先接收一个带标签的数据集和自然语言建模请求；然后运行模型实验并推荐其中一种测试配置；最后总结结果。自然地，我们可以将这项工作分为三个阶段。

1.1 Prepare the Experiment

1.1 准备实验

The purpose of this stage is to translate the modeling request into a brief that contains the objective, evaluation metric, and cross-validation setup. Since we already know the input, the desired operation to perform, and the expected output, a single LLM call is sufficient for this stage. 此阶段的目的是将建模请求转化为包含目标、评估指标和交叉验证设置的简报。由于我们已经明确了输入、所需执行的操作以及预期输出，因此该阶段仅需一次大模型调用即可。

1.2 Hyperparameter Tuning

1.2 超参数调优

At this stage, we need to run the experiments. Here, we know the goal, i.e., finding the best model and the associated hyperparameters. But we don’t know the exact sequence of actions required to reach it. The next useful experiment should depend on the results observed so far. As a result, this stage is better handled by an agent, who can dynamically choose classifier configurations, evaluate them, and continue exploring. 在此阶段，我们需要运行实验。在这里，我们明确了目标，即找到最佳模型及其相关的超参数。但我们不知道实现该目标所需的具体行动顺序。下一步最有用的实验应取决于目前观察到的结果。因此，这个阶段由智能体处理会更好，它可以动态选择分类器配置、进行评估并持续探索。

1.3 Summarization and Reporting

1.3 总结与报告

Finally, we need to summarize the completed run. At this stage, the experiment brief, trial history, and recommendation are already available. Turning them into a structured report is a known operation, so a single LLM call is again sufficient. 最后，我们需要总结已完成的运行。在此阶段，实验简报、试验历史和推荐结果都已经准备就绪。将它们转化为结构化报告是一个已知的操作，因此再次仅需一次大模型调用即可。

As you can see, in our intended LLM application, the overall problem-solving sequence is fixed, with preparation, exploration, and reporting. Within this predefined workflow, we introduce autonomy only at the middle stage to enable adaptive model experimentation. This way, we combine the clarity of a workflow with the flexibility of agentic experimentation. 正如你所见，在我们预想的大模型应用中，整体问题解决顺序是固定的，即准备、探索和报告。在这个预定义的工作流中，我们仅在中间阶段引入自主性，以实现自适应的模型实验。通过这种方式，我们将工作流的清晰性与智能体实验的灵活性结合在了一起。

2. Building the Three-Stage Workflow

2. 构建三阶段工作流

The complete application can be expressed in three calls: 整个应用可以通过三次调用来表示：

experiment_spec = prepare_experiment(modeling_request, dataset_summary)
recommendation, trial_history = await explore_configurations(experiment_spec, dataset_summary)
report = summarize_run(experiment_spec, trial_history, recommendation)

Now, let’s unpack the three functions one by one. 现在，让我们逐一拆解这三个函数。

2.1 Preparing the Experiment with Structured Output

2.1 使用结构化输出准备实验

The first stage converts the modeling request and dataset summary into a compact experiment brief. We use a single structured LLM call to do that. We need to define the output schema, the LLM instruction, and the prompt builder for this stage. 第一阶段将建模请求和数据集摘要转换为简洁的实验简报。我们使用一次结构化大模型调用来完成此操作。我们需要为该阶段定义输出模式、大模型指令以及提示词构建器。

We start with the output schema, which is a Pydantic model: 我们从输出模式开始，这是一个 Pydantic 模型：

class ExperimentSpec(BaseModel):
    objective: str
    primary_metric: str = Field(description="A valid scikit-learn scoring name")
    cv_folds: int

It has three fields, specifying the information needed by the experimentation agent, i.e., what it should accomplish, how configurations should be evaluated, and how many cross-validation folds to use. This is the structured output feature of the LLM: it makes sure that the LLM’s output follows this predefined structure, thus greatly simplifying the downstream consumption of the results. 它包含三个字段，指定了实验智能体所需的信息，即它应该完成什么、如何评估配置以及使用多少折交叉验证。这就是大模型的结构化输出功能：它确保大模型的输出遵循此预定义结构，从而极大地简化了结果的下游处理。