本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Jepsen: Orkes Conductor 5.3.2

1 Background

A workflow orchestrator is a software system which executes computations made up of small tasks. Tasks are connected in a workflow, which routes the outputs of some tasks to the inputs of others, determines which tasks may be scheduled and when, and so on. The orchestrator manages workflows and specific executions of them, runs their tasks, persists state, and tries to drive each execution to completion.

工作流编排器是一种由小型任务组成的计算执行软件系统。任务在工作流中相互连接，将某些任务的输出路由至其他任务的输入，并确定哪些任务可以被调度以及何时调度等。编排器负责管理工作流及其具体执行过程、运行任务、持久化状态，并尝试推动每个执行过程直至完成。

Conductor is a workflow system originally built by Netflix and open sourced in 2016. The open source version, Conductor OSS, is now maintained by the Conductor OSS Foundation. This foundation is led by Orkes, whose founders helped build Conductor at Netflix. Orkes also offers a hosted version of Conductor with additional features, called Orkes Conductor. This analysis focuses on Orkes Conductor, which we call Conductor for short.

Conductor 是最初由 Netflix 构建并于 2016 年开源的工作流系统。其开源版本 Conductor OSS 目前由 Conductor OSS 基金会维护。该基金会由 Orkes 领导，其创始人曾参与 Netflix Conductor 的构建。Orkes 还提供了一个具有额外功能的托管版 Conductor，称为 Orkes Conductor。本分析重点关注 Orkes Conductor，简称 Conductor。

In Conductor, a workflow definition is a JSON structure that defines which tasks must be executed, their order, what their inputs and outputs should be, and so on. Each workflow definition is identified by a name and a version. For example, a workflow definition could specify that to adopt a cat, one must first complete an application, then schedule a vet visit, and receive vaccination records from the vet. A workflow may also accept an input (say, the name of the cat) and produce an output (like a certificate of adoption).

在 Conductor 中，工作流定义是一个 JSON 结构，用于定义必须执行的任务、执行顺序、输入输出要求等。每个工作流定义都由名称和版本标识。例如，工作流定义可以规定，要领养一只猫，必须先完成申请，然后预约兽医就诊，并从兽医处获取疫苗接种记录。工作流还可以接受输入（例如猫的名字）并产生输出（如领养证书）。

A workflow definition may be run many times—say, once for each cat to be adopted. Each instance is called a workflow execution, but we often call an execution a “workflow” for short. Each execution is identified by a unique workflow ID. During an execution a task may also be attempted multiple times (e.g. in case of failure), and each attempt receives a unique execution ID.

工作流定义可以运行多次——例如，每领养一只猫就运行一次。每个实例被称为工作流执行，但我们通常简称执行为“工作流”。每次执行都由唯一的 workflow ID 标识。在执行过程中，任务也可能被尝试多次（例如在失败的情况下），每次尝试都会获得一个唯一的执行 ID。

Some tasks are built into Conductor. Some are system tasks, like making HTTP requests or submitting events to a log like Kafka. Others are called operators—these provide control flow like conditionals, fork/join, loops, and so on. Operators may execute ECMAScript code to perform user-defined logic. Finally there are worker tasks, which are written by users to perform the “business logic” of the system.

有些任务是 Conductor 内置的。有些是系统任务，例如发起 HTTP 请求或向 Kafka 等日志提交事件。其他被称为操作符——它们提供控制流，如条件判断、分支/合并（fork/join）、循环等。操作符可以执行 ECMAScript 代码以实现用户定义的逻辑。最后是工作者任务（worker tasks），由用户编写以执行系统的“业务逻辑”。

To start an execution of a workflow, one can make an API call to Conductor, submit an event through a message broker, or use Conductor’s built-in cron-style scheduler. Once acknowledged, Conductor schedules the workflow’s tasks and attempts to drive it to completion.

要启动工作流执行，可以通过调用 Conductor API、通过消息代理提交事件，或使用 Conductor 内置的 cron 式调度器。一旦确认，Conductor 就会调度工作流的任务并尝试推动其完成。

1.1 Architecture

Conductor has five major components. Conductor servers are stateless processes which manage workflows and schedule execution. They provide an API and user interface, interact with external services, and track state. The definitions of workflows, the state of their executions and the results of tasks are stored in a database like Postgres, Cassandra, or Redis.

Conductor 有五个主要组件。Conductor 服务器是管理工作流和调度执行的无状态进程。它们提供 API 和用户界面，与外部服务交互并跟踪状态。工作流定义、执行状态和任务结果存储在 Postgres、Cassandra 或 Redis 等数据库中。

Conductor also uses a queue—often Redis—to dispatch work to various processes, and to store short-lived state. Finally, there are two pools of stateless workers: standard workers which execute Conductor’s built-in tasks, and custom workers, which are written by users using SDKs for various languages. All workers connect to a Conductor server, poll the server for tasks, execute them, and send results back to the server.

Conductor 还使用队列（通常是 Redis）将工作分发给各个进程，并存储短期状态。最后，有两个无状态工作者池：执行 Conductor 内置任务的标准工作者，以及用户使用各种语言 SDK 编写的自定义工作者。所有工作者都连接到 Conductor 服务器，轮询服务器获取任务，执行任务，并将结果发送回服务器。

Each component may be scaled independently. Any number of servers and workers can run concurrently, and nodes may be killed and restarted at any time. In version 5.3.2, the database and queue were both required to preserve all acknowledged writes. However, version 5.4.0 introduced a new option which enabled the use of a lossy storage system for all queues except for the main decider queue.

每个组件都可以独立扩展。任意数量的服务器和工作者可以并发运行，节点可以随时被终止并重启。在 5.3.2 版本中，数据库和队列都需要保留所有已确认的写入。然而，5.4.0 版本引入了一个新选项，允许除主决策队列（main decider queue）外的所有队列使用有损存储系统。

1.2 Guarantees

Conductor guarantees “at least once message delivery”, ensuring that all messages are persistent and will be delivered to task workers one or more times. Conductor also guarantees that workflows eventually terminate. “If a workflow has started, it will run to completion as long as all its tasks are completed.”

Conductor 保证“至少一次消息传递”，确保所有消息都是持久化的，并将被传递给任务工作者一次或多次。Conductor 还保证工作流最终会终止。“如果工作流已经启动，只要其所有任务都已完成，它就会运行至完成。”

This does not necessarily mean that workflows and tasks execute successfully. Tasks can run out of retry attempts, fail with non-retryable errors, or be explicitly canceled—and a similar state machine applies to workflow executions. However, workflows should eventually converge on a terminal status: COMPLETED, FAILED, TIMED_OUT, or TERMINATED.

这并不一定意味着工作流和任务都能成功执行。任务可能会耗尽重试次数、因不可重试的错误而失败，或被显式取消——类似的状态机也适用于工作流执行。然而，工作流最终应收敛于一个终止状态：COMPLETED（已完成）、FAILED（已失败）、TIMED_OUT（已超时）或 TERMINATED（已终止）。

When tasks are executed, each attempt receives a unique execution ID, and also a unique retry count, which indicates the number of attempts already performed. Orkes says that for each workflow execution, each instance of a task has at most one visible successful execution. Tasks may execute successfully multiple times—for instance, if a Conductor server loses its connection to the database just before persisting the completion of the task execution. However, once the output of a successful task becomes visible, no task, workflow, or API call will observe a different output.

当任务执行时，每次尝试都会获得一个唯一的执行 ID 以及一个唯一的重试计数，该计数表示已执行的尝试次数。Orkes 表示，对于每个工作流执行，任务的每个实例最多有一个可见的成功执行。任务可能会多次成功执行——例如，如果 Conductor 服务器在持久化任务执行完成之前与数据库断开连接。然而，一旦成功任务的输出变得可见，任何任务、工作流或 API 调用都不会观察到不同的输出。

2 Test Design

To verify these properties, we built a test harness for Orkes Conductor using the Jepsen testing library and the Conductor Java SDK at version 5.0.1. We used this harness to test Orkes Conductor 5.3.2, 5.4.0, and 5.5.0-rc1 and -rc2.

为了验证这些属性，我们使用 Jepsen 测试库和 5.0.1 版本的 Conductor Java SDK 为 Orkes Conductor 构建了一个测试工具。我们使用该工具测试了 Orkes Conductor 5.3.2、5.4.0 以及 5.5.0-rc1 和 -rc2 版本。

Orkes typically deploys Conductor in cloud environments with Postgres (or a Postgres-compatible database, like their in-house ParticleDB) as the primary database, and Redis (or a compatible service like Valkey) as the queue. To approximate these deployments, we set up clusters of ten Debian machines, both in containers and as separate EC2 VMs: a single Postgres node for the database, either a single Redis node or Redis Cluster for the queue, and the remaining nodes split evenly between Conductor servers, Conductor’s standard workers, and a custom worker we wrote using the Java SDK.

Orkes 通常在云环境中使用 Postgres（或兼容 Postgres 的数据库，如其内部的 ParticleDB）作为主数据库，并使用 Redis（或兼容服务如 Valkey）作为队列来部署 Conductor。为了模拟这些部署，我们设置了由十台 Debian 机器组成的集群，包括容器和独立的 EC2 虚拟机：一个用于数据库的 Postgres 节点，一个用于队列的单节点 Redis 或 Redis 集群，其余节点平均分配给 Conductor 服务器、Conductor 标准工作者以及我们使用 Java SDK 编写的自定义工作者。

Our test workload defined a single workflow with one task, handled by our custom worker. The task slept up to 30 seconds to simulate work being done, then returned a randomly generated output. It also logged its execution with the test harness, so we could tell exactly which t

我们的测试工作负载定义了一个包含单个任务的工作流，由我们的自定义工作者处理。该任务休眠长达 30 秒以模拟正在进行的工作，然后返回一个随机生成的输出。它还通过测试工具记录了其执行情况，以便我们能够准确判断哪个 t