A Task Without A Check Command Is Not Automated

没有校验指令的任务，就不是自动化

What eight stages of automated production taught me about the limits of agents. Our production line has eight stages and moves an idea to working software: research, architecture, task breakdown, build, review, human approval. Multiple engines run in lanes underneath it, chosen per task. The rule that keeps it alive is one sentence. If a task has no command that verifies its output, that task does not enter the pipeline.

八个自动化生产阶段教会了我关于智能体（Agent）局限性的道理。我们的生产线包含八个阶段，将一个想法转化为可运行的软件：研究、架构、任务拆解、构建、审查、人工审批。底层有多个引擎在不同通道中运行，并根据任务进行选择。维持这条生产线运作的准则只有一句话：如果一个任务没有校验其输出的指令，那么该任务就不能进入流水线。

Why the rule exists

为什么存在这条准则

An agent can produce something for almost any instruction. The output arrives and it looks plausible. Plausible is where the trouble starts. Without a verification command there is no difference between finished and abandoned, so the pipeline advances on the appearance of work. The check does not have to be clever. A test suite, a build that must compile, a parser that must not throw, a count that must match a known figure. It has to be mechanical and it has to be able to fail.

智能体几乎可以针对任何指令生成内容。输出结果看起来似乎合情合理，但“看起来合理”正是问题的开端。如果没有校验指令，就无法区分任务是“已完成”还是“被废弃”，流水线会仅仅因为“看起来有进展”而继续推进。校验过程不必很聪明。它可以是一个测试套件、一个必须编译通过的构建、一个不能报错的解析器，或者一个必须与已知数值匹配的计数。它必须是机械化的，并且必须具备失败的能力。

Tasks that cannot be checked mechanically still exist in our work. Judgement about tone, a design direction, whether a client will accept something. Those do not go into the pipeline. They go to a person, and pretending otherwise just moves the failure later.

工作中依然存在无法通过机械方式校验的任务。例如对语气的判断、设计方向的把控，或是客户是否会接受某项成果。这些任务不应进入流水线，而应交给人工处理；假装机器能处理这些只会将失败推迟到后续环节。

Engines are not interchangeable

引擎并非可以随意互换

We run several engines and route work by lane. I assumed the routing could be naive, and it could not. One engine cannot run our verification commands at all in its sandbox, so every task with a gate attached silently fell back to another lane and consumed a turn doing nothing useful. Another engine cannot read from disk, which means it cannot see the existing code, which means giving it a modification task produces a confident rewrite of something it never saw. Both of those look like model quality problems in the output. Neither is. They are routing problems, and they are only visible if you measure which lane did the work and whether that lane could physically do it.

我们运行着多个引擎，并按通道分配工作。我曾以为路由分配可以很简单，但事实并非如此。其中一个引擎在沙盒中完全无法运行我们的校验指令，因此每个带有门控的任务都会静默地回退到另一个通道，白白浪费了一次执行机会。另一个引擎无法读取磁盘，这意味着它看不到现有代码，如果给它分配修改任务，它会自信地重写它从未见过的东西。这两者看起来都像是模型输出质量的问题，但其实不然。它们是路由问题，只有当你测量是哪个通道执行了工作，以及该通道是否具备物理执行能力时，才能发现这些问题。

The limit I keep running into

我不断碰到的局限性

The pipeline is good at anything with a mechanical definition of done. It is useless at deciding what is worth doing, and it degrades quietly when the definition of done is soft. That is the honest boundary of an agentic setup today. Not the model’s reasoning, and not tool access, which is now routine. The boundary is how much of your work can be stated as a condition a machine can check. In our case that turned out to be most of the production and almost none of the direction.

流水线擅长处理任何具有“机械化完成定义”的任务。它在决定“什么值得做”方面毫无用处，而且当完成定义比较模糊时，它的性能会悄然下降。这就是当今智能体架构的真实边界。这不在于模型的推理能力，也不在于工具访问能力（这已是常规操作）。边界在于：你的工作中有多少比例可以被表述为机器能够校验的条件。在我们的案例中，这涵盖了大部分生产环节，但几乎不涉及任何方向性决策。