Changing One Prompt Can Affect 50 Others — I Built a Prompt Dependency Graph to Find What Needs Retesting
Changing One Prompt Can Affect 50 Others — I Built a Prompt Dependency Graph to Find What Needs Retesting
修改一个提示词可能会影响其他 50 个——我构建了一个提示词依赖图来确定需要重新测试的内容
TL;DR If you build with composable prompts, changing one shared component can leave you with a difficult question: what actually needs to be re-evaluated? I built a pure Python prompt dependency graph that answers that question with two numbers:
- Reachable: everything downstream of the changed component—the structural ceiling.
- Candidate: the smaller set that directly depends on the changed section, plus its downstream consumers.
简而言之 如果你使用可组合的提示词进行构建，修改一个共享组件会让你面临一个难题：到底什么需要重新评估？ 我构建了一个纯 Python 的提示词依赖图，通过两个数字来回答这个问题：
- 可达范围 (Reachable)： 变更组件下游的所有内容——即结构上的上限。
- 候选范围 (Candidate)： 一个较小的集合，包含直接依赖于变更部分的组件及其下游消费者。
I tested the approach on a deterministic 55-node synthetic system. Depending on how selectively a component is shared, section-aware tracking narrowed the evaluation set by anywhere from 0% to 85% in my experiments. The important caveat: these numbers identify what should be evaluated, not what will actually fail. Behavioral impact still requires running the evaluation itself.
我在一个包含 55 个节点的确定性合成系统上测试了该方法。根据组件共享的选择性程度，在我的实验中，这种基于分段的追踪将评估范围缩小了 0% 到 85%。 需要特别注意：这些数字确定的是“应该评估什么”，而不是“实际会失败什么”。行为上的影响仍然需要通过运行评估本身来验证。
The results, before the methodology:
在介绍方法论之前，先看结果：
Here is the data before we get into the code. I ran a 55-node synthetic experiment across a few different change scenarios, tracking the total ceiling versus the actual evaluation set:
在进入代码之前，先看数据。我在几种不同的变更场景下运行了一个 55 节点的合成实验，追踪了总上限与实际评估集之间的对比：
| Change Target | Reachable (Ceiling) | Candidate (Evaluation Set) | Narrowing |
|---|---|---|---|
| base-policy / refunds | 45 | 24 | 47% |
| tone / professional | 55 | 5 | 90% |
| format / json | 55 | 35 | 36% |
| base-policy / privacy | 45 | 24 | 47% |
| safety / no-medical-advice | 15 | 13 | 13% |
The real takeaway here isn’t just that a specific change can slash your test burden by nearly half. The catch is how unpredictable the graph is. Sometimes it finds a massive amount of narrowing, and other times it finds nothing. You literally do not know which outcome you are getting until you actually run it.
这里真正的启示不仅仅是某个特定的变更可以将你的测试负担减少近一半。关键在于这个图表是多么不可预测。 有时它能大幅缩小范围，而有时却毫无作用。在实际运行之前，你根本不知道会得到什么样的结果。
The Line I Changed That I Couldn’t Reason About
我修改的那行代码，让我无法推断其影响
I compose most of my production prompts out of shared pieces. There is a base-policy block inherited by a support agent, a sales agent, and an internal analyst agent. There’s a tone block that almost imports everything, and a format block controlling JSON versus Markdown output. It is a completely ordinary setup once you have more than a handful of agents.
我大部分生产环境的提示词都是由共享片段组成的。有一个被支持代理、销售代理和内部分析代理继承的 base-policy（基础策略）块。还有一个几乎被所有内容导入的 tone（语气）块，以及一个控制 JSON 与 Markdown 输出的 format（格式）块。一旦代理数量超过几个，这便是一个非常普通的设置。
One afternoon I changed one sentence in base-policy, extending the refund window from 30 days to 14. The problem wasn’t making the edit. It was knowing which agents needed re-evaluation before I shipped it. Running the entire suite on everything was expensive. I had been guessing which agents used the policy, but I couldn’t tell when I was wrong. A correct guess and a lucky one look identical until something breaks. Shipping and waiting for a support agent to quote the old refund window to a real customer was obviously worse than either.
一天下午，我修改了 base-policy 中的一句话，将退款窗口从 30 天延长到 14 天。问题不在于修改本身，而在于在发布之前，我无法确定哪些代理需要重新评估。
对所有内容运行整个测试套件成本太高。我一直在猜测哪些代理使用了该策略，但我无法判断自己何时猜错了。在出问题之前，正确的猜测和幸运的猜测看起来是一样的。发布后等待支持代理向真实客户引用旧的退款窗口，显然比上述任何一种情况都要糟糕。
I wanted a structural answer to a simple question: what depends on the thing I just changed? That question already has a name in traditional software engineering, change impact analysis, which means tracing dependency relationships outward from a change to determine what else needs re-verification [1]. Prompt engineering still lacks many of the lifecycle practices that software engineering takes for granted, and a recent academic proposal for promptware engineering makes the same observation [2].
我想要一个简单问题的结构化答案：什么依赖于我刚刚修改的东西？在传统软件工程中，这个问题已经有了一个名字，即“变更影响分析”（change impact analysis），这意味着从变更点向外追踪依赖关系，以确定还有什么需要重新验证 [1]。提示词工程仍然缺乏许多软件工程视为理所当然的生命周期实践，最近一项关于“提示词工程”（promptware engineering）的学术提案也提出了同样的观察 [2]。
So I built the smallest version of that I could, and tested it against a system designed specifically to break my assumptions. Complete code: https://github.com/Emmimal/prompt-dependency-graph/
因此，我构建了该功能的最简版本，并针对一个专门为打破我假设而设计的系统进行了测试。 完整代码：https://github.com/Emmimal/prompt-dependency-graph/
Why Composability Creates a Larger Evaluation Surface
为什么可组合性会产生更大的评估面
The blast radius of composability: A single edit to the shared base-policy cascades through the dependency graph, carrying potential downstream impacts to the final JSON, email, and report outputs.
可组合性的“爆炸半径”：对共享 base-policy 的一次修改会通过依赖图级联，对最终的 JSON、电子邮件和报告输出产生潜在的下游影响。
Every shared component is a single point that many other things depend on. Every edit to it carries a blast radius. The term comes from explosives, but it has a long second life in software engineering describing how far a change’s consequences propagate outward from its source [3]. I am using it in that software-engineering sense here. It means potential downstream impact, not observed behavioral damage.
每个共享组件都是许多其他事物所依赖的单点。对其进行的每一次编辑都带有“爆炸半径”。 这个术语源于爆炸物，但在软件工程中有着悠久的应用历史，用来描述变更的后果从源头向外传播的距离 [3]。我在这里使用的是软件工程意义上的定义。它指的是潜在的下游影响，而不是观察到的行为损害。
This doesn’t measure whether the prompt’s output actually changed. It measures how far that change could spread if it did. Chaos engineering treats minimizing blast radius as a first-class design goal for a simple reason: not to prevent failure, but to keep its consequences bounded and legible [4].
这并不衡量提示词的输出是否真的发生了变化，而是衡量如果发生了变化，它会传播多远。 混沌工程将最小化爆炸半径视为首要设计目标，原因很简单：不是为了防止故障，而是为了将故障的后果限制在可控且清晰的范围内 [4]。
Build systems solve an adjacent problem the same way. Bazel maintains a dependency graph across a codebase specifically so a single file change only triggers rebuilds of what is actually downstream of it, rather than everything [5]. Prompt chaining has its own version of this failure. Because prompts feed into each other, a small and unintended change to an upstream prompt can produce unpredictable results several steps downstream, with no compiler or type system to catch it first [6].
构建系统以同样的方式解决了类似的问题。Bazel 在代码库中维护依赖图，专门用于确保单个文件的更改仅触发其下游内容的重新构建，而不是全部重新构建 [5]。 提示词链也有其特有的故障版本。由于提示词之间相互输入，对上游提示词的一个微小且无意的更改可能会在下游几个步骤产生不可预测的结果，且没有编译器或类型系统来预先捕获它 [6]。
That is exactly what I was worried about with base-policy. I just had no way to measure it.
这正是我对 base-policy 所担心的，只是我之前没有办法去衡量它。
Why a Flat Dependency Lookup Fails
为什么扁平化的依赖查找会失败
Before building anything complex, I tried the simplest approach: check who directly imports the component I edited. I set up two test cases with the exact same total impact (4 downstream nodes), but different layouts:
- Flat: comp-a feeds straight into 4 different agents. (4 direct dependents)
- Deep: comp-b feeds into an agent, which feeds into a workflow, then another agent, and so…
在构建任何复杂的东西之前，我尝试了最简单的方法：检查谁直接导入了我编辑的组件。 我设置了两个总影响相同（4 个下游节点）但布局不同的测试用例：
- 扁平化 (Flat)：
comp-a直接输入到 4 个不同的代理中。（4 个直接依赖项） - 深度 (Deep)：
comp-b输入到一个代理中，该代理输入到一个工作流，然后是另一个代理，依此类推……