When to Ship an AI Feature Behind a Flag

何时应通过功能开关（Feature Flag）发布 AI 功能

Every team already knows how to put a feature behind a flag. What is different here is that the thing most likely to need changing at three in the morning is not whether the feature is on — it is which model it calls, which prompt it uses, and how much it is allowed to do without asking. 每个团队都知道如何通过功能开关（Feature Flag）来发布功能。但这里的不同之处在于，凌晨三点最需要调整的往往不是“功能是否开启”，而是“调用了哪个模型”、“使用了哪个提示词（Prompt）”，以及“在无需确认的情况下允许执行多少操作”。

Why the usual flag is not enough

为什么常规的功能开关是不够的

A conventional feature flag answers one question with a boolean, and it is the right shape because a conventional feature has one failure mode: it is broken. An AI feature has several, and they want different responses. The provider is degraded — you want a different model, not the feature off. A prompt change regressed quality — you want the previous prompt, which is not a code deploy. The feature is fine but a specific customer’s data is producing bad output — you want it off for them and on for everyone else. Spend is running above forecast — you want the cheap model or the degraded path, not an outage. A single boolean answers none of these, so the response to each becomes a deploy, and a deploy is the slowest tool available at the moment you most need speed. 传统的功能开关通过布尔值回答一个问题，这对于传统功能来说是合适的，因为传统功能通常只有一种故障模式：坏了。但 AI 功能有多种故障模式，且需要不同的应对方案。如果服务商性能下降，你想要的是切换模型，而不是关闭功能；如果提示词变更导致质量下降，你想要的是回滚到之前的提示词，这不需要重新部署代码；如果功能正常但特定客户的数据导致了错误输出，你只想为该客户关闭功能，而不影响其他人；如果支出超出预算，你想要的是切换到廉价模型或降级路径，而不是直接停机。单一的布尔值无法解决这些问题，导致每次应对都必须进行代码部署，而当你最需要速度时，部署恰恰是最慢的工具。

There is a second reason, specific to this dependency. The behaviour you are flagging can change without you deploying anything, because the model is somebody else’s and it can be updated underneath you. Flags are usually a mechanism for controlling your own changes; here they are also the mechanism for reacting to changes you did not make, which is why detecting a provider-side behaviour change and having a flag to respond with are two halves of one control. 第二个原因是 AI 功能特有的依赖性。你所控制的功能行为可能会在你没有进行任何部署的情况下发生变化，因为模型属于第三方，他们可能会在后台进行更新。功能开关通常是你控制自身变更的机制；但在 AI 场景下，它也是应对非你所为的变更的机制。因此，监测服务商侧的行为变化并利用开关进行响应，是完整控制体系的两个部分。

Four things to flag separately

四个需要独立控制的维度

Feature on/off: The ordinary flag. Per-tenant and per-segment, because the common case is a problem confined to one customer’s data rather than a global outage. 功能开关（开/关）：常规开关。应支持按租户（Tenant）和分段（Segment）控制，因为常见问题通常仅限于某个客户的数据，而非全局性故障。
Model selection: Which model each call site uses, as configuration. This is what lets you switch providers during an incident, run a canary on a new model, or drop to a cheaper one under budget pressure — without shipping code. 模型选择：将每个调用点使用的模型作为配置项。这让你能够在事故期间切换服务商、对新模型进行灰度测试，或在预算压力下切换到更廉价的模型，而无需发布代码。
Prompt version: Which version of the prompt is served, including the ability to pin the previous one. A prompt rollback is the most common rollback an AI feature needs and the one most likely to require a deploy if nobody planned for it. 提示词版本：当前使用的提示词版本，包括锁定之前版本的能力。提示词回滚是 AI 功能最常见的需求，如果事先没有规划，这往往需要重新部署代码。
Autonomy level: How much the feature may do without confirmation. This is the dial that turns a risky feature into a safe one without turning it off, and it is the most valuable of the four during an incident. 自主级别：功能在无需确认的情况下可以执行多少操作。这是在不关闭功能的前提下，将高风险功能转变为安全功能的调节器，也是事故期间这四个维度中最有价值的一个。

Keeping these independent matters because they are pulled by different people for different reasons. An engineer switches models during a provider incident; a domain owner rolls back a prompt; a support lead disables a tenant; a manager lowers autonomy after a bad week. If all four are one switch, every one of those actions turns the feature off for everybody, and the flag stops being used because its cost is too high. 保持这些维度的独立性至关重要，因为不同的人会出于不同的原因去调整它们。工程师在服务商故障时切换模型；业务负责人回滚提示词；支持主管禁用某个租户；经理在表现不佳的一周后降低自主级别。如果这四个维度合并为一个开关，那么上述任何操作都会导致功能对所有人关闭，最终因为代价过高，这个开关将不再被使用。

One implementation note that pays for itself: the resolved value of all four belongs in the log line for every request, next to the model id and the prompt version. Otherwise you have made behaviour configurable and simultaneously made it impossible to reconstruct what behaviour a given request got. 一个非常值得的实现建议：将这四个维度的解析值记录在每一条请求的日志行中，紧邻模型 ID 和提示词版本。否则，你虽然实现了行为的可配置化，却同时也使得无法还原特定请求当时所处的行为状态。

Rollout criteria

发布标准

A flag with no plan for opening it becomes permanent at one per cent. Write the ramp down before the first user sees the feature; four points, each with an exit condition. 没有发布计划的功能开关最终会永久停留在 1% 的灰度阶段。在第一个用户看到功能之前，请写好发布计划；包含以下四个阶段，每个阶段都有明确的退出条件：

Internal only, no time limit. Exit when the team has used it on real work rather than test input. The purpose is to find the failure modes that only appear on genuine data, and it is free. 仅限内部，无时间限制。 当团队在实际工作中（而非测试输入）使用该功能后即可退出。目的是发现仅在真实数据中才会出现的故障模式，且成本为零。
A small named cohort. Not a random percentage — people you can contact. Exit on qualitative feedback plus the absence of the failure classes you were worried about. Random percentages are for later, when you need statistics rather than explanations. 小规模指定群体。 不要使用随机百分比，而是选择你可以直接联系的用户。当获得定性反馈且你所担心的故障类别未出现时即可退出。随机百分比适用于后期，当你需要统计数据而非解释时。
A percentage ramp with a holdout. Now the statistics matter. Keep a holdout group that never gets the feature, because it is the only way to attribute a change in a business metric to this feature rather than to the season. Exit on the quality, cost and latency metrics you named in advance — the three axes together, since improving one at the expense of the others is the standard way an AI feature looks successful and is not. 带对照组的百分比增长。 此时统计数据变得重要。保留一个永远无法使用该功能的对照组，因为这是将业务指标的变化归因于该功能（而非季节性因素）的唯一方法。根据预先设定的质量、成本和延迟指标退出——这三个维度必须综合考虑，因为以牺牲其中一个为代价来优化另一个，是 AI 功能“看起来成功但实际失败”的典型表现。
Default on, flag retained. Exit — that is, delete the flag — only after a period with no rollbacks and with the replacement control in place. Which, for the model and prompt axes, may be never; those two are configuration rather than a temporary rollout mechanism. 默认开启，保留开关。 仅在经过一段时间且无回滚，并已建立替代控制机制后，才退出（即删除开关）。对于模型和提示词维度，可能永远不会删除；因为它们属于配置，而非临时的发布机制。

The criterion that is easiest to skip and hardest to recover is the holdout. Without it, six months later nobody can say whether the feature helped, and the argument about whether to keep maintaining it is settled by whoever is most confident. 最容易被跳过且最难补救的标准就是“对照组”。没有它，六个月后没人能说清楚该功能是否有帮助，关于是否继续维护它的争论，只能由那个说话声音最大的人说了算。

The kill switch

紧急停机开关（Kill Switch）

A kill switch is not the same control as the flag, and conflating them is why kill switches fail when used. It is defined by four requirements, all of which are about what it must not need. 紧急停机开关与功能开关不是同一个东西，将两者混为一谈是导致停机开关在关键时刻失效的原因。它由四个要求定义，核心在于它“不需要”什么：

No deploy. If pulling it requires a build, it is not a kill switch. This is the requirement everything else follows from. 无需部署。 如果触发它需要构建代码，那它就不是停机开关。这是所有其他要求的基础。
No engineer. The person who notices at 2am is on support. If only the author can pull it, the response time is however long it takes to wake them. 无需工程师。 凌晨两点发现问题的人通常是值班支持人员。如果只有作者能触发它，响应时间就取决于叫醒作者需要多久。
No dependency on the thing being killed. A switch whose evaluation calls the provider, or whose configuration is fetched through the failing path, fails exactly when it is needed. Fail-safe defaults, cached locally, evaluated without a network call. 不依赖于被关闭的对象。 如果开关的评估逻辑需要调用服务商，或者其配置是通过故障路径获取的，那么它恰恰会在最需要的时候失效。应使用故障安全（Fail-safe）默认值，本地缓存，且无需网络调用即可评估。
No error state. Pulling it should produce the degraded feature, not a stack trace. The switch turns the model off, and something else must still answer — which means the degradation path has to exist before the switch is worth having. 无错误状态。 触发它应该产生降级后的功能，而不是堆栈跟踪（Stack Trace）。开关关闭了模型，但系统必须仍能响应——这意味着在拥有停机开关之前，必须先存在降级路径。

And it has to be exercised. A kill switch that has never been pulled in production is a hypothesis. Pull it deliberately during a quiet hour, watch what the feature does, and confirm the degraded path is the one you designed rather than a spinner. Doing this once tends to find at least one thing, most often… 而且它必须经过演练。从未在生产环境中触发过的停机开关只是一个假设。在业务低峰期主动触发它，观察功能表现，确认降级路径是否如你所设计，而不是出现一个转圈的加载动画。做一次演练通常能发现至少一个问题，最常见的是……