Local Edits, Global Ripples: Replay-Informed Policy Adaptation for Workflow Synthesis

局部编辑，全局涟漪：用于工作流合成的重放感知策略自适应

Abstract: Prompt-policy editing offers a practical way to improve agents that synthesize executable workflows without updating the underlying model. However, persistent prompt editing has two coupled properties. First, edit locality does not imply effect locality: an edit confined to one policy segment can ripple through downstream execution, altering behavior beyond the edited segment. Second, edit effects are composition-sensitive: edits that work in isolation can interfere after composition, causing one or both to lose their benefit or become harmful.

摘要： 提示词策略编辑（Prompt-policy editing）为改进合成可执行工作流的智能体提供了一种实用方法，且无需更新底层模型。然而，持久化提示词编辑具有两个相互关联的特性。首先，编辑的局部性并不意味着效果的局部性：局限于某个策略片段的编辑可能会在下游执行中产生涟漪效应，从而改变该片段之外的行为。其次，编辑效果对组合敏感：在孤立状态下有效的编辑在组合后可能会产生干扰，导致其中一个或两个编辑失去增益甚至产生负面影响。

Persistent adaptation must therefore support two distinct decisions: identifying where the policy should change from execution feedback, and determining whether the resulting edit remains safe to persist after composition. To address these challenges, we introduce RIPPLE (Replay-Informed Persistent Policy Localization and Editing), which separates where an edit is made from whether it remains safe after composition. It diagnoses failed trajectories, maps each actionable failure to a predefined policy segment, and restricts the correction to that part of the policy.

因此，持久化自适应必须支持两个不同的决策：根据执行反馈识别策略应在何处进行更改，以及确定由此产生的编辑在组合后是否仍然安全。为了应对这些挑战，我们引入了 RIPPLE（重放感知持久化策略定位与编辑），它将“在何处进行编辑”与“编辑在组合后是否安全”分离开来。它能够诊断失败的轨迹，将每个可操作的失败映射到预定义的策略片段，并将修正限制在该策略部分。

RIPPLE then evaluates candidates against the same iteration-start policy to compare their isolated gains, before replaying promising edits after previously accepted updates to expose downstream effects and interactions. Only edits that remain safe under composition are retained. We evaluate RIPPLE on Flow-HO, a synthetic held-out benchmark for executable workflow synthesis. RIPPLE improves validation success by up to 23.1% and yields positive gains on two additional frozen language-model backbones, while maintaining edit efficiency and low execution cost.

随后，RIPPLE 会针对相同的迭代起始策略评估候选方案，以比较它们的孤立增益，并在之前已接受的更新之后重放有前景的编辑，从而揭示下游的影响和交互。只有在组合下保持安全的编辑才会被保留。我们在 Flow-HO（一个用于可执行工作流合成的合成留出基准）上评估了 RIPPLE。RIPPLE 将验证成功率提高了 23.1%，并在另外两个冻结的语言模型主干上取得了积极的增益，同时保持了编辑效率和较低的执行成本。

Targeted interaction analysis further demonstrates both properties: a segment-local tool-use edit changes downstream resource resolution and validation, while an edit beneficial in isolation becomes harmful after composition.

针对性的交互分析进一步证明了这两个特性：一个片段局部的工具使用编辑改变了下游的资源解析和验证，而一个在孤立状态下有益的编辑在组合后却变得有害。