From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors

Computer Science > Artificial Intelligence arXiv:2610.36146 (cs) [Submitted on 26 Oct 2026] Title: From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors Authors: Wentao Ye, Yali Ye, Zhiqing Xiao, Ru Peng, Xinpeng Ti, Liyao Li, Zhanming Shen, Peng Lu, Jie Zhang, Haobo Wang.

计算机科学 > 人工智能 arXiv:2610.36146 (cs) [提交于 2026 年 10 月 26 日] 标题：从结构反馈到提示策略：学习忠实的文本到图像提示编辑器 作者：Wentao Ye, Yali Ye, Zhiqing Xiao, Ru Peng, Xinpeng Ti, Liyao Li, Zhanming Shen, Peng Lu, Jie Zhang, Haobo Wang。

Abstract: Prompt optimization should improve the image a user receives without changing the scene the user requested. Yet stylistic rewriting can weaken object attributes and relations even when global text-image similarity remains high. We propose StruPO, a structure-aware prompt editor that uses a calibrated scene graph as a shared representation for policy conditioning and image-grounded supervision.

摘要：提示优化应该在不改变用户所请求场景的前提下改善用户接收到的图像。然而，即使全局文本与图像的相似度保持较高，风格化的重写也可能削弱对象的属性和关系。我们提出了 StruPO，这是一种结构感知的提示编辑器，它使用校准后的场景图作为策略调节和基于图像监督的共享表示。

The graph exposes what should be preserved, while deletion, reordering, and insertion provide a token-aligned way to modify the prompt. This turns structural feedback from an external critique into a training signal for a reusable editing policy. Learning from that signal requires handling both uncertain grounding and stochastic image generation.

该图揭示了应该保留的内容，而删除、重排序和插入则提供了一种与标记对齐的提示修改方式。这使得结构反馈从一种外部批评转变为可重用编辑策略的训练信号。从该信号中学习需要同时处理不确定的基础（grounding）和随机的图像生成。

StruPO retains grounding confidence through smooth reward shaping and averages multiple images for each edit before computing group-relative advantages. Experiments span three disjoint 2K-prompt test sets and four frozen generation backbones, with comparisons to learned prompt optimizers, iterative structural feedback, diffusion-model fine-tuning, and inference-time attention control.

StruPO 通过平滑的奖励塑造来保持基础置信度，并在计算组相对优势之前对每次编辑的多张图像进行平均。实验涵盖了三个不相交的 2K 提示测试集和四个冻结的生成主干，并与学习到的提示优化器、迭代结构反馈、扩散模型微调以及推理时注意力控制进行了比较。

StruPO improves the faithfulness-aesthetics trade-off, with consistent gains in relational fidelity. Independent automatic evaluators and a 15-annotator study provide complementary evidence, and human preference margins increase with relation complexity. Controlled ablations connect these gains to graph conditioning, local editing, and image-grounded learning.

StruPO 改善了忠实度与美观度之间的权衡，并在关系保真度方面取得了持续的提升。独立的自动评估器和一项由 15 名标注者参与的研究提供了补充证据，且人类偏好差距随着关系复杂性的增加而扩大。受控消融实验将这些收益与图调节、局部编辑和基于图像的学习联系起来。

The resulting policy produces reusable optimized text in one pass after graph calibration, without an iterative critique-rewrite loop.

由此产生的策略在图校准后只需一次传递即可生成可重用的优化文本，无需迭代的“批评-重写”循环。