OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning
OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning
OmniHarness:通过符号策略学习驾驭可泛化的视觉生成
Unified multimodal large language models (MLLMs) and multi-agent systems have advanced visual generation. However, three limitations remain. (1) Existing methods often distill task-specific experience with limited generalizability. (2) Reflection is often deferred until task completion. (3) Knowledge is often acquired only in response to downstream task demands.
统一的多模态大语言模型(MLLMs)和多智能体系统已经推动了视觉生成技术的发展。然而,目前仍存在三个局限性:(1)现有方法往往提取的是特定任务的经验,泛化能力有限;(2)反思过程通常被推迟到任务完成后才进行;(3)知识的获取往往仅是为了响应下游任务的需求。
To address these limitations, we introduce OmniHarness, a framework for generalizable visual generation via symbolic policy learning. OmniHarness abstracts verified executions into symbolic policies for visual generation task families, capturing shared procedures and applicability conditions while removing instance-specific inputs. The harness instantiates, adapts, and composes these policies for new tasks. Intermediate verification guides refinement and failure recovery during execution.
为了解决这些局限性,我们引入了 OmniHarness,这是一个通过符号策略学习实现可泛化视觉生成的框架。OmniHarness 将经过验证的执行过程抽象为针对视觉生成任务族的符号策略,在剔除实例特定输入的同时,捕捉共享的流程和适用条件。该框架能够为新任务实例化、适配并组合这些策略。中间验证机制则在执行过程中引导策略的优化和故障恢复。
Through self-directed inquiry, OmniHarness autonomously generates and executes practice tasks near its capability limits before downstream objectives are specified. Execution feedback continually refines the policies while model parameters remain fixed.
通过自主探究,OmniHarness 能够在下游目标明确之前,自主生成并执行接近其能力极限的练习任务。执行反馈会不断优化这些策略,而模型参数保持不变。
Experiments across six benchmarks, three MLLM backbones, and three visual agent frameworks demonstrate strong performance and continual capability expansion. On ComfyBench’s Creative tasks, OmniHarness achieves a 95.0% resolve rate, exceeding the strongest baseline by 27.5 percentage points. Frozen policy snapshots improve existing visual agent systems through plug-and-play reuse.
在六个基准测试、三个 MLLM 主干模型和三个视觉智能体框架上的实验表明,该方法表现强劲并具备持续的能力扩展性。在 ComfyBench 的创意任务中,OmniHarness 达到了 95.0% 的解决率,比最强的基准模型高出 27.5 个百分点。冻结的策略快照可以通过即插即用的方式复用,从而提升现有的视觉智能体系统。