Scaffold-Mediated Post-Training: Co-Evolving Model Parameters and Procedural Scaffold Graphs

支架介导的后训练：模型参数与程序化支架图的协同进化

Abstract: Post-training of large language models optimizes only parameters, while inference-time procedural scaffolds are typically designed independently of parameter training. This disconnect makes it difficult to automatically acquire and internalize complex strategies. 摘要： 大型语言模型的后训练通常仅优化参数，而推理阶段的程序化支架（procedural scaffolds）设计往往独立于参数训练。这种脱节使得模型难以自动获取并内化复杂的策略。

We propose scaffold-mediated post-training: procedural scaffolds are organized into an evolvable graph structure that co-evolves with model parameters through discovery, distillation, and dynamic recompilation. 我们提出了“支架介导的后训练”：将程序化支架组织成一种可进化的图结构，通过发现、蒸馏和动态重编译，使其与模型参数协同进化。

We instantiate this paradigm as Skill Training. On FeatureBench, automatically discovered skills improve the passed rate by 8.1pp, and after progressive distillation the model still achieves a 27.7% passed rate without any external scaffold (distillation retention rate 85.2%, defined as post-distillation / with-skill passed rate), significantly outperforming standard SFT on the same data. 我们将这一范式实例化为“技能训练”（Skill Training）。在 FeatureBench 测试集上，自动发现的技能使通过率提升了 8.1 个百分点；在经过渐进式蒸馏后，模型在没有任何外部支架的情况下仍能达到 27.7% 的通过率（蒸馏保留率为 85.2%，定义为蒸馏后通过率与带技能通过率之比），显著优于在相同数据上进行标准监督微调（SFT）的表现。