FusionEval: Benchmarking Model Fusion for Large Language Models

查看原文 / View Original 编辑 / Edit

Computer Science > Computation and Language arXiv:2610.30162 (cs) [Submitted on 30 Sep 2026] Title: FusionEval: Benchmarking Model Fusion for Large Language Models Authors: Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang.

计算机科学 > 计算与语言 arXiv:2610.30162 (cs) [提交于 2026 年 9 月 30 日] 标题：FusionEval：大语言模型模型融合基准测试作者：Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang, Yang Yu, Zhen Li, Zhitian Hou, Guanghao Zhu, Ming Li, Hongxia Yang。

Abstract: Model fusion aims to integrate the capabilities of multi-expert models into a single model. Existing evaluations offer limited coverage of reasoning with long chains of thought and tool use, while strong average performance can mask substantial losses in individual expert capabilities.

摘要：模型融合旨在将多专家模型的能力整合到一个单一模型中。现有的评估对长思维链推理和工具使用的覆盖范围有限，且强劲的平均性能可能会掩盖单个专家能力的重大损失。

We introduce FusionEval, a benchmark that provides controlled expert pools, integrates fusion methods with and without additional training, and offers a unified evaluation suite. The pools contain domain experts independently trained from Qwen3 and Llama base models across math, code, science, instruction following, and tool use.

我们引入了 FusionEval，这是一个提供受控专家池、整合了有无额外训练的融合方法，并提供统一评估套件的基准测试。这些专家池包含基于 Qwen3 和 Llama 基础模型独立训练的领域专家，涵盖数学、代码、科学、指令遵循和工具使用。

The suite evaluates capability retention and computational cost on eight public benchmarks under comparable data access rules, reporting normalized average and worst performance, runtime, and peak GPU memory usage. Extensive experiments show that training-free methods preserve reasoning performance but substantially degrade tool use at both model scales.

该套件在可比较的数据访问规则下，对八个公共基准测试的能力保持情况和计算成本进行了评估，报告了归一化的平均性能、最差性能、运行时间和峰值 GPU 内存使用量。大量实验表明，无需训练的方法在两种模型规模下都能保持推理性能，但会显著降低工具使用能力。

In contrast, MOPD preserves tool use and achieves a more balanced capability profile, but requires additional data preparation, inference, and training. Jointly retaining reasoning, instruction following, and tool use at low computational cost remains a challenge. We hope FusionEval supports future research on efficient model fusion that jointly preserves expert capabilities.

相比之下，MOPD 保留了工具使用能力并实现了更均衡的能力分布，但需要额外的数据准备、推理和训练。在低计算成本下同时保持推理、指令遵循和工具使用能力仍然是一个挑战。我们希望 FusionEval 能支持未来关于同时保持专家能力的高效模型融合研究。