MAMath-Bench: Process-Aware and Robust Evaluation of Multimodal Multi-Agent Mathematical Reasoning

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2610.26943 (cs) [Submitted on 18 Oct 2026] Title: MAMath-Bench: Process-Aware and Robust Evaluation of Multimodal Multi-Agent Mathematical Reasoning Authors: Xinping Wang, Jie Ma, Rui Xing, Wensong Guo, Xiayi Zhang, Jianhao Deng, Zihao Zhou, Boqian Yang, Feijuan He.

计算机科学 > 人工智能 arXiv:2610.26943 (cs) [提交于 2026 年 10 月 18 日] 标题：MAMath-Bench：多模态多智能体数学推理的过程感知与鲁棒性评估作者：Xinping Wang, Jie Ma, Rui Xing, Wensong Guo, Xiayi Zhang, Jianhao Deng, Zihao Zhou, Boqian Yang, Feijuan He。

Abstract: Mathematical reasoning is a fundamental capability for educational applications of Large Language Models (LLMs), where reliable problem solving requires not only correct answers but also coherent and verifiable reasoning processes. Multi-agent systems (MASs) have shown growing potential in this domain by leveraging collaboration and verification among multiple LLM agents.

摘要：数学推理是大型语言模型（LLM）教育应用的一项基本能力，其中可靠的问题解决不仅需要正确的答案，还需要连贯且可验证的推理过程。多智能体系统（MAS）通过利用多个 LLM 智能体之间的协作与验证，在该领域展现出日益增长的潜力。

However, existing evaluations primarily focus on the correctness of final answers in text-only settings, offering limited insight into multimodal reasoning capabilities, the reliability of intermediate reasoning steps, and robustness against reasoning errors. To address these limitations, we introduce MAMath-Bench, a process-aware benchmark designed to evaluate the multimodal mathematical reasoning capabilities of MASs.

然而，现有的评估主要集中在纯文本环境下最终答案的正确性，对于多模态推理能力、中间推理步骤的可靠性以及针对推理错误的鲁棒性，所能提供的见解有限。为了解决这些局限性，我们引入了 MAMath-Bench，这是一个旨在评估 MAS 多模态数学推理能力的过程感知基准。

MAMath-Bench comprises 1,003 problems spanning 11 mathematical categories, multiple difficulty levels, and several modalities, including text, images, and audio. It features fine-grained milestone annotations to evaluate systems across three dimensions: problem-solving correctness, reasoning-process reliability, and robustness to adversarial reasoning-chain perturbations.

MAMath-Bench 包含 1,003 个问题，涵盖 11 个数学类别、多个难度级别以及包括文本、图像和音频在内的多种模态。它具有细粒度的里程碑标注，旨在从三个维度评估系统：问题解决的正确性、推理过程的可靠性以及对对抗性推理链扰动的鲁棒性。

Our evaluation yields three key findings. First, multi-agent collaboration does not consistently improve final-answer accuracy over strong single LLMs. Second, collaboration can substantially enhance the coverage of intermediate reasoning, but such gains may come at the cost of increased reasoning redundancy. Third, current systems remain vulnerable to adversarial reasoning errors, particularly those involving subtle structural perturbations.

我们的评估得出了三个关键发现。首先，多智能体协作并不总是能比强大的单一 LLM 更能提高最终答案的准确性。其次，协作可以显著增强中间推理的覆盖范围，但这种收益可能以增加推理冗余为代价。第三，当前系统仍然容易受到对抗性推理错误的影响，特别是那些涉及细微结构扰动的错误。