LLM Parameters for Math Across Languages: Shared or Separate?

大语言模型跨语言数学推理参数：共享还是独立？

Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-specific parameters or a shared mechanism that manifests differently by language. 大语言模型（LLMs）在数学推理表现上呈现出显著的跨语言差异，但目前尚不清楚这些差异是反映了特定语言的参数，还是反映了一种在不同语言中表现各异的共享机制。

We present a cross-lingual mechanistic analysis of mathematical reasoning in LLMs, enabling us to localize and compare model parameters that support mathematical reasoning across languages. 我们对大语言模型中的数学推理进行了跨语言机制分析，从而能够定位并比较支持不同语言数学推理的模型参数。

We find that the extracted math-associated parameters exhibit partial cross-lingual overlap, with the strongest overlap concentrated in intermediate model layers. 研究发现，提取出的数学相关参数表现出部分跨语言重叠，且最强的重叠集中在模型的中间层。

We further observe that English consistently produces the largest set of math-relevant parameters, whereas lower-resource languages reveal smaller sets of relevant parameters. 我们进一步观察到，英语始终产生最大规模的数学相关参数集，而低资源语言则显示出较小规模的相关参数集。

These results suggest that math-related behavior in multilingual LLMs is neither fully language-invariant nor fully language-specific, but instead exhibits partial cross-lingual parameter overlap with systematic language-dependent differences. 这些结果表明，多语言大语言模型中的数学相关行为既非完全语言无关，也非完全语言特定，而是表现出部分跨语言参数重叠，并伴随系统性的语言依赖差异。