Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

基于流形覆盖与稀疏特征覆盖的大语言模型后训练分层数据选择

Abstract: As supervised fine-tuning data continues to scale, selecting high-value subsets from large candidate pools is crucial for reducing training cost and improving model performance.

摘要： 随着监督微调数据规模的持续扩大，从庞大的候选池中筛选出高价值子集对于降低训练成本和提升模型性能至关重要。

Existing methods often measure diversity directly in the original embedding space, where geometric metrics entangle dominant semantic directions, fine-grained supervision differences, and local noise.

现有方法通常直接在原始嵌入空间中衡量多样性，但这种几何度量方式往往会将主导语义方向、细粒度监督差异以及局部噪声混杂在一起。

We address this limitation by formulating data selection as a coarse-to-fine hierarchical coverage problem and propose MASS.

为了解决这一局限性，我们将数据选择问题建模为一种从粗到细的分层覆盖问题，并提出了 MASS 方法。

MASS learns low-dimensional principal manifold coordinates with a dense autoencoder for coarse semantic grouping, and then performs quality-aware sparse feature coverage within each group using a TopK sparse autoencoder.

MASS 利用密集自动编码器学习低维主流形坐标以进行粗粒度的语义分组，随后在每个组内使用 TopK 稀疏自动编码器执行质量感知的稀疏特征覆盖。

Experiments on Vision Flan and LLaVA-CoT show that MASS consistently outperforms strong data selection baselines across multiple budgets, and in several settings matches or surpasses full data training with only a small subset of data.

在 Vision Flan 和 LLaVA-CoT 上的实验表明，MASS 在多种预算设置下均持续优于强有力的数据选择基准方法，并且在某些设置中，仅使用一小部分数据即可达到甚至超过全量数据训练的效果。