SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning

SLAP：用于在线策略数据高效指令微调的分层损失修剪方法

Abstract: Instruction tuning has optimized the specialized capabilities of large language models (LLMs), but it often requires extensive datasets and prolonged training times. The challenge lies in developing specific capabilities by identifying useful data and efficiently fine-tuning. High-quality and diverse pruned data can help models achieve lossless performance at a lower cost.

摘要： 指令微调优化了大语言模型（LLMs）的专业能力，但通常需要庞大的数据集和漫长的训练时间。当前的挑战在于如何通过识别有用数据并进行高效微调来培养特定的能力。高质量且多样化的修剪数据可以帮助模型以更低的成本实现无损性能。

In this paper, we propose SLAP, a novel batch-aware data selection framework that evaluates the learnability of entire batch compositions rather than individual samples. SLAP ensures comprehensive data distribution coverage through distribution-aware stratified sampling while maximizing intra-batch diversity through relative distance optimization.

在本文中，我们提出了 SLAP，这是一个新颖的批次感知数据选择框架，它评估的是整个批次组合的可学习性，而非单个样本。SLAP 通过分布感知的分层采样确保了数据分布的全面覆盖，同时通过相对距离优化最大化了批次内的多样性。

By leveraging Hessian-approximated gradient information for dynamic batch selection, SLAP significantly outperforms existing state-of-the-art methods across multiple model architectures (LLaMA, ChatGLM) and diverse downstream tasks including multi-turn dialogue, multilingual translation, and question answering.

通过利用海森矩阵近似（Hessian-approximated）梯度信息进行动态批次选择，SLAP 在多种模型架构（LLaMA、ChatGLM）和包括多轮对话、多语言翻译及问答在内的多种下游任务中，显著优于现有的最先进方法。

Most notably, SLAP achieves superior performance with 20-40% less training data compared to full dataset training, substantially reducing computational costs while maintaining or improving model capabilities. These results establish SLAP as a powerful approach for efficient and effective instruction tuning of large language models.

最值得注意的是，与全数据集训练相比，SLAP 在减少 20-40% 训练数据的情况下实现了更优的性能，在保持或提升模型能力的同时，大幅降低了计算成本。这些结果表明，SLAP 是一种用于大语言模型高效且有效指令微调的强大方法。