MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection

MentalMARBERT：用于阿拉伯语心理健康障碍检测的领域自适应预训练与两阶段微调

Abstract: Detecting mental health disorders from Arabic social media text remains challenging due to dialectal variation, informal language, limited high-quality annotated resources, and severe class imbalance. While English mental health natural language processing (NLP) has progressed substantially, Arabic multi-class disorder classification remains insufficiently studied.

摘要： 由于方言差异、非正式语言、高质量标注资源有限以及严重的类别不平衡，从阿拉伯语社交媒体文本中检测心理健康障碍仍然具有挑战性。尽管英语心理健康自然语言处理（NLP）已取得实质性进展，但针对阿拉伯语的多类别障碍分类研究仍显不足。

This study proposes a two-phase framework for Arabic mental health text classification. In phase 1, three Arabic pre-trained language models, AraBERT, CAMeLBERT, and MARBERT, undergo Domain-Adaptive and Task-Adaptive Pretraining (DAPT and TAPT) using a large-scale corpus of unlabeled Arabic mental health tweets. The adapted models are evaluated under a unified protocol to identify the most effective backbone model.

本研究提出了一种用于阿拉伯语心理健康文本分类的两阶段框架。在第一阶段，利用大规模未标注的阿拉伯语心理健康推文语料库，对 AraBERT、CAMeLBERT 和 MARBERT 这三种阿拉伯语预训练语言模型进行领域自适应和任务自适应预训练（DAPT 和 TAPT）。随后，在统一协议下对这些适配后的模型进行评估，以确定最有效的骨干模型。

In phase 2, the selected model is assessed across four configurations combining single-stage and hierarchical two-stage classification architectures with full fine-tuning and Low-Rank Adaptation (LoRA). To support this study, we constructed a novel annotated Arabic mental health dataset comprising 50,670 tweets across six categories, with strong inter-annotator agreement (Krippendorff’s Alpha = 0.733, average pairwise agreement = 0.797).

在第二阶段，研究人员在四种配置下对所选模型进行了评估，这些配置结合了单阶段和分层两阶段分类架构，并分别采用了全量微调和低秩自适应（LoRA）。为支持本研究，我们构建了一个全新的阿拉伯语心理健康标注数据集，包含 50,670 条推文，涵盖六个类别，且具有很高的标注者间一致性（Krippendorff’s Alpha = 0.733，平均成对一致性 = 0.797）。

Experimental results show that the domain-adapted MARBERT (MentalMARBERT) achieves statistically significant improvements over baseline models in both accuracy and macro-F1. The hierarchical two-stage architecture combined with full fine-tuning achieves the best overall performance, reaching a macro-F1 of 0.861 and an accuracy of 0.877. These findings demonstrate the effectiveness of domain-specific adaptive pretraining and hierarchical classification for Arabic mental health disorder detection.

实验结果表明，领域自适应后的 MARBERT（即 MentalMARBERT）在准确率和宏观 F1 值（macro-F1）上均比基准模型有显著的统计学提升。分层两阶段架构结合全量微调取得了最佳的整体性能，宏观 F1 值达到 0.861，准确率达到 0.877。这些发现证明了领域特定自适应预训练和分层分类方法在阿拉伯语心理健康障碍检测中的有效性。