ZeroR@CHiPSAL 2026: Two-Stage Vision-Language Adaptation with Contrastive Learning for Nepali Meme Classification

ZeroR@CHiPSAL 2026：用于尼泊尔语模因分类的对比学习两阶段视觉-语言适配

---

Abstract: This paper presents our system for the CHiPSAL 2026 shared task on multimodal hate speech and sentiment detection in Nepali memes. We address both subtasks: binary hate speech classification and three-class sentiment analysis. Our approach adapts the Robust Adaptation of Hateful Meme Detection (RA-HMD) framework using Qwen3-VL-8B-Instruct, a state-of-the-art vision-language model with native Devanagari support.

摘要： 本文介绍了我们为 CHiPSAL 2026 共享任务开发的系统，该任务旨在检测尼泊尔语模因中的多模态仇恨言论和情感。我们处理了两个子任务：二元仇恨言论分类和三分类情感分析。我们的方法采用了 Qwen3-VL-8B-Instruct 模型，这是一款支持原生天城体（Devanagari）的最先进视觉-语言模型，并对“仇恨模因检测鲁棒适配”（RA-HMD）框架进行了适配。

We employ a two-stage training pipeline: (1) LoRA fine-tuning with an MLP projection head for generative classification, and (2) contrastive backbone fine-tuning with supervised InfoNCE loss. We handle class imbalance through minority oversampling, image augmentation, and focal loss. At inference, we ensemble Stage 1 token probabilities with Stage 2 classifier scores using validation-tuned weights.

我们采用了两阶段训练流程：（1）使用带有 MLP 投影头的 LoRA 微调进行生成式分类；（2）使用监督式 InfoNCE 损失进行对比骨干网络微调。我们通过少数类过采样、图像增强和焦点损失（focal loss）来处理类别不平衡问题。在推理阶段，我们利用验证集调整后的权重，将第一阶段的 Token 概率与第二阶段的分类器得分进行集成。

Our end-to-end approach eliminates error propagation from separate OCR and translation pipelines by leveraging the model’s native Devanagari understanding. Our system achieved 2nd place on hate speech detection (F1: 0.797) and 4th place on sentiment analysis (F1: 0.518). We provide detailed ablations, error analysis, and insights into adapting large vision-language models for low-resource South Asian languages.

我们的端到端方法利用模型对天城体的原生理解能力，消除了独立 OCR 和翻译流程带来的误差传播。我们的系统在仇恨言论检测任务中获得第 2 名（F1: 0.797），在情感分析任务中获得第 4 名（F1: 0.518）。我们提供了详细的消融实验、误差分析，以及关于如何将大型视觉-语言模型适配于低资源南亚语言的见解。