Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes

超越玩笑：用于检测和解释模因中有害幽默的多角度推理

Abstract: Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme understanding systems that can provide reliable and structured reasoning to support both accurate classification and human interpretability.

摘要： 互联网模因（Memes）交织了视觉线索、文本内容和文化背景，这使得在幽默、讽刺和有害意图并存的情况下，对其进行解读变得极具挑战性。这些复杂性凸显了对可解释模因理解系统的需求，该系统需要提供可靠且结构化的推理，以同时支持准确的分类和人类可解释性。

However, existing multimodal classifiers either overlook these interdependencies or provide only limited interpretability. In this paper, we introduce MAR-12, a novel framework that leverages Vision Language Models (VLMs) for meme detection and understanding in settings where humorous and hateful elements may coexist.

然而，现有的多模态分类器要么忽略了这些相互依赖关系，要么仅提供有限的可解释性。在本文中，我们引入了 MAR-12，这是一个利用视觉语言模型（VLM）的新型框架，用于在幽默和仇恨元素可能并存的环境中进行模因检测和理解。

The framework first interprets each meme through twelve structured perspectives derived from humor and hate theories. It then applies a role-aware soft-gated attention mechanism to learn how much each perspective should contribute, followed by a prototype-based classifier for the final prediction.

该框架首先通过源自幽默和仇恨理论的十二个结构化视角来解读每个模因。随后，它应用了一种角色感知的软门控注意力机制（role-aware soft-gated attention mechanism）来学习每个视角应贡献的权重，并结合基于原型的分类器进行最终预测。

Finally, explanations are synthesized using both perspective-specific reasoning and learned attention weights, ensuring transparent and context-grounded justifications. We evaluate MAR-12 on the PrideMM and Memotion datasets, where it achieves up to 80.3% accuracy for humor detection and 75.9% accuracy for hate detection, outperforming state-of-the-art approaches.

最后，通过结合特定视角的推理和学习到的注意力权重来综合生成解释，从而确保了透明且基于上下文的论证。我们在 PrideMM 和 Memotion 数据集上对 MAR-12 进行了评估，其幽默检测准确率高达 80.3%，仇恨检测准确率达到 75.9%，优于目前最先进的方法。

Furthermore, both human and GPT-4-based evaluations confirm that MAR-12 produces coherent and persuasive explanations, particularly for memes in which humorous and harmful cues co-occur.

此外，人类评估和基于 GPT-4 的评估均证实，MAR-12 能够生成连贯且具有说服力的解释，特别是在幽默和有害线索并存的模因中表现尤为突出。