Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B
Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B
Semalith v1.4：一款经过校准的 1.84 亿参数安全分类器，在参数量减少 44 倍的情况下实现了优于 Llama-Guard-3-8B 的提示注入检测性能
Abstract / 摘要
Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses in a single inference pass. Semalith v1.4 is a 184M-parameter DeBERTa-v3-base classifier performing simultaneous three-axis safety classification including prompt injection, general harm, and financial-services regulatory compliance, in a single forward pass.
在金融服务和智能体（Agentic）场景中部署大语言模型，需要安全分类器能够同时处理提示注入（Prompt Injection）、合规性要求以及通用危害。目前尚无现有的开源护栏模型能在单次推理过程中同时解决这些问题。Semalith v1.4 是一款基于 DeBERTa-v3-base 的 1.84 亿参数分类器，它能在单次前向传播中同时执行包括提示注入、通用危害和金融服务合规性在内的三轴安全分类。
Its 22-class head (BENIGN, nine prompt-injection sub-types, general-harm, eleven BFSI labels) is trained with a 4-class auxiliary super-category head under jointly weighted loss, on a 76,204-row corpus mined from 49 public sources with SHA-1 deduplication against every held-out evaluation set, with 21 of 22 benchmarks at zero contamination (max 0.22%).
该模型拥有一个 22 类的分类头（包含良性、9 种提示注入子类型、通用危害以及 11 种银行与金融服务行业（BFSI）标签），并配有一个 4 类的辅助超类别分类头，采用联合加权损失函数进行训练。训练语料库包含 76,204 条数据，挖掘自 49 个公共来源，并通过 SHA-1 去重技术确保与所有留存评估集无重复，在 22 个基准测试中，有 21 个实现了零污染（最高污染率仅为 0.22%）。
Against Llama-Guard-3-8B on 22 held-out benchmarks, Semalith v1.4 wins every prompt-injection evaluation (7/7) and 11 of 18 benchmarks overall at 44x fewer parameters, with FPR = 0.000 on 208 benign agentic prompts vs 0.063 for Llama-Guard-3-8B. On general-harm benchmarks (WildGuardMix, HEx-PHI, HarmBench), Llama-Guard-3 leads; this complementary split is documented in Section 4. Six measured weak spots are disclosed in Section 6.
在 22 个留存基准测试中，与 Llama-Guard-3-8B 相比，Semalith v1.4 在参数量减少 44 倍的情况下，赢得了所有提示注入评估（7/7）以及总计 18 个基准测试中的 11 个。在 208 个良性智能体提示词测试中，其误报率（FPR）为 0.000，而 Llama-Guard-3-8B 为 0.063。在通用危害基准测试（WildGuardMix, HEx-PHI, HarmBench）中，Llama-Guard-3 表现更优；这种互补性差异在论文第 4 节中有详细记录。第 6 节则披露了该模型的 6 个已知弱点。
Deployment guidance: v1.3 is recommended for conversational moderation deployments (ToxicChat F1 0.624); v1.4 is recommended when BFSI label coverage or zero-FPR on benign agentic prompts is the priority.
部署建议：v1.3 版本推荐用于对话审核场景（ToxicChat F1 分数为 0.624）；当 BFSI 标签覆盖率或在良性智能体提示词上实现零误报率（Zero-FPR）为首要任务时，推荐使用 v1.4 版本。