Beyond CLIP: Calibrated (Few-Shot) Defect Detection With System One Decision Models

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.41199 (cs) [Submitted on 30 Oct 2026] Title: Beyond CLIP: Calibrated (Few-Shot) Defect Detection With System One Decision Models Authors: Fadel M. Megahed, Ying-Ju Chen.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.41199 (cs) [提交于 2026 年 10 月 30 日] 标题：超越 CLIP：基于系统一决策模型的校准（少样本）缺陷检测 作者：Fadel M. Megahed, Ying-Ju Chen。

Abstract: Cameras and vision sensors are widely used for product and process inspection in manufacturing environments. Traditionally, this process required thousands of images of both nominal and defective products to train such models. Recent research shows that vision transformer (ViT) models, such as CLIP, can reduce this requirement to 50-100 images per nominal and defective class. However, collecting even 50 defective images can be difficult in high-yield or low-volume manufacturing processes.

摘要：摄像头和视觉传感器被广泛应用于制造环境中的产品和过程检测。传统上，这一过程需要数千张正常产品和缺陷产品的图像来训练模型。最近的研究表明，视觉转换器（ViT）模型（如 CLIP）可以将这一需求降低到每个正常和缺陷类别 50-100 张图像。然而，在高良率或小批量的制造过程中，即使收集 50 张缺陷图像也可能非常困难。

In this paper, we examine the utility of two recently proposed structured decision models (Jev and Clef-Flash) in applications using as few as k=2 reference images per class at inference. In the process, we provide three main contributions: (a) a framework for k-shot learning of these models for image classification/inspection applications; (b) an agentic approach for automatically refining the text descriptions that guide image classification in these decision models; and (c) an approach to calibrate the resulting defect probabilities to improve their reliability for inspection decisions.

在本文中，我们研究了两种近期提出的结构化决策模型（Jev 和 Clef-Flash）在推理阶段每类仅使用 k=2 张参考图像的应用效能。在此过程中，我们做出了三项主要贡献：（a）一个用于图像分类/检测应用中这些模型的 k-shot 学习框架；（b）一种用于自动优化指导这些决策模型中图像分类的文本描述的代理方法；以及（c）一种校准所得缺陷概率以提高其检测决策可靠性的方法。

We test these contributions in five manufacturing case studies. Using two reference images per class, both Jev and Clef-Flash outperform CLIP’s test accuracies with the same references (0.75-1.00 [Jev] and 0.71-0.99 [Clef-Flash] vs 0.67-0.84 [CLIP]). Furthermore, we show that our agentic refinement of text descriptions significantly improves accuracy across 6 of 8 model-application combinations, and our calibration approach reduces the Brier score in 13 of the 15 model-application configurations.

我们在五个制造案例研究中测试了这些贡献。在使用每类两张参考图像的情况下，Jev 和 Clef-Flash 在相同参考集下的测试准确率均优于 CLIP（Jev 为 0.75-1.00，Clef-Flash 为 0.71-0.99，而 CLIP 为 0.67-0.84）。此外，我们证明了我们的文本描述代理优化方法在 8 种模型应用组合中的 6 种里显著提高了准确率，并且我们的校准方法在 15 种模型应用配置中的 13 种里降低了 Brier 分数。

To support the reproducibility of our work, we provide our replication code and have created the R “decisions” package for evaluating additional models. We also provide a publicly available web app that allows practitioners to inspect their own images with no code.

为了支持我们工作的可重复性，我们提供了复现代码，并创建了用于评估其他模型的 R 语言“decisions”包。我们还提供了一个公开的 Web 应用程序，允许从业者无需编写代码即可检测他们自己的图像。