Augmentation with Reinforcement Transformation for Low-textured Scientific Sketch Image Retrieval

Computer Science > Computer Vision and Pattern Recognition arXiv:2611.05408 (cs) [Submitted on 21 Oct 2026] Title: Augmentation with Reinforcement Transformation for Low-textured Scientific Sketch Image Retrieval Authors: Zhiqiang Wang, Sheri L. Barton, Jennifer H. Johnson, Yi He, Xingquan Zhu.

计算机科学 > 计算机视觉与模式识别 arXiv:2611.05408 (cs) [提交于 2026 年 10 月 21 日] 标题：用于低纹理科学草图图像检索的强化变换增强技术 作者：Zhiqiang Wang, Sheri L. Barton, Jennifer H. Johnson, Yi He, Xingquan Zhu。

Abstract: Low-textured sketch images are widely used in scientific applications, including wildlife conservation, archaeological illustration, and forensic composites. Learning from and retrieving such images is challenging because they lack detailed surface textures and visual variations commonly exploited by modern deep learning models.

摘要：低纹理草图图像广泛应用于野生动物保护、考古插图和法医合成等科学领域。由于这些图像缺乏现代深度学习模型通常所利用的详细表面纹理和视觉变化，因此对这类图像进行学习和检索具有挑战性。

This paper proposes Augmentation with Reinforcement Transformation (ART) to fine-tune pretrained models to learn effective representations for accurate sketch image retrieval. Our key observation is that sketches are particularly sensitive to image transformations, such as rotation, flipping, zooming, and stretching. While some transformations preserve their structural integrity, others may introduce distortions that hinder representation learning.

本文提出了强化变换增强（ART）技术，用于微调预训练模型，以学习用于精确草图图像检索的有效表征。我们的关键发现是，草图对旋转、翻转、缩放和拉伸等图像变换特别敏感。虽然某些变换能保持其结构完整性，但其他变换可能会引入阻碍表征学习的畸变。

Instead of randomly choosing transformations to construct fine-tuning views, ART formulates augmentation selection as a multi-armed bandit (MAB) problem, where transformations are treated as arms and their effectiveness is continuously evaluated through rewards. This enables ART to adaptively construct reward-guided view banks before self-supervised fine-tuning; the subsequent representation loss does not feed back to ART.

ART 没有随机选择变换来构建微调视图，而是将增强选择公式化为一个多臂老虎机（MAB）问题，其中变换被视为“臂”，其有效性通过奖励进行持续评估。这使得 ART 能够在自监督微调之前自适应地构建奖励引导的视图库；随后的表征损失不会反馈给 ART。

On Florida manatee sketches provided by Mote Marine Laboratory, ART achieves 71.40% average Top-1 accuracy for transformed-scar retrieval, compared with 68.83% for random augmentation selection, and 55.84% for location-aware sketch retrieval, compared with 18.18% for frozen CLIP.

在莫特海洋实验室提供的佛罗里达海牛草图上，ART 在变换疤痕检索方面实现了 71.40% 的平均 Top-1 准确率，而随机增强选择为 68.83%；在位置感知草图检索方面，ART 实现了 55.84% 的准确率，而冻结的 CLIP 模型仅为 18.18%。