Learning to Read Braille: Benchmarking and Improving Braille Recognition Capabilities of VLMs

查看原文 / View Original 编辑 / Edit

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.36195 (cs) [Submitted on 26 Oct 2026] Title: Learning to Read Braille: Benchmarking and Improving Braille Recognition Capabilities of VLMs Authors: Kartik Jangra, Atharva Gawane, Siddharth Pulipaka, Hemachandran Karah, Raj Dabre.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.36195 (cs) [提交于 2026 年 10 月 26 日] 标题：学习阅读盲文：基准测试并改进视觉语言模型（VLM）的盲文识别能力作者：Kartik Jangra, Atharva Gawane, Siddharth Pulipaka, Hemachandran Karah, Raj Dabre。

Abstract: Despite rapid progress in visual understanding and document transcription, modern vision-language models (VLMs) struggle with a fundamental accessibility task: reading Braille from images. We introduce BrailleBench, a manually curated benchmark of 259 real-world Grade-1 and Grade-2 Braille images. We evaluate proprietary, open-weight, OCR-specialized, and tool-assisted multimodal systems and find that standalone general-purpose models perform poorly at directly transcribing Braille images, whereas tool-assisted systems perform much better.

摘要：尽管在视觉理解和文档转录方面取得了快速进展，但现代视觉语言模型（VLM）在处理一项基本的无障碍任务时仍面临困难：从图像中读取盲文。我们引入了 BrailleBench，这是一个包含 259 张真实世界一级和二级盲文图像的人工策划基准。我们评估了专有模型、开放权重模型、OCR 专用模型以及工具辅助的多模态系统，发现独立的通用模型在直接转录盲文图像方面表现不佳，而工具辅助系统的表现则好得多。

We design a synthetic Braille generation pipeline with diverse layouts and image perturbations. A 2B VLM model trained on our synthetic data reaches 87% exact match on Grade-1 and 61% on Grade-2 BrailleBench, outperforming standalone VLMs and achieving the highest Grade-2 exact match score among the evaluated systems.

我们设计了一个具有多样化布局和图像扰动的合成盲文生成流程。一个在我们的合成数据上训练的 2B VLM 模型在 BrailleBench 的一级盲文上达到了 87% 的精确匹配率，在二级盲文上达到了 61%，优于独立的 VLM，并在所有评估系统中获得了最高的二级盲文精确匹配分数。

We further show that high accuracy on synthetic validation data does not guarantee high accuracy on real photographs in our experiments: models trained on prior synthetic data reach up to 96% on their synthetic validation set but at most 3.26% on Grade-1 BrailleBench. Our results identify an accessibility-relevant capability gap in multimodal systems and show that targeted synthetic supervision can improve real-world Braille transcription.

我们进一步证明，在我们的实验中，合成验证数据上的高准确率并不能保证在真实照片上也能获得高准确率：在先前合成数据上训练的模型在其合成验证集上最高可达 96%，但在 BrailleBench 一级盲文上最高仅为 3.26%。我们的结果确定了多模态系统中与无障碍相关的能力差距，并表明有针对性的合成监督可以改善现实世界中的盲文转录。