I’m Sorry, but I Can’t Help with Braille: Revealing Accessibility Failures in State-of-the-Art LLMs

很抱歉，我无法处理盲文：揭示顶尖大语言模型在无障碍功能上的缺陷

Abstract: Large Language Models (LLMs) perform strongly on many language tasks, but their capability in structurally constrained, accessibility-critical modalities such as Braille remains unclear. 摘要： 大语言模型（LLM）在许多语言任务中表现出色，但它们在盲文（Braille）等结构受限且对无障碍至关重要的模态中的能力尚不明确。

We evaluate state-of-the-art LLMs on bidirectional Korean-Braille translation using a human-annotated dataset. Despite expectations that multilingual, instruction-tuned models can generalize to Braille via text representations, we find consistently poor, unstable outputs and substantial disagreement with human judgments. 我们使用人工标注的数据集，对顶尖大语言模型在韩语与盲文的双向翻译能力进行了评估。尽管人们期望经过指令微调的多语言模型能够通过文本表示泛化到盲文处理，但我们发现其输出结果始终较差且不稳定，并与人类判断存在显著差异。

These results point to missing Braille-aware tokenization and weak alignment between Korean and Braille patterns. In contrast, supervised fine-tuning of a small model (T5-small) on the same data yields large and stable gains over zero-shot and prompted LLM baselines across standard metrics (SacreBLEU, ChrF++, CER, BLEU, ROUGE-L, METEOR, CIDEr). 这些结果表明，模型缺乏对盲文的感知分词能力，且韩语与盲文模式之间的对齐较弱。相比之下，在相同数据上对小型模型（T5-small）进行监督微调，在各项标准指标（SacreBLEU、ChrF++、CER、BLEU、ROUGE-L、METEOR、CIDEr）上均比零样本（zero-shot）和提示词（prompted）大语言模型基准取得了显著且稳定的提升。

Our findings reveal a systematic limitation of current LLMs and demonstrate the effectiveness of modest task-specific supervision. 我们的研究结果揭示了当前大语言模型的系统性局限，并证明了适度的特定任务监督是行之有效的。