A Spectre Is Haunting Unicode
A Spectre Is Haunting Unicode
A Spectre is Haunting Unicode. This post is part of collections on Code and Computers and Japanese Language Technology. In 1978, Japan’s Ministry of Economy, Trade and Industry established the encoding that would later be known as JIS X 0208, which still serves as an important reference for all Japanese encodings. However, after the JIS standard was released, people noticed something strange—several of the added characters had no obvious sources, and nobody could tell what they meant or how they should be pronounced. Nobody was sure where they came from. These are what came to be known as the ghost characters (幽霊文字).
一个幽灵正在 Unicode 中徘徊。本文收录于“代码与计算机”及“日语语言技术”系列。1978 年，日本通商产业省（现经济产业省）确立了后来被称为 JIS X 0208 的编码标准，该标准至今仍是所有日语编码的重要参考。然而，在 JIS 标准发布后，人们注意到了一些奇怪的现象——其中新增的几个字符没有明显的来源，没人知道它们的含义或读音，也没人确定它们从何而来。这些字符后来被称为“幽灵文字”（Ghost Characters）。
For a long time, the ghost characters remained an unexplained and mostly forgotten curiosity, but in 1997 an investigation was launched to discover where they had come from. While all characters in the JIS standard were supposed to have a record of their sources, even when it existed it wasn’t very specific, typically just listing the document it was sourced from. You’d think that listing the source would make tracking down the origins of the characters easy, but it’s important to clarify what counts as a “source”—one of the more common sources for the ghost characters was the “Overview of National Administrative Districts” (国土行政区画総覧), a comprehensive list of place names in Japan. You might, as I initially did, imagine this to be a kind of atlas, an oversize book with at most a few hundred pages. It turns out the latest edition is a seven-volume set with each volume having roughly nine hundred pages. Imagine tracking down a single character without a page reference.
长期以来，这些幽灵文字一直是一个无法解释且几乎被遗忘的奇闻。直到 1997 年，一项调查启动，旨在探究它们的起源。虽然 JIS 标准中的所有字符理论上都应有来源记录，但即便记录存在，也往往不够具体，通常仅列出出处文档。你可能会认为列出来源会让追溯字符起源变得容易，但必须明确什么是“来源”——幽灵文字最常见的来源之一是《国土行政区画总览》，这是一份详尽的日本地名列表。你可能和我最初一样，以为这只是一本类似地图集的大型书籍，最多几百页。但事实证明，其最新版本是一套七卷本，每卷约九百页。试想一下，在没有页码索引的情况下寻找一个字符有多困难。
Despite the difficulty, the investigation into the ghost characters was successful in discovering their origins—mostly. By interviewing the catalogers involved in the creation of the standard, the investigators established that some characters were inadvertently invented as mistakes in the cataloging process. For example, 妛 was an error introduced while trying to record “山 over 女”. “山 over 女” occurs in the name of a particular place and was thus suitable for inclusion in the JIS standard, but because they couldn’t print it as one character yet, 山 and 女 were printed separately, cut out, and pasted onto a sheet of paper, and then copied. When reading the copy, the line where the two little pieces of paper met looked like a stroke and was added to the character by mistake. The original character (𡚴) was not added to JIS or Unicode until much later and doesn’t display on most sites for me.
尽管困难重重，针对幽灵文字的调查还是成功发现了它们的起源——至少大部分是这样。通过采访参与标准制定的编目人员，调查人员确定一些字符是在编目过程中因失误而无意中创造出来的。例如，“妛”字就是在试图记录“山”字下加“女”字时产生的错误。“山”下加“女”出现在某个地名中，因此适合收录进 JIS 标准，但由于当时无法将其作为一个字符打印出来，工作人员便将“山”和“女”分别打印、剪下，粘贴在纸上，然后再进行复印。在阅读复印件时，两张小纸片的接缝处看起来像是一笔画，于是被误当作笔画加到了字符中。原本的字符（𡚴）直到很久以后才被加入 JIS 或 Unicode，且在大多数网站上无法正常显示。
The core ghost characters: 妛挧暃椦槞蟐袮閠駲墸壥彁. In the end, only one character had neither a clear source nor any historical precedent: 彁. The most likely explanation is that it was created as a misreading of the 彊 character, but no specific incident was uncovered. Following the general adoption of the JIS standards, these characters all made their way into Unicode, which has its own separate set of ghost characters introduced during CJK unification.
核心幽灵文字包括：妛、挧、暃、椦、槞、蟐、袮、閠、駲、墸、壥、彁。最终，只有一个字符既没有明确来源，也没有任何历史先例：彁。最可能的解释是它被误读为“彊”字，但并未发现具体的误读事件。随着 JIS 标准的普及，这些字符全部进入了 Unicode，而 Unicode 在中日韩统一表意文字（CJK unification）过程中也引入了自己的一套幽灵文字。
To sum up—in 1978 a series of small mistakes created some characters out of nothing. The errors went undiscovered just long enough to be set in stone, and now these ghosts are, at least in potential, a part of every computer on the planet, lurking in the dark corners of character tables. At this rate, they’ll presumably be with humanity forever.
总结一下：1978 年的一系列小错误凭空创造出了一些字符。这些错误在未被发现的情况下被固化下来，如今，这些幽灵至少在潜能上已成为地球上每一台计算机的一部分，潜伏在字符表的阴暗角落里。照此发展，它们大概会永远伴随着人类。