Pangram Has Emerged as the Gold Standard of AI Detection. Should You Trust It?
Pangram Has Emerged as the Gold Standard of AI Detection. Should You Trust It?
Pangram 已成为 AI 检测的黄金标准，你该信任它吗？
Pangram is an AI startup with 24 employees, headquartered above a Popeyes in Brooklyn. It has raised $13 million to date—about 0.0072 percent of what OpenAI has—and was virtually unknown until earlier this year. But if Pangram is to be believed, it’s the one force standing in the way of a full machine takeover of the written word.
Pangram 是一家拥有 24 名员工的 AI 初创公司，总部位于布鲁克林一家 Popeyes 快餐店的楼上。迄今为止，该公司已筹集了 1300 万美元——大约是 OpenAI 融资额的 0.0072%——在今年年初之前几乎默默无闻。但如果 Pangram 的说法可信，它就是目前阻挡机器全面接管文字创作的唯一力量。
Pangram promises to detect how much AI was involved in the generation of any given text, and it does so as a best-guess percentage (using AI, of course). In January, speculation swirled on Reddit and YouTube that the author Mia Ballard may have used AI to write Shy Girl, a self-published novel picked up for traditional publication by Hachette. Although Ballard denied it, Pangram’s CEO posted on X that the book was 78 percent AI-generated. Hachette later canceled the release.
Pangram 承诺能够检测出给定文本中 AI 参与生成的比例，并以“最佳猜测百分比”的形式呈现（当然，这是通过 AI 实现的）。今年 1 月，Reddit 和 YouTube 上流传着一种猜测，认为作家 Mia Ballard 可能使用 AI 撰写了自出版小说《Shy Girl》，该书此前已被阿歇特图书集团（Hachette）签下准备进行传统出版。尽管 Ballard 予以否认，但 Pangram 的首席执行官在 X 上发文称，该书有 78% 的内容是由 AI 生成的。随后，阿歇特取消了该书的发行。
A barrage of similar accusations followed. The New York Times was called out for running an AI-generated installment of its Modern Love column. Pangram score: 100 percent. Then came a winner of the Commonwealth Short Story Prize (100 percent), the novel Daggermouth (60 percent), and a thriller titled Call Me, I’ll Hide the Body that had sold for $2.4 million (97 percent). In late July, Substack announced it was integrating Pangram into its platform so readers could quickly determine possible AI use.
随后，一系列类似的指控接踵而至。《纽约时报》因刊登了一篇由 AI 生成的《现代爱情》（Modern Love）专栏文章而受到质疑。Pangram 给出的评分是：100%。紧接着，英联邦短篇小说奖的一位获奖者（100%）、小说《Daggermouth》（60%），以及以 240 万美元售出的惊悚小说《Call Me, I’ll Hide the Body》（97%）也纷纷“中招”。7 月下旬，Substack 宣布将 Pangram 集成到其平台中，以便读者能快速判断内容是否存在 AI 使用痕迹。
Not all writers see this as a good thing. “There is such distaste and anger at the AI detection software,” says Jane Friedman, an author and publishing expert. “There’s this feeling like they are just as evil, if not more evil, than the AI companies themselves.” Outside of publishing and higher education, Pangram isn’t a household name, but I suspect it’s about to be; the demand to distinguish LLM churn from human writing grows with each passing day. The question to ask now is: How much can the company be trusted?
并非所有作家都认为这是一件好事。“人们对 AI 检测软件感到非常厌恶和愤怒，”作家兼出版专家 Jane Friedman 表示，“大家有一种感觉，认为这些检测公司和 AI 公司一样邪恶，甚至可能更甚。”在出版和高等教育领域之外，Pangram 并非家喻户晓，但我怀疑它很快就会变得广为人知；随着区分大语言模型（LLM）生成内容与人类写作的需求日益增长，现在的问题是：这家公司到底有多值得信任？
Max Spero, Pangram’s 30-year-old cofounder and CEO, dials into our Google Meet from his phone, takeout box in one hand and skyscrapers in the background. He’s wearing a beige T-shirt and has short dark hair, a boyish expression, and a wide smile. He says he’s rushing home with lunch and will call back. Ten minutes later, Spero reappears inside his Brooklyn apartment. He’s been on hiring calls all morning, he says. In July, Pangram raised $9 million and announced the launch of its newest model, Pangram 4. The company’s website lists six new job openings, which would increase headcount by 25 percent.
Pangram 30 岁的联合创始人兼首席执行官 Max Spero 通过手机接入了我们的 Google Meet 会议，他一手拿着外卖盒，背景是摩天大楼。他穿着一件米色 T 恤，留着深色短发，表情稚气，笑容灿烂。他说他正赶着回家吃午饭，稍后会回拨。十分钟后，Spero 在他的布鲁克林公寓里重新上线。他说他整个上午都在忙着面试招聘。7 月，Pangram 筹集了 900 万美元，并宣布推出其最新模型 Pangram 4。公司网站列出了六个新职位空缺，这将使员工人数增加 25%。
I ask Spero about growing up in California, a softball question that makes him visibly uncomfortable. Discussing the nuances of Pangram’s tech comes more naturally to him than answering personal queries—or speaking to the company’s involvement in literary scandals. Spero takes a bite of his meal and says he was raised in the Los Angeles suburb of La Crescenta. He loved programming and joined his school’s robotics team. As an undergrad at Stanford, he met Bradley Emi, his eventual Pangram cofounder. (My several emailed requests to interview Emi were ignored by the company’s comms team.) After school, Spero went to Google, where he worked on FLoC, a technology designed to replace third-party cookies by grouping Chrome users by interests. (Google killed off FLoC in 2022 after the product was met with privacy concerns.) Spero later worked for the autonomous car company Nuro, while Emi made his way through Tesla and AI biotech company Absci. After ChatGPT launched in 2022, the duo saw a business opportunity to confront a future filled with AI content. In 2023, they founded Checkfor.ai, then renamed the company Pangram a year later. By then, at least a dozen other companies were crowding the detection space, including Originality.ai, GPTZero, and Turnitin; Pangram performed well in some early, independent testing and emerged as a front-runner.
我问 Spero 关于他在加州成长的经历，这是一个非常简单的问题，却让他明显感到不适。相比回答个人问题或谈论公司卷入的文学丑闻，讨论 Pangram 的技术细节对他来说更自然。Spero 吃了一口饭，说他在洛杉矶郊区的拉克雷森塔（La Crescenta）长大。他热爱编程，并加入了学校的机器人团队。在斯坦福大学读本科时，他遇到了后来成为 Pangram 联合创始人的 Bradley Emi。（我多次通过电子邮件请求采访 Emi，但都被公司公关团队无视了。）毕业后，Spero 进入 Google，参与了 FLoC 的研发，这是一种旨在通过按兴趣对 Chrome 用户进行分组来替代第三方 Cookie 的技术。（在遭遇隐私担忧后，Google 于 2022 年终止了 FLoC。）Spero 后来在自动驾驶汽车公司 Nuro 工作，而 Emi 则先后在特斯拉和 AI 生物技术公司 Absci 任职。2022 年 ChatGPT 发布后，两人看到了应对充满 AI 内容的未来的商业机会。2023 年，他们创立了 Checkfor.ai，一年后更名为 Pangram。到那时，至少有十几家其他公司涌入检测领域，包括 Originality.ai、GPTZero 和 Turnitin；Pangram 在一些早期的独立测试中表现出色，并脱颖而出成为领跑者。
As our interview progresses, Spero’s responses seem sticky, stopping and starting, and not just because he’s eating. I ask about his hiring ethos. Spero murmurs “hmm” before turning away without apology to microwave his food. Fifteen long seconds pass in silence. He finally faces me again and says, “The average person is at Pangram because they care about the mission.”
随着采访的进行，Spero 的回答显得有些迟滞，断断续续，这不仅仅是因为他在吃饭。我问及他的招聘理念。Spero 嘟囔了一声“嗯”，然后没有道歉就转过身去加热他的食物。沉默持续了漫长的 15 秒。他终于再次面对我，说道：“来到 Pangram 的普通人，都是因为他们关心我们的使命。”
To detect AI, Pangram uses a method called “synthetic mirroring” by which it takes human writing and has LLMs generate a close match. This teaches its model how AI writes. Pangram also uses “hard negative mining,” searching datasets for false positives that it can synthetically mirror and use to augment its training set—using mistakes to retrain the machine. “All of our datasets are properly licensed, which I think is kind of rare in the AI world today,” Spero says, though that’s partly because Pangram’s product is far less data-hungry than ChatGPT or Claude. “I don’t want to completely throw the AI companies under the bus,” he adds, “but I think they’ve lost a lot of trust, especially in the world of creatives.” Today, Pangram caters to several industries, including education, the legal field, and recruitment, but creative writing constitutes the largest segment of the training text.
为了检测 AI，Pangram 使用了一种称为“合成镜像”（synthetic mirroring）的方法，即获取人类写作样本，并让大语言模型生成高度相似的匹配内容。这教会了模型 AI 是如何写作的。Pangram 还使用“硬负样本挖掘”（hard negative mining），在数据集中搜索假阳性案例，通过合成镜像将其用于扩充训练集——利用错误来重新训练机器。“我们所有的数据集都经过了正当授权，我认为这在当今的 AI 世界中是比较罕见的，”Spero 说，尽管这部分是因为 Pangram 的产品对数据的需求远低于 ChatGPT 或 Claude。“我不想完全把 AI 公司推向对立面，”他补充道，“但我认为它们已经失去了很多信任，尤其是在创意领域。”如今，Pangram 服务于教育、法律和招聘等多个行业，但创意写作构成了其训练文本的最大部分。
The Shy Girl story put Pangram on the map. Spero had already been calling out suspected AI writing from his social accounts, so it wasn’t a surprise when, in January, he was tagged in a Reddit post and subsequently sent a PDF of Ballard’s manuscript. “I put it in [Pangram], I posted it, and then later The New York Times asked me for comment,” he says. “I think people overstate the importance of Pangram in the Shy Girl story.” Except that wasn’t exactly how it happened. Like a game of AI-scandal telephone, it was a Pangram account executive who discussed the story with a publishing industry analyst who, in turn, brought it to the Times.
《Shy Girl》事件让 Pangram 一举成名。Spero 此前就一直在社交账号上指出疑似 AI 写作的内容，因此当 1 月份他在 Reddit 的帖子中被标记，并随后收到 Ballard 的手稿 PDF 时，这并不令人意外。“我把它放进 [Pangram] 里，发布了结果，后来《纽约时报》找我评论，”他说，“我认为人们夸大了 Pangram 在《Shy Girl》事件中的重要性。”但事实并非完全如此。就像一场 AI 丑闻的“传声筒游戏”，实际上是 Pangram 的一名客户经理与一位出版业分析师讨论了此事，后者随后将其带到了《纽约时报》。
Another twist: Critics of Pangram, including the investigative project The Drey Dossier, pointed out that Spero’s copy of the manuscript came from a pirating website. When I ask Spero about this, he says, “Yeah. And, like, yeah … it is what it is. I hadn’t look
另一个转折是：包括调查项目 The Drey Dossier 在内的 Pangram 批评者指出，Spero 手中的手稿副本来自一个盗版网站。当我问及此事时，Spero 说：“是的。而且，嗯……事实就是这样。我当时没看……”