Peer review is overwhelmed—can it survive in the AI era?
Peer review is overwhelmed—can it survive in the AI era?
同行评审已不堪重负——它能在人工智能时代生存吗？
Jason Semprini was excited about his research on policies mandating that elementary school students receive the human papillomavirus vaccine. HPV causes most cases of cervical cancer, but counterintuitively, Semprini found that mandates don’t do that much to reduce the overall rate of cervical cancer in a population. That’s odd, but it’s also not completely surprising—we know that mandating a vaccine can motivate some people to find ways to avoid it. 杰森·森普里尼（Jason Semprini）对自己关于强制小学生接种人乳头瘤病毒（HPV）疫苗政策的研究感到非常兴奋。HPV导致了大多数宫颈癌病例，但出乎意料的是，森普里尼发现，强制接种政策对降低人群中宫颈癌的总体发病率并没有太大作用。这很奇怪，但也并非完全令人惊讶——我们知道，强制接种疫苗可能会促使一些人想方设法去规避它。
Semprini wrote up the study and submitted the manuscript to a journal, where it was sent out for peer review, a long-standing process through which other researchers in the field assess the validity of research and make a recommendation on whether or not it should be published. This is usually done anonymously and on a volunteer basis. 森普里尼撰写了这项研究并将其手稿提交给了一家期刊，随后该期刊将其送交同行评审。这是一个由来已久的流程，领域内的其他研究人员通过这一流程评估研究的有效性，并就其是否应该发表提出建议。这一过程通常是匿名且基于志愿服务的。
In this case, the reviewer did not share Semprini’s excitement. That may have stemmed from a misunderstanding of the study’s central message: The reviewer mistakenly thought Semprini was questioning whether the HPV vaccine itself prevents cervical cancer rather than studying the effectiveness of a policy meant to increase vaccination rates. “There’s a very big difference there,” said Semprini, who is a health economist at Des Moines University. 在这种情况下，审稿人并没有分享森普里尼的兴奋。这可能源于对研究核心信息的误解：审稿人错误地认为森普里尼是在质疑HPV疫苗本身是否能预防宫颈癌，而不是在研究旨在提高疫苗接种率的政策的有效性。“这两者之间有很大的区别，”德梅因大学（Des Moines University）的卫生经济学家森普里尼说。
Often, papers are reviewed by two or three experts, so if one misunderstands, the others can add clarity. But in this case, there was only one reviewer, so the paper was rejected. It’s frustrating for Semprini to see his work held up because a reviewer didn’t read it carefully, and it’s an experience some researchers say is becoming increasingly common. 通常，论文会由两到三位专家进行评审，因此如果其中一人误解了，其他人可以进行澄清。但在本案中，只有一名审稿人，因此论文被拒稿了。看到自己的研究因为审稿人没有仔细阅读而受阻，森普里尼感到非常沮丧，一些研究人员表示，这种经历正变得越来越普遍。
Across disciplines, the number of publications is marching steadily upward. Reviewers are struggling to keep up, and the system appears to be faltering. The number of papers indexed in the databases Scopus and Web of Science, for example, has recently been increasing exponentially, at a rate of 5.6 percent per year. 在各个学科中，出版物的数量都在稳步上升。审稿人难以跟上进度，整个系统似乎正在动摇。例如，Scopus和Web of Science数据库中收录的论文数量最近呈指数级增长，年增长率为5.6%。
By one estimate, researchers around the globe are devoting a collective 15,000 years of work to peer review every year—work that, if paid, would cost $1.5 billion for the share done in the US alone. Increasingly, researchers are finding it hard to donate that kind of time. 据估计，全球研究人员每年在同行评审上投入的总工作量相当于1.5万年——如果这些工作是有偿的，仅在美国完成的部分就需要耗资15亿美元。研究人员越来越难以贡献出这么多时间。
“I struggle like anything to get peer reviewers,” said Haseeb Irfanullah, who is on the editorial board of Wiley’s Learned Publishing but whose comments represent his own views as an independent consultant. Steven Mack, an editor for Human Immunology, has recently had to email around thirty researchers to find just one who would review a paper. Five years ago, he estimates it would have taken only five to 10 emails to find three willing reviewers. “我为了找同行评审员简直费尽了心思，”威利（Wiley）旗下《学术出版》（Learned Publishing）编委会成员哈西布·伊尔凡努拉（Haseeb Irfanullah）说，但他强调其评论仅代表他作为独立顾问的个人观点。《人类免疫学》（Human Immunology）的编辑史蒂文·麦克（Steven Mack）最近不得不给大约三十名研究人员发邮件，才找到一位愿意审稿的人。他估计，五年前，只需发五到十封邮件就能找到三位愿意审稿的专家。
In this landscape, it’s not surprising that editors are assigning reviews to people who aren’t qualified or don’t have the time to concentrate on them. In most fields, researchers need peer-reviewed publications to secure jobs and grants, so they continue to slog through the system even though it’s becoming “extremely protracted and painful,” in the words of microbiologist Sebastian Lourido from the Whitehead Institute. 在这种环境下，编辑将评审任务分配给那些不具备资格或没有时间专注评审的人也就不足为奇了。在大多数领域，研究人员需要同行评审的出版物来获得工作和资助，因此尽管正如怀特黑德研究所（Whitehead Institute）的微生物学家塞巴斯蒂安·卢里多（Sebastian Lourido）所言，这个系统正变得“极其漫长且痛苦”，他们仍不得不继续在其中挣扎。
Increasingly, many are advocating for change. Meanwhile, a few bold researchers have decided that the traditional publishing system just isn’t worth it anymore—AI research in particular has seen a shift toward communicating impactful findings through very informal means, like blogs. Peer review is “the bedrock of science,” Semprini said. But at this moment, “it’s not standing strong.” 越来越多的人呼吁变革。与此同时，一些大胆的研究人员认为传统的出版系统已经不值得再投入了——特别是人工智能研究领域，已经转向通过博客等非常非正式的方式来交流具有影响力的研究成果。森普里尼说，同行评审是“科学的基石”。但目前，“它已经不再稳固了。”
Newer than you think: Despite its central position in today’s research culture, peer review has been a universal feature of academic publishing for only about 50 years. Scientific societies had mechanisms for reviewing submissions and assessing their suitability for publication as long ago as the early 1800s, but their inner workings were variable, and the entire ethos of research was completely different at the time, says historian of science Aileen Fyfe from the University of St Andrews. 比你想象的更年轻：尽管同行评审在当今的研究文化中占据核心地位，但它作为学术出版的普遍特征仅有约50年的历史。圣安德鲁斯大学（University of St Andrews）的科学史学家艾琳·法伊夫（Aileen Fyfe）表示，早在19世纪初，科学协会就有审查投稿并评估其是否适合发表的机制，但其内部运作方式各不相同，而且当时整个研究的精神风貌也完全不同。
“Imagine all the people who are interested in astronomy in your country—and it’s not a very large number of people—but they all get together once a week on Thursday evenings to chat about astronomy, and they publish a journal together,” Fyfe told me, illustrating what science was like in the early days of reviewing. Researchers were independent gentlemen rather than university professors, so publishing wasn’t tied to career progression. Moreover, working for a journal was a fun thing to do with friends, so they were usually happy to donate their time as reviewers. “想象一下你所在国家所有对天文学感兴趣的人——人数并不多——他们每周四晚上聚在一起聊聊天文，并共同出版一本期刊，”法伊夫在向我描述早期评审时代的科学状况时说道。当时的研究人员是独立的绅士，而不是大学教授，因此发表论文与职业晋升并无关联。此外，为期刊工作是与朋友们一起做的趣事，所以他们通常很乐意贡献自己的时间担任审稿人。
Peer review became universal almost accidentally, as a way to solve a branding problem. In the 1970s, the US economy was tanking. Inflation was up, employment was down, there was a gas crisis, and the National Science Foundation found itself under public pressure to justify its spending. The pressure followed a period of post-World War II enthusiasm for science during which federal funding increased 25-fold. 同行评审几乎是偶然地成为了一种普遍做法，作为解决品牌问题的一种手段。20世纪70年代，美国经济陷入低谷。通货膨胀加剧，就业率下降，还爆发了石油危机，美国国家科学基金会（NSF）发现自己面临着证明其支出合理性的公众压力。这种压力出现在二战后科学热潮之后，当时联邦政府的科研资助增加了25倍。
The National Science Foundation had used external reviewers to assess grant applications since the 1950s—which was unusual at the time—and that turned out to be its saving grace. Because panels of independent reviewers decided how to allocate funding, their decisions had an air of legitimacy. Ultimately, that legitimacy was what kept lawmakers from putting the foundation’s funding decisions under their direct political control. 自20世纪50年代以来，美国国家科学基金会一直使用外部评审员来评估资助申请——这在当时是不寻常的——事实证明，这成了它的救命稻草。由于独立的评审小组决定了资金的分配方式，他们的决定具有一种合法性。最终，正是这种合法性阻止了立法者将基金会的资助决策置于其直接的政治控制之下。
“It was really, in a lot of ways, a sales pitch from scientists to other stakeholders that made peer review into this cornerstone of science,” said Melinda Baldwin, a historian of science from the University of Maryland. The United Kingdom went through a similar moment in the late 1980s and early 1990s when a number of controversial scientific issues—from HIV to the cause of autism to cold fusion—garnered a great deal of press attention. Suddenly, the public had an interest in distinguishing “在很多方面，这实际上是科学家向其他利益相关者进行的一次推销，才使得同行评审成为了科学的基石，”马里兰大学（University of Maryland）的科学史学家梅琳达·鲍德温（Melinda Baldwin）说。20世纪80年代末和90年代初，英国也经历了类似的时刻，当时一系列有争议的科学问题——从艾滋病毒到自闭症的成因，再到冷核聚变——引起了媒体的高度关注。突然之间，公众开始有兴趣去区分……