AI in Scientific Research: How to Tell Where It Is Actually Working
AI in Scientific Research: How to Tell Where It Is Actually Working
科研中的人工智能：如何判断它是否真的有效
“AI discovered a new material.” “AI found a drug candidate.” “AI solved protein folding.” Each of those sentences can be true, badly misleading, or flatly wrong depending on one thing the sentence does not tell you: how far the result got from the model before somebody wrote it down. “AI 发现了一种新材料。”“AI 找到了一种候选药物。”“AI 解决了蛋白质折叠问题。”这些句子可能是真实的，也可能是极具误导性的，甚至是完全错误的，这取决于一个句子中没有告诉你的关键点：在结果被记录下来之前，它距离模型输出已经走到了哪一步。
The sentence that hides four different claims. Take a single headline: a model proposed a molecule that binds a protein implicated in a disease. That sentence is compatible with at least four very different states of the world. The molecule might exist only as a string in a file. It might have been synthesised. It might have bound the protein in a test tube. Or it might have improved an outcome in a person. Those four are separated by years, by orders of magnitude in cost, and by a probability of success that drops at every step — and press coverage routinely reports the first as though it were the fourth. 隐藏了四种不同主张的句子。以一个标题为例：一个模型提出了一种能与某种疾病相关蛋白质结合的分子。这句话可能对应着四种截然不同的现实情况。该分子可能仅仅存在于文件中的一个字符串里；它可能已经被合成出来；它可能已经在试管中与蛋白质结合；或者它可能已经改善了某位患者的病情。这四种情况之间相隔数年，成本相差几个数量级，且成功概率在每一步都在下降——但新闻报道往往将第一种情况描述得好像是第四种一样。
This is not a complaint about journalism. It is the single most useful thing to internalise about the whole field, because once you have the ladder in your head you can grade a claim in about ten seconds, and you can do it for a subject you know nothing about. 这并不是在抱怨新闻业。这是理解整个领域最有用的一点，因为一旦你脑海中建立了这个“阶梯”模型，你就能在十秒钟内评估一个主张的含金量，即使是对你一无所知的学科也能做到。
The ladder
阶梯模型
The rungs are the same in every discipline. Only the names of the instruments change. 每个学科的阶梯层级都是一样的，只是所用工具的名称不同。
| Rung | Description |
|---|---|
| 1 · Output | The model emitted something: a structure, a score, a candidate, a forecast. Nothing has been checked. Everything downstream is conditional on this being worth checking. |
| 2 · Retrospective | The output was compared against data that already existed — held-out structures, historical weather, known compounds. This is where nearly all published numbers live, and it is entirely dependent on the held-out set resembling the future. |
| 3 · Prospective | The prediction was made first and the answer arrived afterwards. A forecast verified against what the weather then did. A candidate synthesised after being proposed. This rung is qualitatively stronger than rung 2 and much rarer. |
| 4 · Confirmed | An independent method established the result: a crystal structure solved, a compound characterised, a proof mechanically checked, an effect replicated in another lab. |
| 5 · Adopted | Practitioners changed what they do. The predicted structure is now the starting point for the experiment; the ML forecast is in the operational suite. This is the only rung that means the field actually moved. |
| 层级 | 描述 |
|---|---|
| 1 · 输出 | 模型产生了一些东西：一个结构、一个分数、一个候选对象或一个预测。没有任何东西经过验证。后续的一切都取决于这些结果是否值得进一步检查。 |
| 2 · 回顾性 | 将输出结果与已有的数据进行对比——如留出的结构数据、历史天气、已知化合物。几乎所有已发表的数据都处于这一层级，且完全依赖于留出的数据集是否能代表未来。 |
| 3 · 前瞻性 | 先做出预测，随后得到答案。例如，根据实际天气验证预报，或在提出候选分子后进行合成。这一层级在质量上比第 2 层级更强，也更为罕见。 |
| 4 · 已确认 | 通过独立方法确立了结果：解析了晶体结构、表征了化合物、机械化验证了证明，或在另一个实验室复制了实验效果。 |
| 5 · 已采用 | 从业者改变了工作方式。预测的结构现在成为实验的起点；机器学习预测已进入业务流程。这是唯一意味着该领域真正取得进展的层级。 |
Where most claims actually sit
大多数主张的实际位置
Rungs 1 and 2. Overwhelmingly. And rung 2 is a great deal weaker than it looks, because retrospective evaluation on scientific data has a specific failure that ordinary machine learning does not: the held-out set is usually not independent of the training set in the way the split implies. Protein structures share evolutionary ancestry, so a random split leaves near-homologues on both sides. Materials databases are full of compositional substitutions of the same structure type. Patient cohorts share a scanner, a hospital and a labelling convention. The consequence is that a retrospective score is best read as an upper bound on what will happen prospectively, and often a generous one. That is not a reason to dismiss rung 2 — it is how methods get developed — but a claim that stops there has not yet said anything about the world. 绝大多数处于第 1 和第 2 层级。而且第 2 层级比看起来要脆弱得多，因为科学数据的回顾性评估存在一种普通机器学习所没有的特定缺陷：留出的数据集通常不像划分时所暗示的那样与训练集完全独立。蛋白质结构具有进化上的同源性，因此随机划分会导致近缘同源物出现在两边。材料数据库中充满了相同结构类型的成分替代。患者队列共享同一台扫描仪、同一家医院和同一种标注规范。结果就是，回顾性分数最好被解读为前瞻性表现的上限，而且往往是一个过于乐观的上限。这并不是否定第 2 层级的理由——毕竟方法就是这样发展出来的——但如果一个主张止步于此，它实际上还没有对现实世界做出任何实质性的说明。
Why some fields move and others do not
为什么有些领域进步了，而有些没有
Compare two verification steps. Checking a proposed proof step in a formal system takes a computer a fraction of a second and the answer is binary. Checking whether a drug candidate helps a person takes years, costs an enormous amount, and returns a noisy answer. Both fields have had excellent models pointed at them. Only one of them has been visibly transformed, and the reason is not the quality of the models. 比较两个验证步骤。在形式系统中检查一个证明步骤，计算机只需几分之一秒，且答案是二元的（对或错）。而检查一种候选药物是否对人有效，则需要数年时间，耗资巨大，且结果往往充满噪声。这两个领域都有出色的模型介入，但只有一个领域发生了明显的变革，原因不在于模型质量。
So the useful predictor of where machine learning changes a science is the cost and latency of that field’s verification step, relative to the cost of generating a candidate. Where verification is cheap — formal proof, code that either runs or does not, a simulation you can re-run, a forecast that is checked by tomorrow — a model that produces many mediocre candidates is enormously valuable, because you can afford to discard almost all of them. Where verification is expensive, the bottleneck was never candidate generation, and speeding it up moves very little. 因此，判断机器学习在何处能改变科学的一个有效指标是：该领域验证步骤的成本和延迟，相对于生成候选对象的成本。在验证成本低廉的地方——如形式化证明、能运行或不能运行的代码、可重复运行的模拟、明天就能验证的天气预报——一个能产生大量平庸候选对象的模型非常有价值，因为你可以负担得起丢弃其中绝大多数。而在验证成本高昂的地方，瓶颈从来都不是候选对象的生成，加速生成过程也无法带来实质性进展。
Five questions that grade any claim
评估任何主张的五个问题
-
Which rung? If the answer is 1 or 2, the correct word is “predicted” and any sentence using “discovered”, “found” or “solved” has been upgraded somewhere between the paper and you.
-
Who checked it, and were they the same people? Independent confirmation is a different claim from the authors validating their own candidates, particularly when the validation method is also automated.
-
What exactly did the model contribute? Proposing the candidate, ranking a list somebody else generated, and choosing which experiment to run next are three different contributions, and the third is usually the valuable one.
-
Was the split honest? Ask what would have had to be true for the test set to be genuinely unseen. In most scientific data it requires a deliberate split by structure, by time or by site, not a random one.
-
What is the base rate? A hit rate is only interpretable against what the existing method achieves on the same problem. A candidate list with no comparator is not evidence about the model. None of this is scepticism for its own sake. Several fields in this cluster have genuinely reached…
-
处于哪个层级？ 如果答案是 1 或 2，正确的用词应该是“预测”，任何使用“发现”、“找到”或“解决”的句子，都是在从论文到你手中的过程中被夸大了。
-
谁进行了检查，是同一批人吗？ 独立确认与作者验证自己的候选对象是完全不同的主张，尤其是当验证方法也是自动化的时候。
-
模型到底贡献了什么？ 提出候选对象、对他人生成的列表进行排序、以及选择下一步进行什么实验，这是三种不同的贡献，而第三种通常才是最有价值的。
-
数据划分是否诚实？ 思考一下，测试集要达到什么条件才能真正做到“未见过”。在大多数科学数据中，这需要按结构、时间或地点进行刻意的划分，而不是随机划分。
-
基准率是多少？ 命中率只有在与现有方法在同一问题上所取得的成果进行对比时才有意义。没有对比对象的候选列表不能作为模型有效的证据。 这些并不是为了怀疑而怀疑。在这个领域集群中，有几个领域确实已经达到了……