Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data

停止移除停用词：一种继承的预处理默认设置如何扭曲法律文本数据

Abstract: Empirical legal scholarship increasingly treats judicial text as data, and much of it still runs on sparse, interpretable pipelines — TF-IDF features and linear classifiers — because the textual feature is often the object of study, not merely a means to a prediction. Yet these pipelines inherit a chain of preprocessing defaults from mid-century information retrieval that were never validated against classification accuracy, the most entrenched being stopword removal.

摘要： 实证法律研究日益将司法文本视为数据，其中大部分研究仍运行在稀疏、可解释的流水线上（如 TF-IDF 特征和线性分类器），因为文本特征往往是研究的对象，而不仅仅是预测的手段。然而，这些流水线继承了一系列源自 20 世纪中叶信息检索的预处理默认设置，这些设置从未经过分类准确性的验证，其中最根深蒂固的便是“停用词移除”。

This study introduces an exhaustive single-word ablation that measures a preprocessing step’s effect directly against the downstream objective, and applies it to stopword removal as the hardest case to dislodge. Matching Supreme Court Database labels to Caselaw Access Project opinion texts, it examines two binary tasks that bracket F1 headroom, ideological direction (no-removal baseline F1 ~ 0.68) and constitutional versus non-constitutional law type (~ 0.92), across 7,668 and 7,001 opinions.

本研究引入了一种详尽的单词消融法，直接衡量预处理步骤对下游目标的影响，并将其应用于最难撼动的“停用词移除”案例。通过将最高法院数据库的标签与案例法访问项目（Caselaw Access Project）的判决文本进行匹配，研究考察了两个涵盖 F1 分数上限的二元任务：意识形态倾向（不移除基准 F1 约为 0.68）以及宪法与非宪法法律类型（约为 0.92），样本量分别为 7,668 份和 7,001 份判决书。

For each task the analysis approximates the best stoplist any expert could build, removing each of roughly 18,500 candidate words and measuring the effect directly. Three findings follow: generic stoplists in common use fall below the no-removal baseline in every test; even optimized stoplists are statistically indistinguishable from removing nothing; and meta-models trained on word-level features cannot predict which removals help, so list curation has nothing to target.

对于每项任务，分析过程都模拟了专家所能构建的最佳停用词表，通过移除约 18,500 个候选词中的每一个并直接测量其影响。研究得出三个结论：常用的通用停用词表在所有测试中均低于不移除的基准线；即使是优化后的停用词表，在统计学上也与“不移除任何词”没有区别；此外，基于词级特征训练的元模型无法预测哪些移除操作是有益的，因此词表整理工作缺乏明确的优化目标。

The method generalizes to any inherited preprocessing default, and the result is a caution specific to interpretable legal text-as-data: a step that silently reshapes which features a model sees can distort the very doctrinal and ideological signal such research exists to recover. Leaving stopwords in place is a question of measurement validity.

该方法可推广至任何继承的预处理默认设置。研究结果对可解释的法律文本数据分析提出了特别警示：一个悄无声息地重塑模型所见特征的步骤，可能会扭曲此类研究旨在恢复的法理和意识形态信号。保留停用词不仅是处理方式的选择，更是一个关于测量有效性的问题。