Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian

Computer Science > Computation and Language arXiv:2611.05385 (cs) [Submitted on 3 Aug 2026] Title: Same Origin, Different Verdicts: Text Production Conditions Confound Machine-Generated Text Detection in Russian Authors: Stanislav Kirichenko.

计算机科学 > 计算与语言 arXiv:2611.05385 (cs) [提交于 2026 年 8 月 3 日] 标题：同源异判：文本生成条件干扰俄语机器生成文本的检测 作者：Stanislav Kirichenko。

Abstract: Automated detection of machine-generated text is applied where the verdict carries a cost for the author, yet origin is a property of the production process, not an observable property of the text. The conditions under which a text is produced push it in the same direction in which a generative model is presumed to push it. This study measures the contribution of these conditions while origin is held fixed.

摘要：机器生成文本的自动检测通常应用于判定结果会对作者产生影响的场景，然而“来源”是生产过程的一种属性，而非文本本身可观察的属性。文本生成的条件会将其推向与生成模型预期影响相同的方向。本研究在保持来源固定的前提下，衡量了这些条件的影响贡献。

The Russian-language corpus of 1882 documents comprises a machine part, generated by four generation channels (each backed by its own model family) across 45 tasks in three task modes, and an archival human part annotated for source regimentation level before the texts were read. The primary contrast is computed within the machine corpus: both halves of a pair are generated by the same model for the same task and differ only in task wording.

该俄语语料库包含 1882 份文档，其中包括由四个生成渠道（每个渠道由各自的模型系列支持）在三种任务模式下针对 45 项任务生成的机器部分，以及在阅读文本前已标注来源规范化水平的存档人类部分。主要对比是在机器语料库内部进行的：一对样本的两部分均由同一模型针对同一任务生成，仅在任务措辞上存在差异。

Machine-likeness is measured by four operationalisations: an interpretable index with weights fixed before data collection, a trained classifier, a zero-shot metric on an open model, a judge model. The main result is disagreement between procedures: changing the task mode shifted the scores of all four, but three gave a negative shift, the judge a positive one, and none of the four intervals covers zero.

机器相似度通过四种操作化方法进行衡量：数据收集前固定权重的可解释指标、训练分类器、开放模型上的零样本度量以及判别模型。主要结果是各程序之间存在分歧：改变任务模式会改变所有四种方法的得分，其中三种方法显示负向偏移，而判别模型显示正向偏移，且四个区间均不包含零。

The comparison plan and the specifications of procedures 2-4 were fixed after the result of procedure 1, before procedures 2-4 ran; the disagreement survived correction of a corpus defect and of the nested validation. A baseline on the source identifier alone reaches a median AUROC of 1.0000 on the thirteen group splits where it is defined, and removing emphasis markers flipped a confident machine verdict to human without changing the full text by a single character.

对比计划以及程序 2-4 的规范是在程序 1 的结果得出后、程序 2-4 运行前确定的；这种分歧在修正语料库缺陷和嵌套验证后依然存在。仅基于来源标识符的基准测试在定义的 13 个组拆分中达到了 1.0000 的中位数 AUROC，且移除强调标记将一个确定的机器判定结果转变为人类结果，而全文未改变任何一个字符。

The scores are properties of a specific measurement system and corpus: they reflect data collection, repetition, formatting, and preprocessing decisions as well as linguistic style.

这些得分是特定测量系统和语料库的属性：它们反映了数据收集、重复、格式化和预处理决策，以及语言风格。