Humanized but Still True? Auditing Facts and Detection in AI-Text Humanizers

Computer Science > Computation and Language arXiv:2611.00359 (cs) [Submitted on 30 Oct 2026] Title: Humanized but Still True? Auditing Facts and Detection in AI-Text Humanizers Authors: Ziyuan Wang, Genliang Zhu, Xiaobo Chen.

计算机科学 > 计算与语言 arXiv:2611.00359 (cs) [提交于 2026 年 10 月 30 日] 标题：人性化但依然真实？AI 文本“人性化”工具中的事实审计与检测 作者：Ziyuan Wang, Genliang Zhu, Xiaobo Chen。

Abstract: AI “humanizers” rewrite machine-generated text so that detectors classify it as human-written. Public leaderboards score evasion and embedding similarity, but not whether checkable details survive. We check whether numbers, dates, person and organization names, and quoted spans survive each rewrite, using a rule-based checker validated by two AI annotators (precision 0.84-1.00, recall 0.93-0.95; no human annotation).

摘要：AI“人性化”工具通过重写机器生成的文本，使其被检测器归类为人类撰写。公共排行榜对规避能力和嵌入相似度进行评分，但并不评估可核查的细节是否得以保留。我们使用一个经由两个 AI 注释器验证的基于规则的检查器（精确率 0.84-1.00，召回率 0.93-0.95；无人工注释），检查数字、日期、人名、组织名以及引用片段在重写过程中是否被保留。

In the October 2026 cycle of a public humanizer leaderboard, all 11 commercial tools changed or dropped at least one of these units in 35-65% of the documents that contained one. Most changes were to quoted spans; counting numbers and names alone, 10 of the 11 tools still altered at least one in 20% or more of the documents. The September 2026 cycle was similar (19-50%).

在 2026 年 10 月的公共人性化工具排行榜周期中，所有 11 款商业工具在 35-65% 包含上述单元的文档中，至少更改或删除了其中一个单元。大多数更改发生在引用片段中；仅计算数字和名称，11 款工具中有 10 款在 20% 或更多的文档中至少更改了一个。2026 年 9 月的周期情况类似（19-50%）。

An open 12B humanizer kept at least 97% of numbers and names in fact-dense U.S. federal agency text, but reworded 67-78% of the quoted spans, against its own prompt. Unit retention and detector evasion were not clearly related across tools (Spearman rho = -0.46, 95% CI -0.65 to 0.08), and the open model’s four released files (two of them quantization-aware low-bit files) did not show the pre-registered monotone trade-off.

一款开源的 12B 人性化模型在事实密集的美国联邦机构文本中保留了至少 97% 的数字和名称，但违背其自身提示词，重写了 67-78% 的引用片段。各工具的单元保留率与检测器规避率之间没有明显的关联（斯皮尔曼等级相关系数 rho = -0.46，95% 置信区间 -0.65 至 0.08），且该开源模型发布的四个文件（其中两个是感知量化的低位文件）并未表现出预注册的单调权衡关系。

On federal texts with matched human, AI-styled and humanized versions, a general-purpose decision model, Jev, asked a single zero-shot question, separated human originals from humanized rewrites with AUROC 0.864 (0.827-0.902), 0.18-0.22 higher than Binoculars, Fast-DetectGPT and the adversarially trained RADAR, while calling 24% of the human originals AI at a 0.5 threshold.

在包含匹配的人类撰写、AI 风格和人性化重写版本的联邦文本上，一个通用决策模型 Jev 通过单一的零样本提问，以 0.864 (0.827-0.902) 的 AUROC 值将人类原创文本与人性化重写文本区分开来，比 Binoculars、Fast-DetectGPT 和对抗训练的 RADAR 高出 0.18-0.22，同时在 0.5 的阈值下将 24% 的人类原创文本误判为 AI 生成。

On the AI-styled rewrites by 2026 models, both perplexity-based detectors scored below chance (AUROC 0.37 and 0.39), also with line breaks removed. Hypotheses, data rules and analyses were frozen before the main runs; two earlier data contacts are disclosed. We release the checker and the per-item results.

在 2026 年模型生成的 AI 风格重写文本上，两个基于困惑度的检测器得分均低于随机水平（AUROC 分别为 0.37 和 0.39），且在移除换行符的情况下也是如此。假设、数据规则和分析在主运行前已确定；披露了两次早期的数据接触。我们发布了检查器和逐项结果。