What SHAP Can’t Explain About Agentic AI Fraud

SHAP 无法解释的智能体 AI 欺诈问题

Fraud stopped being a purely human activity sometime in the last year. Not all of it, obviously; I mean, most fraud is still someone typing on a phone somewhere trying to move money that isn’t theirs. But enough of it has shifted that Experian’s 2026 Future of Fraud Forecast put a name on it. AI agents transacting and shopping on people’s behalf are now a real driver of financial fraud. The report calls it “machine-to-machine mayhem,” which sounds like a headline written to get clicks, and honestly, it worked. Because a legitimate shopping agent and a fraudulent bot can show up in a transaction log looking basically identical. Nobody’s figured out the tell yet.

在过去的一年里，欺诈行为不再仅仅是人类的活动。当然，并非全部如此；我的意思是，大多数欺诈行为仍然是某人在手机上打字，试图转移不属于他们的钱。但已经有足够多的欺诈行为发生了转变，以至于益博睿（Experian）的《2026 年欺诈未来预测》报告为此命名。代表人类进行交易和购物的 AI 智能体现在已成为金融欺诈的真正驱动力。该报告将其称为“机器对机器的混乱”（machine-to-machine mayhem），这听起来像是一个为了博取点击率而写的标题，老实说，它确实奏效了。因为合法的购物智能体和欺诈机器人出现在交易日志中时，看起来基本是一模一样的。目前还没有人找到识别它们的方法。

I want to sit with why that’s a bigger deal than it sounds. Most fraud detection, at every scale, from a local fintech to a Fortune 500 bank, has quietly rested on the same assumption for a decade or more. Which is that there’s a human on the other end of the transaction, and humans leave fingerprints. Things like how they type, when they switch devices, basically the way someone behaves when they’re anxious or in a hurry. Every fraud model, whether anyone building it says so out loud, is trained to notice when a human stops acting like themselves.

我想探讨一下为什么这比听起来更严重。在过去十年甚至更久的时间里，大多数欺诈检测系统（无论是本地金融科技公司还是财富 500 强银行）都默默地基于同一个假设：交易的另一端是人类，而人类会留下指纹。比如他们打字的方式、切换设备的时间，基本上就是一个人在焦虑或匆忙时的行为方式。每一个欺诈模型，无论构建它的人是否公开承认，都是为了识别一个人何时不再表现得像他自己而训练的。

I ran into a smaller, dumber version of this exact problem while building a fraud detection system, one that leaned hard on SHAP to explain itself. It’s worth telling because the thing that got me out of trouble at the time is the exact thing I now think is running out of road.

我在构建一个欺诈检测系统时，遇到了这个问题的较小、较简单的版本，该系统非常依赖 SHAP 来进行自我解释。这个故事值得一讲，因为当时让我摆脱困境的方法，正是现在我认为即将走到尽头的东西。

The question that wasn’t really about the model

一个并非关于模型本身的问题

At my presentation, one of the panelists leaned back in his chair and asked a question that had nothing to do with precision or recall. If a CEO makes a one-million-dollar transaction, and a university student makes the same transaction, does your system flag them the same way?

在我的演示会上，一位评审员靠在椅背上，问了一个与精确率或召回率毫无关系的问题。如果一位首席执行官进行了一笔一百万美元的交易，而一名大学生进行了同样的交易，你的系统会以同样的方式标记他们吗？

I remember the room going quiet, and I remember taking longer to answer than I wanted to. My model, a Random Forest classifier I’d trained on the PaySim dataset after benchmarking it against XGBoost and LightGBM, didn’t know or care who was transacting. It looked at the transaction: amount, timing, device, velocity. Basically the same features for a CEO’s account as for a broke undergrad’s. On paper, that’s what fairness in ML is supposed to look like. In front of a panel whose entire job is finding holes in risk models, it looked like I’d missed something obvious.

我记得当时房间里一片寂静，我回答的时间比我预想的要长。我的模型是一个随机森林分类器，我在 PaySim 数据集上对其进行了训练，并与 XGBoost 和 LightGBM 进行了基准测试，它不知道也不关心是谁在交易。它只看交易本身：金额、时间、设备、频率。对于首席执行官的账户和贫困大学生的账户，它使用的特征基本相同。从理论上讲，这就是机器学习中公平性的样子。但在一个专门负责寻找风险模型漏洞的评审团面前，这看起来就像我忽略了一些显而易见的东西。

Here’s what actually saved me, and it wasn’t a smarter model. I’d wired SHAP into the pipeline sitting on top of the classifier, mostly because a supervisor earlier in the year had pushed back on treating the model as a black box, and it turned out to be the reason I survived that question. I could pull up the specific transaction, show the ranked SHAP values, and say: here’s what actually drove this prediction, in order, and with numbers attached.

真正救了我的是什么呢？并不是一个更聪明的模型。我将 SHAP 植入到了分类器之上的流水线中，主要是因为今年早些时候一位主管反对将模型视为“黑盒”，事实证明，这就是我能挺过那个问题的关键。我可以调出具体的交易，展示排序后的 SHAP 值，并说：这就是驱动该预测的实际因素，按顺序排列，并附有具体数值。

That question has stayed with me longer than almost anything else from that presentation. Not because of what it exposed about my model specifically, but because of the assumption sitting underneath the whole field: that the thing making a transaction has a life outside the transaction, a job, a history, a reason to be moving that particular amount of money. In simpler words, context.

那个问题在我脑海中停留的时间比那次演示中的任何其他内容都要长。这不仅是因为它暴露了我模型的问题，更是因为它揭示了整个领域背后的假设：即进行交易的事物在交易之外拥有生活、工作、历史，以及转移特定金额资金的理由。简单来说，就是“背景信息”。

Why explainability stopped being optional

为什么可解释性不再是可选项

Black-box fraud models have always had a trust problem, agentic AI or not. The models that perform best, deep ensembles, gradient-boosted trees, tend to be the hardest to interpret, and that’s a genuinely annoying trade-off if you’ve ever had to explain a flagged transaction to someone who isn’t a data scientist.

无论是否涉及智能体 AI，黑盒欺诈模型一直存在信任问题。表现最好的模型（如深度集成模型、梯度提升树）往往最难解释，如果你曾经需要向非数据科学家解释一笔被标记的交易，就会发现这是一个非常令人头疼的权衡。

If a compliance analyst can’t tell whether a flag is a real signal or just noise the model latched onto, they’ll either rubber-stamp everything or ignore the model entirely. I’ve seen both happen, and neither one is why you built the thing.

如果合规分析师无法判断一个标记是真实的信号，还是模型捕捉到的噪音，他们要么会盲目批准所有交易，要么会完全忽略模型。这两种情况我都见过，而这两种结果都不是你构建该系统的初衷。

SHAP earned its spot in fraud stacks by answering a narrow, unglamorous question, not “is this model good” but “why did it do this, for this one transaction, right now.” It’s less exciting than model architecture, which is maybe why it doesn’t come up in conversation as much as it probably should. A risk score with no reasoning attached is close to useless the moment a human has to act on it. That was my exact problem at the presentation. Instead of “92% fraud probability,” you get something like: the amount is high for this account, the account is under 48 hours old, or the merchant has prior flags. People trust that. Or at least they trust it more than a number with nothing under it.

SHAP 在欺诈检测技术栈中赢得了一席之地，因为它回答了一个狭窄且枯燥的问题：不是“这个模型好不好”，而是“为什么它在此时此刻针对这笔交易做出了这个判断”。这比模型架构要乏味得多，也许这就是为什么它在讨论中出现的频率不如应有的那么高。当人类必须根据风险评分采取行动时，没有理由支持的风险评分几乎毫无用处。这正是我在演示时遇到的问题。与其得到一个“92% 的欺诈概率”，你得到的是类似这样的解释：该账户金额过高、账户注册时间不足 48 小时，或者商户有之前的违规记录。人们信任这些。或者至少，他们比信任一个没有任何支撑的数字更信任这些解释。

The ground is shifting under the fingerprint

指纹之下的地基正在动摇

Here’s the part that bugs me. Every explainability method built for fraud detection, SHAP included, explains a decision using features that describe a transaction: amount, time, device, velocity. Now stay with me here. Those features exist because they correlate with how humans behave when something’s wrong. A student’s account suddenly moving CEO-sized money looks strange precisely because we assume a stable human behind it, with a pattern of life the anomaly is breaking from.

这就是让我困扰的地方。每一种为欺诈检测构建的可解释性方法（包括 SHAP）都是通过描述交易的特征（金额、时间、设备、频率）来解释决策的。请跟上我的思路：这些特征之所以存在，是因为它们与人类在出现异常时的行为方式相关联。一个学生账户突然转移出首席执行官级别的资金看起来很奇怪，正是因为我们假设其背后是一个稳定的人类，且该异常行为打破了其原有的生活模式。

An agent doesn’t have a pattern of life in any way that maps onto those features. It doesn’t get tired at 2 am. It doesn’t fat-finger a transfer or panic-send money to the wrong account. It can transact at a speed and consistency no human could sustain for five minutes, and that breaks the whole premise fraud detection has quietly run on: that deviation from a human baseline is the signal.

智能体在这些特征上没有任何可以映射的“生活模式”。它不会在凌晨 2 点感到疲倦，不会因为手滑而转错账，也不会因为恐慌而把钱发错账户。它能以人类五分钟都无法维持的速度和一致性进行交易，这打破了欺诈检测一直以来默默运行的前提：即偏离人类基准的行为就是信号。