The Outbox Pattern Is Not Enough

Outbox 模式还不够

The textbook version of the transactional outbox is tight. You save the domain entity and an outbox row in one local transaction. A background scheduler picks up PENDING rows and publishes them to Kafka. You never publish inside the request thread — no dual-write, no atomicity breach. The pattern closes the consistency gap. 教科书式的事务性 Outbox（发件箱）模式非常严谨。你在一个本地事务中同时保存领域实体和 Outbox 记录。后台调度器会获取状态为 PENDING 的记录并将其发布到 Kafka。你永远不会在请求线程内执行发布操作——从而避免了双写问题和原子性破坏。该模式弥补了数据一致性的鸿沟。

Then you load-test it. I ran 1,000 authenticated requests through my event-driven platform in 70 seconds. The gateway returned 201 for every one of them. The outbox absorbed every row. The consumer drained everything. By every visible metric the system looked healthy. Underneath that health, I found three production-grade problems the textbook never mentioned. 接着你进行负载测试。我在我的事件驱动平台上运行了 1,000 个经过身份验证的请求，耗时 70 秒。网关对每一个请求都返回了 201 状态码。Outbox 吸收了每一行记录，消费者也处理了所有数据。从所有可见指标来看，系统看起来都很健康。但在这种健康表象之下，我发现了三个教科书从未提及的生产级问题。

What a correct implementation looks like

正确的实现长什么样

Before the problems, the shape of the solution. The outbox publisher runs on a @Scheduled virtual-thread worker: 在讨论问题之前，先看看解决方案的形态。Outbox 发布者运行在一个基于 @Scheduled 的虚拟线程工作器上：

@Scheduled(fixedDelay = 5000)
@Transactional
public void publishPendingEvents() {
    List<OutboxEvent> batch = outboxRepository
        .findTop20ByStatusOrderByCreatedAtAsc(OutboxStatus.PENDING);
    for (OutboxEvent event : batch) {
        event.setStatus(OutboxStatus.PROCESSING);
        outboxRepository.save(event);
        try {
            kafkaTemplate.send(event.getTopic(), event.getPayload()).get();
            event.setStatus(OutboxStatus.PUBLISHED);
        } catch (Exception e) {
            event.incrementRetryCount();
            if (event.getRetryCount() >= MAX_RETRIES) {
                event.setStatus(OutboxStatus.FAILED);
            } else {
                event.setStatus(OutboxStatus.PENDING);
            }
        }
        outboxRepository.save(event);
    }
}

This is correct. The PROCESSING state prevents another scheduler instance from claiming the same row. The retry cap prevents infinite cycling. The PENDING fallback on transient errors gives the event another chance. The dual-write problem is genuinely closed. Here is what that correctness does not cover. 这是正确的。PROCESSING 状态防止了另一个调度器实例获取同一行记录。重试上限防止了无限循环。针对瞬时错误的 PENDING 回退机制为事件提供了再次尝试的机会。双写问题确实得到了解决。但这种“正确性”并未涵盖以下内容。

Gap 1: Your throughput ceiling is a config line

差距 1：你的吞吐量上限是一行配置

fixedDelay = 5000 means the scheduler runs every 5 seconds. findTop20 means it picks up 20 rows per cycle. Maximum publish throughput: 20 events ÷ 5 seconds = 4 events per second. That number does not appear in your unit tests. It does not appear in your monitoring unless you specifically look for it. It is a ceiling determined by two config values chosen without measurement. fixedDelay = 5000 意味着调度器每 5 秒运行一次。findTop20 意味着每个周期获取 20 行记录。最大发布吞吐量为：20 个事件 ÷ 5 秒 = 每秒 4 个事件。这个数字不会出现在你的单元测试中，除非你专门去查看，否则它也不会出现在监控中。这是一个由两个未经测量就选定的配置值所决定的上限。

During the 1,000-event baseline run, the gateway processed ~14.3 requests per second. The publisher was draining at 4 per second. The backlog grew to 720 rows before the burst ended and the scheduler caught up. outbox_oldest_pending_age_seconds — the gauge that measures the age of the oldest PENDING row — peaked at 191 seconds. 在 1,000 个事件的基准测试中，网关每秒处理约 14.3 个请求，而发布者每秒仅处理 4 个。在突发流量结束且调度器追赶上来之前，积压数据增长到了 720 行。outbox_oldest_pending_age_seconds（衡量最旧 PENDING 记录存活时间的指标）峰值达到了 191 秒。

The system worked correctly. No events were lost. No data was corrupted. But the freshness SLO — “events delivered within 30 seconds of creation” — was structurally impossible to meet at any input rate above 4 req/s. The ceiling isn’t a bug. It’s a design constant hiding in plain sight. 系统运行正常，没有丢失事件，也没有数据损坏。但“事件在创建后 30 秒内送达”这一新鲜度 SLO（服务等级目标），在任何超过每秒 4 个请求的输入速率下，从结构上讲都是无法实现的。这个上限不是 Bug，而是一个隐藏在眼皮底下的设计常数。

Gap 2: The alert you write will fire for the wrong reason

差距 2：你编写的告警会因为错误的原因触发

The natural monitoring instinct for the outbox is an age threshold. During the 720-row burst, outbox_oldest_pending_age_seconds crossed 191 seconds. The threshold is 60 seconds. Two rules went pending. Neither fired. The for: 5m clause — which distinguishes a transient burst from a sustained incident — held. The burst resolved in under 5 minutes. Both rules sat pending through the whole event and silently cleared when the backlog drained. 对于 Outbox，自然的监控直觉是设置一个年龄阈值。在 720 行的突发流量期间，outbox_oldest_pending_age_seconds 超过了 191 秒，而阈值是 60 秒。虽然有两条规则进入了 pending 状态，但都没有触发告警。for: 5m 子句（用于区分瞬时突发和持续性故障）起到了作用。突发流量在 5 分钟内解决了，两条规则在整个事件期间保持 pending，并在积压处理完毕后静默清除。

The naive response to a pending alert is to shorten the window. Drop for: from 5 minutes to 30 seconds to “catch problems faster.” What you actually get is pages for every deployment spike, every cold-start burst, every Schema Registry restart. The alert stops being a signal and becomes noise that engineers learn to dismiss — which is worse than no alert at all. 对 pending 告警的幼稚反应是缩短时间窗口。将 for: 从 5 分钟缩短到 30 秒以“更快发现问题”。你实际得到的是：每次部署高峰、冷启动突发或 Schema Registry 重启都会收到告警。告警不再是信号，而变成了工程师学会忽略的噪音——这比没有告警更糟糕。

Gap 3: The terminal state is invisible by design

差距 3：终结状态在设计上是不可见的

This is the one that costs you. When the publisher exhausts its retry budget, the row becomes FAILED and the scheduler never touches it again. Your outbox_oldest_pending_age_seconds goes back to zero — there are no more PENDING rows to report age for. Your backlog count goes to zero. Your age alert stays silent. The event is gone. No notification was sent. 这是最让你头疼的一点。当发布者耗尽重试预算时，记录会变为 FAILED，调度器将不再处理它。你的 outbox_oldest_pending_age_seconds 会归零——因为没有更多的 PENDING 记录来报告年龄了。积压计数归零，年龄告警保持静默。事件就这样消失了，没有任何通知发出。