I made my Go reverse proxy 3.8x faster by fixing one line

我通过修改一行代码，将我的 Go 反向代理速度提升了 3.8 倍

I’ve been building a reverse proxy in Go. Not a hello world one, something meant to sit in production, in the actual path of every request, doing real work on each one before forwarding it along. For a while I didn’t think much about its raw throughput. It worked, requests came in, got processed, got forwarded, came back. Then I actually sat down to benchmark it properly, and the number I got back was rough. Nothing was broken. Nothing errored. It was just slow in a way I couldn’t immediately explain.

我一直在用 Go 构建一个反向代理。它不是那种“Hello World”级别的玩具，而是旨在部署到生产环境、处于每个请求的必经之路上，并在转发前对每个请求进行实际处理的系统。起初，我并没有太在意它的原始吞吐量。它能正常工作，请求进来、被处理、被转发、再返回。后来，当我真正坐下来进行基准测试时，得到的数据非常糟糕。系统没有崩溃，也没有报错，它只是慢得让我无法立即解释原因。

The setup

系统架构

The proxy uses Go’s standard library httputil.ReverseProxy under the hood, which is the normal, idiomatic way to build something like this in Go. It’s a well-built piece of the standard library. The problem wasn’t the tool. It was how I was using it. Here’s roughly what my handler looked like:

该代理底层使用了 Go 标准库中的 httputil.ReverseProxy，这是在 Go 中构建此类工具的标准且地道的方式。它是标准库中构建得非常出色的一部分。问题不在于工具本身，而在于我的使用方式。以下是我处理程序（handler）的大致代码：

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    upstream := h.resolveUpstream(r)
    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            req.URL.Scheme = upstream.Scheme
            req.URL.Host = upstream.Host
        },
    }
    proxy.ServeHTTP(w, r)
}

Looks reasonable at a glance. Every request resolves its upstream, builds a proxy for it, and serves the request. It’s readable. It’s also wrong in a way that doesn’t show up until you actually put load on it.

乍一看似乎很合理。每个请求解析其上游（upstream），为其构建一个代理，然后处理请求。代码可读性很好。但它存在一个错误，只有当你真正给它施加负载时，这个问题才会显现出来。

What was actually happening

实际发生了什么

Every single request was constructing a brand new ReverseProxy, and by extension a brand new http.Transport, from scratch. That matters because http.Transport is the thing responsible for connection pooling and reuse in Go’s HTTP client. When you build a fresh one on every request, you lose that pooling entirely. Every request pays the full cost of establishing a new connection to the upstream, instead of reusing one that’s already open and warm.

每一个请求都在从零开始构建一个全新的 ReverseProxy，进而构建了一个全新的 http.Transport。这一点至关重要，因为 http.Transport 是 Go HTTP 客户端中负责连接池和复用的组件。当你为每个请求都创建一个新的实例时，你就完全失去了连接池的优势。每个请求都要承担与上游建立新连接的全部开销，而不是复用已经打开且处于活跃状态的连接。

I didn’t catch this by reading the code and spotting it immediately. I caught it by running a load test and looking at where time was actually going. The latency numbers had a strange shape, most requests were fine, but there was a persistent, ugly tail of slow ones that didn’t line up with anything else I was doing.

我并没有通过阅读代码一眼看出这个问题。我是通过运行负载测试并观察时间消耗在哪里才发现的。延迟数据呈现出一种奇怪的形态：大多数请求表现正常，但始终存在一串令人头疼的慢请求，这与我正在执行的其他操作完全对不上。

The fix

解决方案

Stop building a new ReverseProxy per request. Build one per upstream, once, and reuse it.

停止为每个请求构建新的 ReverseProxy。为每个上游构建一次，然后复用它。

var proxyCache sync.Map // map[string]*httputil.ReverseProxy

func (h *Handler) getProxy(upstream Upstream) *httputil.ReverseProxy {
    if p, ok := proxyCache.Load(upstream.Host); ok {
        return p.(*httputil.ReverseProxy)
    }
    proxy := &httputil.ReverseProxy{
        Director: func(req *http.Request) {
            req.URL.Scheme = upstream.Scheme
            req.URL.Host = upstream.Host
        },
    }
    actual, _ := proxyCache.LoadOrStore(upstream.Host, proxy)
    return actual.(*httputil.ReverseProxy)
}

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    upstream := h.resolveUpstream(r)
    proxy := h.getProxy(upstream)
    proxy.ServeHTTP(w, r)
}

sync.Map instead of a regular map plus a mutex, because this cache is read constantly and written to rarely, once per distinct upstream, ever. That access pattern is exactly what sync.Map is designed for. Now the underlying http.Transport, and its connection pool, gets built exactly once per upstream and reused across every request that targets it. Connections stay warm. Nothing gets rebuilt from scratch on the hot path.

我使用了 sync.Map 而不是普通的 map 加互斥锁，因为这个缓存是高频读取、极低频写入的（每个不同的上游只会写入一次）。这种访问模式正是 sync.Map 的设计初衷。现在，底层的 http.Transport 及其连接池在每个上游中只会被构建一次，并被所有指向该上游的请求复用。连接保持活跃，热路径（hot path）上没有任何东西需要从零重建。

What changed

结果对比

I want to be upfront about what this benchmark actually measures, because it’s not a bare hello world server. Go can do 50 to 100 thousand requests a second doing nothing. This proxy does real work on every request, policy evaluation, field transformation, PII detection, audit logging, so the absolute numbers below aren’t meant to compete with a raw throughput benchmark. They’re meant to show the effect of one specific fix, holding everything else constant.

我想先说明这个基准测试实际测量的是什么，因为它不是一个简单的 Hello World 服务器。Go 在空转时每秒可以处理 5 到 10 万个请求。而我的代理在每个请求上都执行了实际工作，包括策略评估、字段转换、PII（个人身份信息）检测和审计日志记录，因此下面的绝对数值并不是为了与原始吞吐量基准进行比较，而是为了展示在保持其他条件不变的情况下，这一特定修复的效果。

Metric	Before	After
Requests/sec	1,710	6,541
Average latency	~58ms	7.6ms
p50	~49ms	5.9ms
p90	~110ms	13.7ms
p95	~123ms	18.1ms
Error rate	0%	0%

Tested with hey -n 400000 -c 50 against a real upstream, full policy enforcement enabled on every request, sustained over a 61 second run rather than a short burst. 400,000 requests, 50 concurrent, and every single one came back 200. No degradation over the run, no memory creep, the throughput held steady from the first second to the last. 97.5% of requests resolved in under 25ms while actively running field level policy evaluation, PII scanning, audit logging, and JSON transformation on each one.

测试使用 hey -n 400000 -c 50 对真实上游进行，每个请求都启用了完整的策略执行，测试持续了 61 秒，而不是短时间的爆发测试。40 万个请求，50 个并发，每一个都返回了 200 状态码。运行期间没有性能下降，没有内存泄漏，吞吐量从第一秒到最后一秒保持稳定。在每个请求都进行字段级策略评估、PII 扫描、审计日志记录和 JSON 转换的情况下，97.5% 的请求在 25 毫秒内完成。

Throughput went up roughly 3.8x. Average latency dropped by roughly 8x. Error rate didn’t move, it was already at zero before and after, this wasn’t a correctness fix, purely a performance one.

吞吐量提升了约 3.8 倍。平均延迟下降了约 8 倍。错误率没有变化，前后均为零，这并非修复了正确性问题，纯粹是性能优化。

The p95 number is the one I actually care about most. Averages hide tail behavior, and tail behavior is what users actually feel. Going from a p95 of roughly 123ms to 18.1ms, sustained across 400,000 requests rather than a quick burst, means the worst 5% of requests got dramatically less bad, not just the typical case, and it held up under real sustained load instead of just looking good in a short test.

我最关心的是 p95 数值。平均值会掩盖尾部延迟，而尾部延迟才是用户真正感受到的。从约 123ms 的 p95 降至 18.1ms，且是在 40 万个请求的持续负载下而非短时爆发中实现，这意味着最慢的那 5% 的请求得到了显著改善，而不仅仅是典型情况下的优化，并且它在真实的持续负载下表现稳健，而不仅仅是在短时测试中看起来不错。

The lesson

经验教训

This is a genuinely easy mistake to make in Go, because the code that causes it doesn’t look wrong. There’s no compiler warning for it. Nothing crashes. It just quietly throws away one of the more important performance characteristics of the standard library’s HTTP client, and the only way to notice is to actually measure.

在 Go 中，这是一个非常容易犯的错误，因为导致问题的代码看起来并没有错。编译器不会发出警告，程序也不会崩溃。它只是悄悄地丢弃了标准库 HTTP 客户端最重要的性能特性之一，而发现这一点的唯一方法就是进行实际测量。

If you’re building anything on top of httputil.ReverseProxy, or honestly anything on top of http.Client or http.Transport, construct it once per destination and hold onto it. Don’t rebuild it inside your request handler. It’s a small change, and in my case it was worth more than 3x throughput for free.

如果你正在基于 httputil.ReverseProxy 构建任何东西，或者老实说，基于 http.Client 或 http.Transport 构建任何东西，请为每个目标构建一次并保持引用。不要在请求处理程序中重建它。这是一个小改动，但在我的案例中，它免费换取了超过 3 倍的吞吐量提升。