What GLM-5.3 Flash running on Chinese hardware actually means

GLM-5.3 Flash 在国产硬件上运行的真正含义

Z.AI confirmed that their most recent model release was running all inference on Chinese manufactured hardware. While no doubt an impressive feat, Western companies still have a huge advantage that I can’t see changing quickly. Where is Chinese AI hardware at? To start with, it’s worth looking into where Chinese AI hardware is. I’m focusing entirely on the HiSilicon parts - the most competitive parts from Huawei. There are (many, actually) other manufacturers building AI hardware, but it’s widely believed that they are no further ahead than HiSilicon, so I think that for brevity it’s a fair starting point.

Z.AI 确认其最新发布的模型完全在国产硬件上进行推理。虽然这无疑是一项令人印象深刻的成就，但西方公司仍然拥有巨大的优势，且我认为这种局面短期内不会改变。中国的 AI 硬件目前处于什么水平？首先，有必要了解一下中国 AI 硬件的现状。我将完全聚焦于海思（HiSilicon）的产品——即华为最具竞争力的部件。虽然还有（实际上有很多）其他制造商在制造 AI 硬件，但普遍认为它们并不比海思领先，因此为了简洁起见，我认为这是一个合理的切入点。

One caveat before I go further: Z.AI didn’t actually name a chipmaker, and didn’t publish throughput or power numbers either. Nobody has independently verified the claim. So I’m assuming HiSilicon here because it’s the only plausible candidate at that scale, not because anyone has confirmed it. It’s also worth mentioning that the US export restrictions (CSIS has a good overview) of high end AI hardware have made this an enormous priority, understandably, for the Chinese. And it’s definitely worth mentioning that finding accurate sources for many of the numbers I’ll cite are difficult to be confident in, so take the exact numbers with a pinch of salt.

在深入探讨之前，需要说明一点：Z.AI 实际上并未指明芯片制造商，也没有公布吞吐量或功耗数据。目前没有任何第三方独立验证过这一说法。因此，我在此假设是海思，因为它是该规模下唯一合理的候选者，而非因为有人证实了这一点。同样值得一提的是，美国对高端 AI 硬件的出口限制（CSIS 有很好的概述）使得这在中国成为了一个极其优先的事项，这是可以理解的。此外，我必须指出，我所引用的许多数字很难找到准确的来源，因此请对具体数值持保留态度。

The current ‘scale-up’ series of HiSilicon chip, the 910c series, pairs 96GB of HBM 2e memory with two compute dies, probably achieving something like 1.6PFLOP/s of INT8 compute with ~3TB/sec of memory bandwidth, at around 600W. In essence, this is substantially behind even the H100 from Nvidia, which is now 4 years old. These are around 60% as fast as the H100, and has various other footguns (no native FP8 support for example), which probably restrict efficiency further for many use cases.

海思目前的“Scale-up”系列芯片（910c 系列）将 96GB 的 HBM 2e 内存与两个计算裸片配对，在大约 600W 的功耗下，可能实现约 1.6PFLOP/s 的 INT8 计算能力和约 3TB/s 的内存带宽。本质上，这甚至大幅落后于英伟达四年前推出的 H100。它们的性能大约是 H100 的 60%，并且存在其他各种“坑”（例如不支持原生 FP8），这可能会在许多用例中进一步限制其效率。

The next generation 950-series doesn’t meaningfully increase compute as far as I can see, but does use domestically produced HiZQ/HiBL HBM memory. Interestingly the cards are configured in two variants - the 950PR and 950DT, with the former focusing on prefill and the latter on decode. In reality, the two products are very similar, but the prefill variant using slower HiBL memory vs the decode HiZQ memory. It does however support more quantisation types, like FP8.

据我观察，下一代 950 系列在计算能力上并没有显著提升，但确实使用了国产的 HiZQ/HiBL HBM 内存。有趣的是，这些卡配置了两个变体——950PR 和 950DT，前者侧重于预填充（prefill），后者侧重于解码（decode）。实际上，这两款产品非常相似，区别在于预填充变体使用了较慢的 HiBL 内存，而解码变体使用了 HiZQ 内存。不过，它确实支持更多的量化类型，例如 FP8。

The constraints I think this shows the limitations of what Chinese hardware can do - at least for the near future. Yes, they can run inference, but so can many sets of hardware now - AMD, Google and Amazon all have competitive solutions, and OpenAI are making significant progress on their Jalapeño inference chip, which in the first published benchmarks did 1.5-1.9x the work per watt of Nvidia’s GB300. Inference hardware while no doubt complex, is a pretty solved problem right now with a lot of competition - and that’s before you bring in the Cerebras and Groq approach chips.

我认为这些限制展示了中国硬件在短期内所能达到的上限。是的，它们可以运行推理，但现在许多硬件都能做到这一点——AMD、谷歌和亚马逊都有极具竞争力的解决方案，而 OpenAI 在其 Jalapeño 推理芯片上也取得了重大进展，在首次公布的基准测试中，其每瓦性能达到了英伟达 GB300 的 1.5-1.9 倍。推理硬件虽然无疑很复杂，但目前已是一个相当成熟的问题，且竞争激烈——这还没算上 Cerebras 和 Groq 的创新芯片方案。

The wall that these Chinese hardware manufacturers are hitting is the lack of viable EUV (extreme ultraviolet) fabrication. This is the next generation silicon manufacturing process from ASML and it is extremely hard. I’d really, really recommend reading Chip War by Chris Miller for the full story, but regardless until there is significant progress on this - and by significant progress, I don’t mean the reverse engineered prototype in a Shenzhen lab. I mean reliable, scale production. The industry would be astonished if they got this to scale production before 2030.

这些中国硬件制造商所面临的瓶颈是缺乏可行的 EUV（极紫外光）光刻技术。这是 ASML 的下一代芯片制造工艺，难度极大。我非常推荐阅读克里斯·米勒（Chris Miller）的《芯片战争》（Chip War）来了解完整背景。无论如何，除非在此方面取得重大进展——我指的不是深圳实验室里逆向工程出的原型机，而是可靠的规模化生产——否则行业内如果看到他们在 2030 年前实现规模化生产，将会感到非常震惊。

Bear in mind the Shenzhen prototype hasn’t produced a working chip yet, and the more optimistic forecasts have them doing that around 2030 - volume production is a further step beyond it. It took ASML 25 years to figure out this technology - and a good 5+ years of this was scaling it up from the lab to “real” production lines. While China no doubt has incredible engineering talent and the ability to reverse engineer some of ASML’s work, it’s still a daunting challenge.

请记住，深圳的原型机尚未生产出可用的芯片，而最乐观的预测认为他们要在 2030 年左右才能做到这一点——而量产则是更遥远的一步。ASML 花了 25 年才攻克这项技术，其中有 5 年多的时间是从实验室扩展到“真正”的生产线。虽然中国无疑拥有令人难以置信的工程人才，也有能力对 ASML 的部分成果进行逆向工程，但这仍然是一项艰巨的挑战。

Without EUV it is not possible to go (much) below the “7nm” fabrication size. Without being able to go below that size, you quickly hit a wall in thermal efficiency, and you reach a point where you simply cannot make the chip(s) any bigger or faster because you cannot expel the heat quickly enough. Added to that, the additional export restrictions on HBM memory to China are clearly causing significant issues, hence the strange use of two different home grown memory technologies in the 950-series - no doubt because they can’t produce enough fast (which is still comparatively slow) memory.

没有 EUV，就不可能（大幅）突破“7nm”制造工艺。无法突破这一尺寸，就会迅速触及热效率的瓶颈，达到一个无法再让芯片变大或变快的临界点，因为你无法足够快地排出热量。此外，针对 HBM 内存的额外出口限制显然正在造成严重问题，这就是为什么 950 系列奇怪地使用了两种不同的国产内存技术——毫无疑问是因为他们无法生产足够多的高速（尽管相对而言仍然较慢）内存。

These are really the same base constraint - without EUV manufacturing technology you can’t produce the latest generations of very fast HBM memory either. But maybe this doesn’t matter? Clearly the approach China is taking is instead of really looking for solid incremental leaps in compute and memory from better manufacturing techniques, the idea is to build a lot of them. Even if your fastest chips are at best 5 years behind the latest Nvidia GPUs, you can just build 10 times as many for the same overall inference capacity.

这些本质上是同一个基础限制——没有 EUV 制造技术，你也无法生产最新一代的超高速 HBM 内存。但也许这并不重要？显然，中国采取的策略不是通过更好的制造技术在计算和内存上寻求实质性的渐进式飞跃，而是通过“堆数量”来解决。即使你最快的芯片充其量也比英伟达最新的 GPU 落后 5 年，你也可以通过制造 10 倍数量的芯片来达到相同的总推理能力。

And it really is roughly 10x - not against the H100 I was comparing to above, but against what Nvidia actually ships today. A Rubin VR200 is somewhere around 35PFLOP/s of dense FP4 with 22TB/sec of HBM4 bandwidth. The 910c is 60% of a four year old H100; Rubin is another order of magnitude past that. No doubt China is uniquely positioned in being able to do this - with enormous power generation capacity to power this, and huge quantities of skilled engineering and manufacturing labour to build the facilities and cooling required.

这确实大约是 10 倍的差距——不是与我上面比较的 H100 相比，而是与英伟达今天实际出货的产品相比。Rubin VR200 的密集 FP4 算力大约在 35PFLOP/s 左右，拥有 22TB/s 的 HBM4 带宽。910c 仅相当于四年前 H100 的 60%；而 Rubin 的性能又高出了一个数量级。毫无疑问，中国在实现这一目标方面具有独特的优势——拥有巨大的发电能力来提供电力，以及大量的熟练工程和制造劳动力来建设所需的设施和冷却系统。

But really, it’s far from ideal. As models get larger, you have to split them over more and more underpowered sets of hardware. Another problem is it makes the models slow - Z.ai’s own API is noticeably slower than Western providers serving the same weights. The bit I keep coming back to though is power. And here you have to be careful, because 10x the throughput gap is not 10x the power bill - the 910c pulls about 600W against something like 2000W for a Rubin part.

但说实话，这远非理想。随着模型变得越来越大，你必须将它们拆分到越来越多的低性能硬件上。另一个问题是这会导致模型运行缓慢——Z.ai 自己的 API 明显比提供相同权重的西方供应商要慢。但我一直关注的重点是功耗。在这里必须谨慎，因为 10 倍的吞吐量差距并不意味着 10 倍的电费——910c 的功耗约为 600W，而 Rubin 芯片的功耗约为 2000W。