本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Cerebras Deploys Trillion-Parameter AI Models

One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.

我们最常被问到的问题之一，就是我们如何在 Cerebras 晶圆级芯片上部署大型模型。单片 Cerebras WSE-3T 晶圆拥有 44 GB 的片上内存，而万亿参数的前沿模型仅权重就需要数 TB 的空间。

In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.

在这篇文章中，我们将解释如何将大型前沿模型拆分到多个 WSE-3T 晶圆上，以实现每秒 1000-2000+ token 的多万亿参数模型运行速度。

We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.

我们将以拥有约 3 万亿参数的开源模型 Kimi K3 为例进行说明。

By the end, you’ll understand:

读完本文，你将了解：

Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.

Kimi K3 是一个开源的 3 万亿参数模型，包含 69 层线性注意力层和 24 层密集层。

It scores an impressive 88.3% on Terminal-Bench 2.1 (vs. GPT-5.6 Sol’s 88.8%) and 91.2% on BrowseComp (vs. GPT-5.6 Sol’s 90.4%). These results demonstrate frontier-level coding and agentic capabilities, even outperforming leading proprietary models on some benchmarks. Read more about Kimi K3 here.

它在 Terminal-Bench 2.1 上取得了 88.3% 的优异成绩（GPT-5.6 Sol 为 88.8%），在 BrowseComp 上取得了 91.2% 的成绩（GPT-5.6 Sol 为 90.4%）。这些结果展示了其前沿水平的编码和智能体能力，在某些基准测试中甚至超过了领先的专有模型。点击此处了解更多关于 Kimi K3 的信息。

For this illustrative example, we use:

在此示例中，我们使用：

To calculate the total number of Cerebras WSE-3T wafers we need to support Kimi K3, let’s first calculate the size of each model layer.

为了计算支持 Kimi K3 所需的 Cerebras WSE-3T 晶圆总数，我们首先计算每个模型层的大小。

Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:

Kimi K3 拥有 3 万亿参数。如果所有 3 万亿参数均以 4 位存储，则原始参数负载为：

Dividing that payload by 93 layers then gives:

将该负载除以 93 层，得出：

Now that we know each layer needs about 16 GB, we can figure out how many layers fit on each 44 GB wafer. The answer depends on the layer type:

既然我们知道每一层大约需要 16 GB，我们就可以计算出每片 44 GB 的晶圆能容纳多少层。答案取决于层类型：

Therefore, with Kimi K3’s 69 linear layers and 24 dense layers, we get:

因此，对于 Kimi K3 的 69 层线性层和 24 层密集层，我们得到：

This gives us a total wafer count of 35 + 24 = 59 wafers.

这使我们的晶圆总数为 35 + 24 = 59 片。

Cerebras executes models in a pipeline. After the Cerebras compiler maps the model layers across regions of wafers, the runtime maps the path for tokens to flow between wafers. For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet. This is roughly 2 microseconds per hop.

Cerebras 以流水线方式执行模型。在 Cerebras 编译器将模型层映射到晶圆区域后，运行时会映射 token 在晶圆间流动的路径。为了实现超低延迟，只需在晶圆间传递激活值，这可以通过以太网上的 RDMA 完成。每次跳转大约需要 2 微秒。

It’s important to emphasize that because each Cerebras wafer has 44 GB of on-chip SRAM and each layer is only 16 GB, we are typically able to fit entire layers of the model directly on the chip.

需要强调的是，由于每片 Cerebras 晶圆拥有 44 GB 的片上 SRAM，而每一层仅为 16 GB，我们通常能够将整个模型层直接放入芯片中。

This means that in most cases, the model weights never have to move, and layers don’t need to be split across wafers. Only activations cross from one wafer to the next.

这意味着在大多数情况下，模型权重无需移动，层也不需要跨晶圆拆分。只有激活值会在晶圆之间传输。

Now that we have our wafers connected together and the model laid out, we can think of each wafer as one stage in an assembly line: the model’s weights for each stage stay in place, and the request’s intermediate results move through the layer stages.

现在我们已经连接了晶圆并部署了模型，我们可以将每片晶圆视为装配线上的一个阶段：每个阶段的模型权重保持不变，而请求的中间结果则在各个层阶段之间移动。

For example, Wafer 1 runs model layers 1–2 and sends its activations to wafer 2. Wafer 2 runs layers 3–4 and passes its results to Wafer 3, and so on until the last wafer produces the next token.

例如，晶圆 1 运行模型第 1-2 层并将激活值发送到晶圆 2。晶圆 2 运行第 3-4 层并将结果传递给晶圆 3，依此类推，直到最后一片晶圆生成下一个 token。

This raises a common misconception. If a token has to travel through each wafer in order, doesn’t each wafer sit idle while it waits for its turn? In effect, wouldn’t the system be running at a batch size of one?

这引发了一个常见的误解。如果 token 必须按顺序通过每个晶圆，那么每个晶圆在等待轮到自己时岂不是处于空闲状态？实际上，系统难道不是以 1 的批处理大小在运行吗？

No! If we only sent one request down the line at a time and waited for it to complete before starting the next request, this would be an inefficient use of the pipeline. Instead, we keep the entire pipeline full with different requests. Because the model is split into pipeline stages already, each stage can work on a different request at the same time.

不！如果我们一次只发送一个请求，并等待其完成后再开始下一个请求，这将是对流水线的低效利用。相反，我们让整个流水线充满不同的请求。由于模型已经被拆分为流水线阶段，每个阶段可以同时处理不同的请求。

While stage 3 processes a token for request A, stage 2 processes one for request B, and stage 1 one for request C. In our example, each stage is a layer. In production, Cerebras uses much finer-grained stages, and many more of them.

当阶段 3 处理请求 A 的 token 时，阶段 2 处理请求 B 的 token，阶段 1 处理请求 C 的 token。在我们的示例中，每个阶段就是一层。在生产环境中，Cerebras 使用粒度更细、数量更多的阶段。

Each stage can also process a micro-batch, a small group of requests, rather than a single request. So the number of requests in flight is roughly the number of stages times the micro-batch size.

每个阶段还可以处理微批次（一组小请求），而不是单个请求。因此，正在处理的请求数量大约等于阶段数乘以微批次大小。

To understand the tradeoff, smaller micro-batches make each request even faster, while larger micro-batches increase total throughput.

为了理解其中的权衡，较小的微批次使每个请求更快，而较大的微批次则提高了总吞吐量。

The amount of on-chip memory allocated to KV cache determines the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer, so the size of on-chip memory sets the ceiling on concurrent requests. As explained above, we only need to calculate KV cache requirements for dense layers.

分配给 KV 缓存的片上内存量决定了并发请求的最大数量。每个正在处理的请求都需要在每个密集层晶圆上有自己的 KV 缓存，因此片上内存的大小设定了并发请求的上限。如上所述，我们只需要计算密集层的 KV 缓存需求。

For each 44 GB wafer that already has one dense layer, we can calculate the remaining space for the KV cache with:

对于每片已经包含一个密集层的 44 GB 晶圆，我们可以通过以下方式计算 KV 缓存的剩余空间：

Total wafer storage - size of one dense layer. In this case:

晶圆总存储量 - 一个密集层的大小。在本例中：

Calculating the # of Concurrent Requests

计算并发请求数

To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:

为了找到特定模型映射所支持的并发请求数，我们首先计算单个 256k 请求占用的缓存大小：

To then calculate the total number of concurrent requests this configuration would support, we divide:

然后，为了计算此配置所支持的并发请求总数，我们进行除法运算：

The total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.

并发请求总数代表在任何时间点通过流水线同时移动的请求数量。与 GPU 批处理不同，这些请求不需要等待彼此完成，它们在流水线中持续流动。

Of course, neither concurrency nor batch size tells you exactly how efficiently an inference system performs. When evaluating the throughput of an inference system, the more meaningful metric is total token throughput per unit of power (e.g. tokens per second per megawatt), when measured at the same output generation speed.

当然，并发性或批处理大小都不能准确说明推理系统的性能效率。在评估推理系统的吞吐量时，更有意义的指标是在相同的输出生成速度下，单位功耗的总 token 吞吐量（例如每兆瓦每秒生成的 token 数）。

The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.

同样的方法也适用于更大的模型：计算权重和请求状态所需的内存，将模型分布在晶圆上，并将它们连接成流水线。每增加一片晶圆都会增加计算能力和片上内存，使我们能够容纳超过单片芯片容量的模型。

Thanks for learning with us! Interested in deploying a model on Cerebras or building with our inference cloud?

感谢您的阅读！有兴趣在 Cerebras 上部署模型或使用我们的推理云进行构建吗？

Designs by Halley Chang. Thanks to Atharva Talpade, Zhenwei Gao, Angela Yeung, Jessica Liu, Joyce Er, Emma Call, and Rohan Gupta for their review and feedback.

设计：Halley Chang。感谢 Atharva Talpade、Zhenwei Gao、Angela Yeung、Jessica Liu、Joyce Er、Emma Call 和 Rohan Gupta 的审阅与反馈。

Performance comparisons are based on third-party benchmarking or internal testing. Observed inference speed improvements versus GPU-based systems may vary.

性能比较基于第三方基准测试或内部测试。与基于 GPU 的系统相比，观察到的推理速度提升可能会有所不同。