Samsung’s Processing-in-Memory (PIM)

三星的内存内处理（PIM）技术

Hot Chips 2026: Samsung’s Processing-in-Memory (PIM) In-memory compute with LPDDR5X Hot Chips 2026：三星的内存内处理（PIM）——基于 LPDDR5X 的内存内计算

In-memory compute has been an attractive proposition for many years because compute within a memory chip can exploit its higher internal bandwidth. Additionally, in-memory compute avoids the long latency path between DRAM and traditional compute cores. At Hot Chips 2026, Samsung discusses their continued pursuit of in-memory compute with their PIM (Processing-in-Memory) push. They’re implementing MAC units within LPDDR5X chips, while preserving the chip’s ability to interface with a standard memory controller. 多年来，内存内计算一直是一个极具吸引力的方案，因为在内存芯片内部进行计算可以利用其更高的内部带宽。此外，内存内计算还避免了 DRAM 与传统计算核心之间漫长的延迟路径。在 Hot Chips 2026 大会上，三星讨论了他们通过 PIM（内存内处理）技术对内存内计算的持续追求。他们正在 LPDDR5X 芯片内部实现乘加（MAC）单元，同时保留了芯片与标准内存控制器进行交互的能力。

DRAM chips are internally divided into banks, each with their own read and write logic. During a normal DRAM access, the memory controller selects a bank, activates a row within it, and then accesses data via column access strobe (CAS) commands. Bandwidth is limited by the chip’s external DRAM interface. Even if the memory controller could activate all of the banks simultaneously, it wouldn’t be able to get its hands the full bandwidth available across all the banks. DRAM 芯片在内部被划分为多个存储库（Bank），每个存储库都有自己的读写逻辑。在正常的 DRAM 访问过程中，内存控制器会选择一个存储库，激活其中的一行，然后通过列地址选通（CAS）命令访问数据。带宽受到芯片外部 DRAM 接口的限制。即使内存控制器能够同时激活所有存储库，也无法获得跨所有存储库的全部可用带宽。

Samsung’s LPDDR5X-PIM is like a normal LPDDR5X-9600 chip with 16 banks, but places a PIM (Processing-in-Memory) block at each bank. These PIM blocks access their attached DRAM bank without being constrained by the chip’s external bus. Together, they can utilize the chip’s internal bandwidth across all 16 banks, which comes out to 614 GB/s. For comparison, regular DRAM accesses can hit two banks in parallel and max out at 76.8 GB/s. 三星的 LPDDR5X-PIM 类似于普通的 LPDDR5X-9600 芯片，拥有 16 个存储库，但在每个存储库处都放置了一个 PIM（内存内处理）模块。这些 PIM 模块在访问其连接的 DRAM 存储库时，不受芯片外部总线的限制。它们共同作用，可以利用跨越所有 16 个存储库的芯片内部带宽，总带宽达到 614 GB/s。相比之下，常规 DRAM 访问最多只能并行访问两个存储库，带宽上限仅为 76.8 GB/s。

PIM blocks internally consist of a MAC tree with surrounding register files and control logic. A 1024-bit instruction register file holds up to 64 16-bit instructions. A 4 kbit source register file is meant for activation vectors, and supplies one source operand for the MAC array. Samsung expects software to load model weights into DRAM, so the attached DRAM block supplies the second operand. Model weights can be scaled before the MAC computation, with scale factors coming from a 2 kbit scale register. PIM 模块内部由一个 MAC 树以及周围的寄存器文件和控制逻辑组成。一个 1024 位的指令寄存器文件最多可容纳 64 条 16 位指令。一个 4 kbit 的源寄存器文件用于存储激活向量，并为 MAC 阵列提供一个源操作数。三星期望软件将模型权重加载到 DRAM 中，因此连接的 DRAM 块会提供第二个操作数。模型权重可以在 MAC 计算前进行缩放，缩放因子来自一个 2 kbit 的缩放寄存器。

The PIM block’s MAC array supports a variety of low precision formats. Numbers from Samsung’s presentation suggest each PIM block’s MAC array can sustain four INT8 or FP8 MAC operations per data clock, or eight per cycle when not counting the double data rate. Throughput doubles for 4-bit input weights, bringing package-wide compute throughput to 2.4 TOPS. PIM 模块的 MAC 阵列支持多种低精度格式。三星演示文稿中的数据显示，每个 PIM 模块的 MAC 阵列在每个数据时钟周期内可维持 4 次 INT8 或 FP8 MAC 操作，如果不计算双倍数据速率（DDR），则每个周期可进行 8 次操作。对于 4 位输入权重，吞吐量会翻倍，使整个封装的计算吞吐量达到 2.4 TOPS。

This isn’t a very high figure, but an implementation with many LPDDR5X chips will have higher aggregate throughput. For example, eight LPDDR5X chips together would have 9.6 INT8 TOPS, which just about matches the NPU in Intel’s Meteor Lake. That would also be an expensive setup, because eight 16 GB LPDDR5X chips would correspond to 128 GB of system memory. 这个数字并不算高，但如果采用多颗 LPDDR5X 芯片实现，总吞吐量会更高。例如，八颗 LPDDR5X 芯片组合在一起将拥有 9.6 INT8 TOPS 的算力，这几乎与英特尔 Meteor Lake 中的 NPU 相当。不过，这也将是一个昂贵的配置，因为八颗 16 GB 的 LPDDR5X 芯片意味着 128 GB 的系统内存。

Accessing Compute with Standard DDR Commands 使用标准 DDR 命令访问计算功能

One highlight of LPDDR5X-PIM is that it stays within the standard LPDDR5X protocol while exposing compute capabilities that aren’t part of the memory standard. Samsung achieves this by setting aside special row addresses, which act like MMIO addresses of sorts. Each channel has a pair of predefined rows for mode control. Activating one of those rows sets the chip to single-bank mode, while the other sets the chip to multi-bank mode. Single-bank is the regular mode, while multi-bank applies commands across all 16 banks to exploit the chip’s internal bandwidth. LPDDR5X-PIM 的一个亮点在于，它在保持符合标准 LPDDR5X 协议的同时，暴露出了不属于内存标准的计算能力。三星通过预留特殊的行地址来实现这一点，这些地址类似于某种 MMIO 地址。每个通道都有一对预定义的行用于模式控制。激活其中一行会将芯片设置为单存储库模式，而激活另一行则会将芯片设置为多存储库模式。单存储库是常规模式，而多存储库模式会将命令应用于所有 16 个存储库，以利用芯片的内部带宽。

Special per-bank rows change how read and write commands behave. Activating one of these special rows makes read and write commands access PIM registers instead of regular DRAM bank contents (PIM Registers Activated mode). Samsung envisions a ML use case where software loads model weights into DRAM while the chip is in normal single-bank mode. Then, software switches into multi-bank mode and enters PIM Registers Activated mode. This lets code write activation values into PIM source registers, set scale factors in PIM scale registers, and specify an operation that’s filled into PIM instruction registers. 特殊的存储库行会改变读写命令的行为。激活这些特殊行中的一行，会使读写命令访问 PIM 寄存器，而不是常规的 DRAM 存储库内容（即“PIM 寄存器激活模式”）。三星设想了一种机器学习应用场景：软件在芯片处于正常单存储库模式时将模型权重加载到 DRAM 中；然后，软件切换到多存储库模式并进入 PIM 寄存器激活模式。这使得代码能够将激活值写入 PIM 源寄存器，在 PIM 缩放寄存器中设置缩放因子，并指定填充到 PIM 指令寄存器中的操作。

Because the chip is in multi-bank mode, each PIM register write gets broadcast across all 16 banks. PIM compute therefore works like a very constrained SIMD processor, where the operation, scale factor, and one source operand are the same across all banks. Samsung does allow writing PIM registers in single-bank mode, but that functionality is meant for debugging purposes. Each DRAM packet is 256 bits (BL=16) Filling each source register takes 16 write commands. Doing that one bank at a time across each of the 16 banks would mean 256 write commands, turning host to PIM register write bandwidth into the limiting factor. 由于芯片处于多存储库模式，每次 PIM 寄存器写入都会广播到所有 16 个存储库。因此，PIM 计算的工作方式类似于一个受限的 SIMD 处理器，其中操作、缩放因子和一个源操作数在所有存储库中都是相同的。三星确实允许在单存储库模式下写入 PIM 寄存器，但该功能仅用于调试目的。每个 DRAM 数据包为 256 位（BL=16），填充每个源寄存器需要 16 条写入命令。如果一次只对 16 个存储库中的一个进行操作，则意味着需要 256 条写入命令，这使得主机到 PIM 寄存器的写入带宽成为了瓶颈。

After priming PIM registers, software switches back into multi-bank mode and issues read commands. Instead of reading DRAM contents, these read commands initiate computations and get results accumulated into PIM vector register files. Then, write commands tell PIM blocks to write VRF contents back into the DRAM banks. 在准备好 PIM 寄存器后，软件切换回多存储库模式并发出读取命令。这些读取命令不再读取 DRAM 内容，而是启动计算并将结果累加到 PIM 向量寄存器文件中。随后，写入命令会指示 PIM 模块将 VRF（向量寄存器文件）的内容写回 DRAM 存储库。

PIM has to handle reordering that a normal memory controller might carry out. When code sets up PIM by activating the bank, PIM conventionally sets up its instruction register files so that instructions sequentially access each source register element. For instance, the first instruction would reference the first source register element, the second instruction would reference the second source register element, and so on. However, that falls apart if the memory controller reorders accesses. Samsung gets around this with an Address Align Mode (AAM), which makes each instruction infer its source register index from the column address being accessed. PIM 必须处理常规内存控制器可能执行的重排序问题。当代码通过激活存储库来设置 PIM 时，PIM 通常会设置其指令寄存器文件，以便指令按顺序访问每个源寄存器元素。例如，第一条指令引用第一个源寄存器元素，第二条指令引用第二个源寄存器元素，依此类推。然而，如果内存控制器对访问进行了重排序，这种机制就会失效。三星通过地址对齐模式（AAM）解决了这个问题，该模式使每条指令都能从正在访问的列地址中推断出其源寄存器索引。

When the host finishes using in-memory compute and wants to read results, it switches the DRAM chip back into single-bank mode. Then, regular DRAM reads and writes will start accessing DRAM contents as normal. 当主机完成内存内计算并想要读取结果时，它会将 DRAM 芯片切换回单存储库模式。此时，常规的 DRAM 读写操作将恢复正常，开始访问 DRAM 内容。

Software Headaches? 软件方面的难题？

Samsung internally achieved huge performance gains when taking advantage of LPDDR5X-PIM, compared to using standard LPDDR5X. The chip’s ability to operate with a standard memory controller is imp… 三星在内部利用 LPDDR5X-PIM 时，相比使用标准 LPDDR5X 实现了巨大的性能提升。该芯片与标准内存控制器协同工作的能力是……