Retrospectively Reverse-Engineering Apple’s Neural Engine

回顾性逆向工程 Apple Neural Engine (ANE)

Aug 10, 2026 (5089 words) 2026年8月10日（5089字）

I stopped working on the reverse-engineered Apple Neural Engine (ANE) driver three years ago, upon a sad mini realization that the ANE block is just not that useful, and I could be doing more useful things, and moved onto upstreaming other, more useful, blocks. The ANE’s architecture was too opinionated to build a general-purpose accelerator platform around it, and a linux driver effectively opening ANE hardware API access could not broaden the class of workloads it could do. Even macOS only regularly uses their own ANE to generate upsampled preview images in Finder. 三年前，我停止了对 Apple Neural Engine (ANE) 驱动程序的逆向工程工作。当时我产生了一个令人沮丧的小小认知：ANE 模块其实并没有那么有用，我完全可以去做更有意义的事情，于是我转而去向上游提交其他更有价值的模块。ANE 的架构过于固化，难以围绕它构建一个通用的加速器平台；即便通过 Linux 驱动程序开放 ANE 硬件 API 访问权限，也无法拓宽它所能处理的工作负载类型。即便是 macOS 本身，也只是常规地利用 ANE 在 Finder 中生成预览图的放大版本。

https://github.com/eiln/ane/tree/main M1 die shot: https://mastodon.social/@dougall/115149886886125067 （链接见原文）

The M5 (2025)‘s headline feature was “LLM performance”, and they also conveniently folded the ANE cores inside the GPU cores — I knew it was coming, but it officially feels like the beginning of the end for the standalone NPU. So, in honor of the ANE’s apparent demise, we will do something even more useless: go back and reverse-engineer the ANE on the M1, finish what we started. It’s been three years (fuck), and I should know more than I did when I first worked on this. M5 (2025) 的主打功能是“大语言模型 (LLM) 性能”，而且他们顺手将 ANE 核心整合进了 GPU 核心内部——我早就预料到会有这一天，但这确实标志着独立 NPU 时代的终结。因此，为了纪念 ANE 的“离世”，我们将做一件更无用的事：回到过去，对 M1 上的 ANE 进行逆向工程，完成我们未竟的事业。三年过去了（该死），现在的我应该比当初刚开始做这件事时了解得更多。

If the goal three years ago was to make the ANE useful by running ops on it; this time, it’s more about mapping the full internal architecture — compute, datapath, scheduler, memory, and execution model — because those internal design decisions reveal the assumptions about ML workloads that Apple was willing to commit to silicon first in the A11 Bionic (2017), and what that says about the shift from CNN-era NPUs to today’s GPUs running transformer workloads. 如果说三年前的目标是通过在 ANE 上运行算子来使其变得有用；那么这一次，重点在于完整映射其内部架构——计算、数据通路、调度器、内存和执行模型。因为这些内部设计决策揭示了 Apple 在 A11 Bionic (2017) 中首次投入硅片时对机器学习工作负载的假设，以及这反映了从 CNN 时代 NPU 到如今运行 Transformer 工作负载的 GPU 之间的范式转移。

1. Compute

1. 计算

The 16 compute cores are probably the least interesting part of the ANE. Apple originally targeted dense image-processing CNN workloads, which consists of dense tensor reductions with predictable reuse. The M1 ANE compute core is a large parallel array of multiply-accumulate (MAC) units, but that alone says almost nothing about what workloads it was designed for and accels at. 16 个计算核心可能是 ANE 中最无趣的部分。Apple 最初的目标是密集的图像处理 CNN 工作负载，这类工作由具有可预测重用性的密集张量归约组成。M1 ANE 的计算核心是一个庞大的乘加 (MAC) 单元并行阵列，但仅凭这一点，几乎无法说明它究竟是为哪些工作负载设计的，以及它在哪些方面具有加速优势。

A convolutional layer does a dot product between an activation window and learned kernel weights, and attention does a dot product between a query and key vector. A dot product is a dot product, and a MAC does just that. What specialized ANE to the 2017 CNN models is not the MAC, but dataflow surrounding the MACs: when and where MAC inputs and outputs enter, stay, move. The assumption that transformers broke, especially with autoregressive decode, was predictable reuse patterns, which the ANE exploited to architect a dataflow efficient enough to run on phones. The M5 decision confirms that ANE’s compute core remained still useful for transformers, but inside a different dataflow. 卷积层是在激活窗口和学习到的内核权重之间进行点积，而注意力机制是在查询向量和键向量之间进行点积。点积就是点积，MAC 单元做的正是这件事。使 ANE 专门针对 2017 年 CNN 模型进行优化的并不是 MAC 本身，而是围绕 MAC 的数据流：MAC 的输入和输出何时、何地进入、停留和移动。Transformer（尤其是自回归解码）打破了“可预测重用模式”这一假设，而 ANE 正是利用了这种模式来构建足够高效的数据流，从而在手机上运行。M5 的决策证实了 ANE 的计算核心对于 Transformer 依然有用，只是需要置于不同的数据流中。

Still, here’s the datapath inside each of the 16 compute cores: 尽管如此，以下是 16 个计算核心中每一个内部的数据通路：

[Diagram omitted / 图表略]

Multiply-Accumulate

乘加运算 (MAC)

ANE has 16 parallel compute cores. Each compute core has 128 FP16 (or 256 INT8) parallel multiply-accumulate (MAC) lanes. Each MAC lane performs the recurrence: ANE 拥有 16 个并行计算核心。每个计算核心有 128 个 FP16（或 256 个 INT8）并行乘加 (MAC) 通道。每个 MAC 通道执行以下递归运算：

[ s\leftarrow s+a\times b ]

Multiply two operands (a) and (b), and then add the product to the running sum (accumulator). Repeating the MAC operation over T cycles computes a T-term dot product: 将两个操作数 (a) 和 (b) 相乘，然后将乘积加到运行总和（累加器）中。在 T 个周期内重复 MAC 操作即可计算出 T 项点积：

[ s_T=s_0 + \sum_{t=0}^{T-1} a_t , b_t. ]

A MAC lane thus performs a scalar reduction over time. A 16-core ANE has 2048 parallel MAC lanes, 因此，一个 MAC 通道随时间执行标量归约。16 核 ANE 拥有 2048 个并行 MAC 通道，

[ 128\ \text{lanes/core}\times16\ \text{cores} = 2048\ \text{parallel MAC lanes} ]

So each cycle performs 2048 parallel reductions spatially, with time being the only reduction axis: 因此，每个周期在空间上执行 2048 次并行归约，时间是唯一的归约轴：

[ S_T[q,p] = S_0[q,p] + \sum_{t=0}^{T-1} a_t[q,p],b_t[q]. ]

An individual MAC lane does not know what dimension of the matrix or tensor it is reducing over. It’s important to note that a dot product vs matrix multiplication vs convolution arises from how the operands are mapped and scheduled onto the core. The ANE core (with the exception of kernel memory, discussed later) does not encode a 4-channel CNN layer into the hardware. 单个 MAC 通道并不知道它正在对矩阵或张量的哪个维度进行归约。需要注意的是，点积、矩阵乘法和卷积的区别在于操作数如何映射和调度到核心上。ANE 核心（除了稍后讨论的内核内存外）并没有将 4 通道 CNN 层硬编码到硬件中。

Internally, the MAC datapath consists of a multiplier, adder, and a 32-bit accumulator register. Each cycle, the adder adds the fresh multiplier output with the previous sum, which then becomes the new running sum. 在内部，MAC 数据通路包含一个乘法器、一个加法器和一个 32 位累加器寄存器。每个周期，加法器将新的乘法器输出与之前的总和相加，然后成为新的运行总和。

[Diagram omitted / 图表略]

This feedback path keeps the partial sum in memory local to the MAC lane, so it does not need fetched from an external memory far away, between MAC cycles. 这条反馈路径将部分和保留在 MAC 通道的本地内存中，因此在 MAC 周期之间无需从遥远的外部内存中获取数据。

Regarding resolution, it does fixed-point reduction with FP16 at readout. The multiplier is 16-bit, accumulated in a 32-bit register as Q16.16, then read out as FP16 via sign-extend and etc. Working in integer (hex) FP16 representation, to probe the accumulator range, build a CoreML ANE program that computes a dot product with a vector of all (1)s, so each multiplier results in a bounded v, but the running sum in the accumulator keeps growing: 关于精度，它执行定点归约，并在读取时转换为 FP16。乘法器为 16 位，以 Q16.16 格式在 32 位寄存器中累加，然后通过符号扩展等方式以 FP16 格式读出。使用整数（十六进制）FP16 表示法，为了探测累加器范围，构建一个 CoreML ANE 程序，计算与全 (1) 向量的点积，这样每个乘法器都会产生一个有界的 v，但累加器中的运行总和会持续增长：

[ s=\sum_{i=0}^{255}v=256v. ]

[Table omitted / 表格略]

Since 32768 is itself a valid FP16 word (0x7800), the ANE’s 0x7c00 can’t be FP16 output overflow, the clamp happens inside the accumulator, at (2^{15}). Thus the accumulator saturates at (2^{15}), exactly the range of a signed 32-bit fixed-point value with 16 fractional bits. 由于 32768 本身是一个有效的 FP16 字 (0x7800)，ANE 的 0x7c00 不可能是 FP16 输出溢出，钳位发生在累加器内部，即 (2^{15}) 处。因此，累加器在 (2^{15}) 处饱和，这正好是有 16 位小数的 32 位有符号定点值的范围。

Nonlinear Activation

非线性激活

For a fused layer, the ANE computes: 对于融合层，ANE 计算：

[ y = f(\sum_k x_k w_k + b) ]

Importantly, completed MAC sums feed directly into the post-MAC activation block, avoiding an intermediate me 重要的是，完成的 MAC 总和直接馈送到 MAC 后激活模块，避免了中间的…