Speeding Up a Python Service with CinderX: JIT and Static Typing

Every Python optimizer has a number: how many times faster it is. It is measured on kernels, sorting, tree traversal, and arithmetic in a loop. The service works differently: the handler queries the database, performs calculations in NumPy, serializes the response, and the bytecode to which this number applies may be almost entirely absent from it. CinderX accelerates the bytecode, and this number does not extend beyond the bytecode. The proportion of bytecode in your service is a property of your code, not the extension, and until it is calculated, the decision of “whether or not to use it” is based on guesswork.

每个 Python 优化器都有一个指标：它能带来多少倍的性能提升。这个指标通常是在内核、排序、树遍历和循环算术运算中测得的。但实际服务的工作方式不同：处理程序查询数据库、在 NumPy 中执行计算、序列化响应，而该指标所适用的字节码在其中可能几乎不存在。CinderX 加速的是字节码，而这个性能提升指标并不会超出字节码的范畴。你服务中字节码的占比取决于你的代码，而非扩展本身；在计算出这一比例之前，是否使用它的决定只能基于猜测。

What Is CinderX?

什么是 CinderX？

CinderX is a CPython extension: a binary package that is installed into an existing interpreter. It replaces the frame evaluator (the function that CPython calls to execute each frame), and execution proceeds through it from that point on. Inside:

• JIT: compiles the entire object code into machine code.
• Static Python: a separate compiler for a typed subset of Python. It generates its own opcodes, which the JIT then translates into machine code.
• Parallel garbage collector: parallelizes the two garbage collection phases. Enabled by a call.
• Lightweight frames: in a compiled frame, only the subset of fields required by the machine code itself is filled in. The rest is completed if the runtime requests a full frame. Works in conjunction with the JIT.
• Library primitives: typed containers and primitive equivalents of built-in functions.

CinderX 是一个 CPython 扩展：一个安装到现有解释器中的二进制包。它替换了帧评估器（CPython 调用以执行每个帧的函数），此后执行过程将通过它进行。其内部包含：

• JIT：将整个对象代码编译为机器码。
• Static Python：一个针对 Python 类型化子集的独立编译器。它生成自己的操作码，然后由 JIT 翻译成机器码。
• 并行垃圾回收器：将垃圾回收的两个阶段并行化，通过调用启用。
• 轻量级帧：在编译后的帧中，仅填充机器码本身所需的字段子集。如果运行时请求完整帧，则补全其余部分。与 JIT 配合工作。
• 库原语：类型化容器和内置函数的原始等价物。

JIT

JIT

The stock CPython 3.14 already includes a JIT: --enable-experimental-jit from PEP 744, also known as tier 2. So the question should be phrased differently: why do we need another one? These are two completely different compilers.

原生 CPython 3.14 已经包含了一个 JIT：即 PEP 744 中提到的 --enable-experimental-jit，也称为 tier 2。因此，问题应该换个问法：为什么我们需要另一个 JIT？因为这是两个完全不同的编译器。

(Comparison table omitted for brevity)

What You Need to Build It

构建需求

• Nothing vs LLVM 19, clang only

• 无（原生） vs 仅限 LLVM 19, clang

CPython: Copying and Patches

CPython：复制与补丁 (Copy-and-Patch)

The idea behind “copy-and-patch” is to avoid having a compiler at runtime altogether. The CPython tree contains the file Tools/jit/template.c. This is the body of a single UOP (micro-operation into which the bytecode is decomposed), wrapped in a function with the signature (frame, stack_pointer, tstate). Everything that will vary at runtime is declared as holes with descriptive names: _JIT_OPARG, _JIT_OPERAND0, _JIT_TARGET, _JIT_CONTINUE.

“复制与补丁”背后的理念是完全避免在运行时使用编译器。CPython 源码树中包含 Tools/jit/template.c 文件。这是单个 UOP（字节码分解成的微操作）的主体，被包装在一个签名为 (frame, stack_pointer, tstate) 的函数中。所有在运行时会发生变化的内容都被声明为带有描述性名称的“空洞（holes）”：_JIT_OPARG、_JIT_OPERAND0、_JIT_TARGET、_JIT_CONTINUE。

During the CPython build phase, Clang compiles this stencil once for each uop and assembles the resulting pieces of machine code into a stencil table. This explains the requirement for LLVM 19 with clang: the template relies on “musttail,” which GCC does not support.

在 CPython 构建阶段，Clang 会为每个 uop 编译一次该模板，并将生成的机器码片段组装成一个模板表。这就解释了为什么需要 LLVM 19 和 clang：该模板依赖于 GCC 不支持的 “musttail” 特性。

In runtime, _PyJIT_Compile receives a track and does exactly what it says on the box: it follows the track, copies the desired stencil into executable memory, and fills the holes with real values. No parsing, no analysis, no instruction selection, that’s why it’s so cheap.

在运行时，_PyJIT_Compile 接收一条追踪路径（track），并按字面意思执行：它跟踪路径，将所需的模板复制到可执行内存中，并用真实值填充空洞。没有解析、没有分析、没有指令选择，这就是它成本极低的原因。

The chaining is held together by tail calls. Each stencil ends like this: #define PATCH_JUMP(ALIAS) do { PATCH_VALUE(jit_func_preserve_none, jump, ALIAS); __attribute__((musttail)) return jump(frame, stack_pointer, tstate); } while (0)

链式调用通过尾调用连接在一起。每个模板的结尾如下： #define PATCH_JUMP(ALIAS) do { PATCH_VALUE(jit_func_preserve_none, jump, ALIAS); __attribute__((musttail)) return jump(frame, stack_pointer, tstate); } while (0)

In other words, a trace is a chain of functions, each of which jumps to the next without growing the stack. The call convention preserve_none allows frame, stack_pointer, and tstate to be stored in registers throughout the entire chain. A value is passed from one uop to another via memory. Since each uop is a separately compiled unit, the compiler has no way to pass a value from one uop to another via a register. The value produced by the _LOAD_FAST is placed on the frame value stack (in memory), from where the next uop will retrieve it. There is no intermediate representation. There is nothing to distribute.

换句话说，追踪是一系列函数的链，每个函数跳转到下一个函数时都不会增加栈空间。调用约定 preserve_none 允许在整个链中将 frame、stack_pointer 和 tstate 存储在寄存器中。值通过内存从一个 uop 传递到另一个 uop。由于每个 uop 都是单独编译的单元，编译器无法通过寄存器在 uop 之间传递值。_LOAD_FAST 产生的值被放置在帧值栈（内存中），下一个 uop 从那里获取它。这里没有中间表示，也没有什么需要分发的。

What “copy-and-patch” achieves: the dispatch loop, with its switch and unpredictable branch, disappears. Plus, the abstract interpreter in Python/optimizer_analysis.c runs a data flow analysis along the path and eliminates what is provably redundant, duplicate type version checks.

“复制与补丁”实现了什么：分发循环（及其 switch 语句和不可预测的分支）消失了。此外，Python/optimizer_analysis.c 中的抽象解释器会沿着路径运行数据流分析，并消除可证明是冗余的内容，例如重复的类型版本检查。

What it doesn’t do: adjacent UOPs don’t share a common register, intermediate values are passed through the frame stack in memory, and operations on the reference counter aren’t reordered based on liveliness. The analyzer does one thing with counters: it uses the borrowing variant for loading constants if the constant is immortal.

它没做的是：相邻的 UOP 不共享通用寄存器，中间值通过内存中的帧栈传递，引用计数操作不会根据活跃度进行重排序。分析器对计数器做了一件事：如果常量是“永生（immortal）”的，它会使用借用变体来加载常量。

The path is constructed as follows. translate_bytecode_to_trace projects it along the hot path from the backtrack, and at each fork, it checks the branching history. Confidence starts at 1,000 and is multiplied by the fraction of matching transitions. If it drops below 333, the projection is terminated. The path is no longer than 800 uops and extends no deeper than five frames.

路径构建如下：translate_bytecode_to_trace 从回溯中沿着热路径进行投影，并在每个分支点检查分支历史。置信度从 1,000 开始，并乘以匹配转换的分数。如果降至 333 以下，投影终止。路径长度不超过 800 个 uop，深度不超过 5 个帧。