H3-metal – Native MiniMax-H3 inference for Apple Silicon

h3-metal Native MiniMax-H3 inference for Apple Silicon. The project is being built as a sequence of working vertical slices: deterministic host/model metadata first, then portable Metal block parity, prompt encoding, prompt-to-video/audio, and first/last-frame conditioning and then ordered references. Prompt-to-video/audio, first/last-frame conditioning, and ordered Ref2VA image/video/audio references work end to end. The current work is incremental H3-specific Metal performance and memory optimization on M3 Max and M5 Max.

h3-metal 是针对 Apple Silicon 的原生 MiniMax-H3 推理实现。该项目正通过一系列可工作的垂直切片（vertical slices）进行构建：首先是确定性的主机/模型元数据，随后是可移植的 Metal 块对齐、提示词编码、提示词转视频/音频、首/尾帧条件控制，以及有序参考。目前，提示词转视频/音频、首/尾帧条件控制以及有序的 Ref2VA 图像/视频/音频参考功能已实现端到端运行。当前的工作重点是在 M3 Max 和 M5 Max 上进行针对 H3 的增量式 Metal 性能与内存优化。

Tutorial 1. Build and inspect the model

教程 1. 构建并检查模型

The examples assume that the Hugging Face snapshot is in ./MiniMax-H3 and that FFmpeg and FFprobe are available on PATH. 这些示例假设 Hugging Face 快照位于 ./MiniMax-H3 目录下，且 FFmpeg 和 FFprobe 已添加到系统 PATH 中。

make -j8
mkdir -p outputs
./h3 --info -d ./MiniMax-H3

—info checks the model layout and prints the selected Metal device without mapping all weights or generating media. Run ./h3 —help for the complete CLI reference. —info 参数用于检查模型布局并打印所选的 Metal 设备，而无需映射所有权重或生成媒体文件。运行 ./h3 —help 可获取完整的命令行参考。

Without -p, the same binary starts an Iris-style interactive session: 如果不加 -p 参数，同一个二进制文件将启动一个 Iris 风格的交互式会话：

./h3 -d ./MiniMax-H3 --width 512 --height 512 --steps 6

Type a prompt to generate a numbered video. The session keeps the exact BF16 prompt conditioning, prepared DiT, and video decoder in memory, so repeating a prompt with another seed avoids loading and encoding them again. Useful commands are !status, !seed random, !seconds 2, !show, !save output.mp4, and !cache. Use !help for the full, short list. 输入提示词即可生成带编号的视频。该会话会将精确的 BF16 提示词条件、准备好的 DiT 和视频解码器保留在内存中，因此使用不同种子重复同一个提示词时，无需重新加载和编码。常用的命令包括 !status、!seed random、!seconds 2、!show、!save output.mp4 和 !cache。使用 !help 可查看完整的简短命令列表。

First/last-frame conditioning is persistent in the session: 首/尾帧条件控制在会话中是持久的：

h3> !first opening.png
h3> !last ending.png
h3> The camera moves slowly around the subject.

Use !first clear or !last clear to remove an anchor. Generated videos are written to the session directory printed at startup. For a general Ref2VA conditioning image, use !ref-image PATH instead. Images are appended in order and exposed to the model as <Picture 1>, <Picture 2>, and so on; filenames have no meaning to the model. 使用 !first clear 或 !last clear 可移除锚点。生成的视频会写入启动时打印的会话目录中。对于通用的 Ref2VA 条件图像，请改用 !ref-image PATH。图像会按顺序添加，并以 <Picture 1>、<Picture 2> 等形式呈现给模型；文件名对模型没有实际意义。

h3> !ref-image person.png
h3> Make the person shown in Picture 1 wave to the camera.

!refs lists the current order, !ref-remove N removes one entry, and !refs clear removes them all. Ref2VA references cannot be mixed with !first/!last anchors. !refs 列出当前顺序，!ref-remove N 移除某一项，!refs clear 移除所有项。Ref2VA 参考不能与 !first/!last 锚点混合使用。

2. Make a first fast video

2. 制作第一个快速视频

Start with the validated balanced preset. It generates 22 frames at 24 fps (about 0.92 seconds), displays the evolving middle-video frame after every denoising transition in a supported graphical terminal, and prints phase timings: 从经过验证的平衡预设开始。它以 24 fps 生成 22 帧（约 0.92 秒），在支持的图形终端中显示每次去噪转换后演变的视频中间帧，并打印阶段耗时：

./h3 --profile \
    -d ./MiniMax-H3 \
    -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind." \
    --width 512 --height 512 \
    --frames 22 --steps 20 \
    --layers 45 --reuse 2 \
    --show \
    -o outputs/fox-fast.mp4

This is deliberately not the most aggressive configuration: —steps 20 performs the default 20 denoising passes. —reuse 2 computes 11 fresh denoiser velocities instead of all 20 and extrapolates the skipped transitions. —layers 45 runs 45 of the 50 transformer blocks, reducing both time and unified-memory use. 这特意没有采用最激进的配置：—steps 20 执行默认的 20 次去噪。—reuse 2 计算 11 次新的去噪速度，而不是全部 20 次，并对跳过的转换进行外推。—layers 45 运行 50 个 Transformer 块中的 45 个，从而减少了时间和统一内存的使用。

—show is optional. It supports Kitty/Ghostty and iTerm2/WezTerm/Konsole graphical protocols. It loads a resident preview VAE, displays one representative middle-video frame after every Euler transition, and then displays all final frames. Display dimensions default to 2x so the image has its intended logical size on macOS Retina screens; use —zoom 1 on a non-HiDPI display. This adds preview decode time and roughly 10 GiB of temporary model residency; runs without —show are unchanged. —show 是可选的。它支持 Kitty/Ghostty 和 iTerm2/WezTerm/Konsole 图形协议。它会加载一个常驻预览 VAE，在每次 Euler 转换后显示一个代表性的视频中间帧，最后显示所有最终帧。显示尺寸默认为 2 倍，以便图像在 macOS Retina 屏幕上具有预期的逻辑大小；在非 HiDPI 显示器上请使用 —zoom 1。这会增加预览解码时间和约 10 GiB 的临时模型内存占用；不带 —show 的运行不受影响。

—profile is optional and does not select a different generation path. The first process invocation also pays model loading and filesystem-cache costs. Compare performance using repeated runs, and alternate variants when the machines are warming up because this workload is sensitive to thermal throttling. —profile 是可选的，不会选择不同的生成路径。首次进程调用还需要承担模型加载和文件系统缓存的开销。请通过重复运行来比较性能，并在机器预热时交替测试不同变体，因为该工作负载对热节流非常敏感。

For a very short iteration, request four denoising passes directly: 对于极短的迭代，可以直接请求四次去噪：

./h3 --profile \
    -d ./MiniMax-H3 \
    -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur." \
    --width 512 --height 512 --frames 22 \
    --steps 4 --layers 50 --reuse 1 \
    --show \
    -o outputs/fox-four-step.mp4

—steps N always means exactly N denoising passes. Four through seven passes use the same schedule that won the low-budget comparison; increasing from 4 to 7 progressively improves detail and motion. Keep —reuse 1 at such small budgets so every requested pass runs the model. —show displays one preview after each pass. —steps N 始终意味着精确的 N 次去噪。4 到 7 次去噪使用相同的调度方案，该方案在低预算对比中表现最优；从 4 增加到 7 可逐步改善细节和运动效果。在如此小的预算下请保持 —reuse 1，以便每次请求的去噪都运行模型。—show 会在每次去噪后显示一个预览。

Several tail-heavy schedules were evaluated because most visible cleanup happens late in a long run. They preserved too few early composition updates and produced woven texture, weak motion, or clipped colors. The retained mode uses the released linear base grid with one terminal point. On the 512-square, 22-frame fox test, the selected four-pass result had 0.556 full-video SSIM against a 29-pass reference; an independent surfer test measured 0.547. The four-pass denoise took about 3.5 seconds on M5 Max, versus 26.4 seconds for the reference. 研究人员评估了几种“尾重型”（tail-heavy）调度方案，因为大部分可见的清理工作发生在长运行的后期。但它们保留的早期构图更新太少，导致出现编织纹理、运动微弱或色彩截断。保留的模式使用了带有终点的线性基础网格。在 512x512、22 帧的狐狸测试中，选定的四步结果相对于 29 步参考的完整视频 SSIM 为 0.556；独立的冲浪者测试测得为 0.547。在 M5 Max 上，四步去噪耗时约 3.5 秒，而参考运行耗时 26.4 秒。

For a low-memory run, add —ssd-streaming: 对于低内存运行，请添加 —ssd-streaming：

./h3 --profile \
    -d ./MiniMax-H3 \
    -p "A red fox walks through fresh snow in a pine forest." \
    --width 512 --height 512 --frames 22 --steps 20 \
    --layers 50 --reuse 1 --ssd-streaming \
    -o outputs/fox-ssd.mp4

This uses the original BF16 checkpoint without conversion or quantization. It keeps two DiT blocks in memory and reads the next block from SSD while the GPU runs the current one. On M5 Max, tracked DiT storage fell from about 36.5 GiB to 2.0 GiB at 512 square and 2.1 GiB at 864x480. A warm 50-block forward measured 1.35 versus 2.49 seconds at 512 square (84% slower), and 2.14 versus 2.68 seconds at 864x480 (26% slower). These are comparisons against the same full-residency BF16 path, and the results were byte-identical in both checks. The 2.0—2.1 GiB figure is the DiT’s tracked tensor storage, not total system RAM. Prompt encoding and the two VAEs run in separate phases rather than adding their full peaks to it; the OS, media buffers, and output resolution still… 这使用了原始的 BF16 检查点，无需转换或量化。它将两个 DiT 块保留在内存中，并在 GPU 运行当前块的同时从 SSD 读取下一个块。在 M5 Max 上，DiT 的跟踪存储占用从约 36.5 GiB 降至 512 分辨率下的 2.0 GiB 和 864x480 分辨率下的 2.1 GiB。在 512 分辨率下，预热后的 50 块前向传播耗时为 1.35 秒（对比全内存模式的 2.49 秒，慢 84%）；在 864x480 下为 2.14 秒（对比 2.68 秒，慢 26%）。这些是与相同全驻留 BF16 路径的对比，两次检查的结果在字节上完全一致。2.0—2.1 GiB 的数据是 DiT 跟踪的张量存储，而非总系统内存。提示词编码和两个 VAE 在单独的阶段运行，而不是将其峰值内存叠加在上面；操作系统、媒体缓冲区和输出分辨率仍然…