本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

microsoft / BitNet

bitnet.cpp 📰 News 07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF < 1 with very few threads on x86 (AVX2) and ARM (NEON) platforms. [Code] [Models] [Report]

bitnet.cpp 📰 新闻 2026年7月23日：📣 我们发布了 VibeASR.cpp — 一个基于 CPU 的实时多语言 ASR 推理引擎，使用 BitNet I2_S 量化，在 x86 (AVX2) 和 ARM (NEON) 平台上仅需极少线程即可实现 RTF < 1。[代码] [模型] [报告]

07/20/2026: 📣 We released BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face — the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs. 1.42x to 2.28x speedup over F16 on BitNet-embedding-0.6B prefill (8 threads) 1.32x to 1.74x speedup over F16 on BitNet-embedding-270M prefill (8 threads) Supports I2_S conversion with optimized kernels on x86 CPUs Lossless inference with 2 bits per weight

2026年7月20日：📣 我们在 Hugging Face 上发布了 BitNet-embedding-0.6B 和 BitNet-embedding-270M — 这是首批 1-bit 嵌入模型，在提供具有竞争力的嵌入质量的同时，显著加快了 CPU 上的推理速度。BitNet-embedding-0.6B 预填充（8 线程）比 F16 快 1.42 倍至 2.28 倍；BitNet-embedding-270M 预填充（8 线程）比 F16 快 1.32 倍至 1.74 倍。支持在 x86 CPU 上使用优化内核进行 I2_S 转换，并实现每权重 2 位的无损推理。

Overview: bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels that support fast and lossless inference of 1.58-bit models on CPU and GPU (NPU support coming next). Try it out via this online demo, or build and run it on your own CPU or GPU.

概述：bitnet.cpp 是 1-bit 大语言模型（如 BitNet b1.58）的官方推理框架。它提供了一套优化内核，支持在 CPU 和 GPU 上对 1.58-bit 模型进行快速且无损的推理（NPU 支持即将推出）。您可以通过在线演示进行尝试，或在自己的 CPU 或 GPU 上构建并运行它。

bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%.

bitnet.cpp 在 ARM CPU 上实现了 1.37 倍至 5.07 倍的加速，模型越大性能提升越明显。此外，它将能耗降低了 55.4% 至 70.0%，进一步提高了整体效率。在 x86 CPU 上，加速范围为 2.37 倍至 6.17 倍，能耗降低了 71.9% 至 82.2%。

Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. Please refer to the technical report for more details.

此外，bitnet.cpp 可以在单颗 CPU 上运行 100B 参数的 BitNet b1.58 模型，达到与人类阅读相当的速度（每秒 5-7 个 token），显著增强了在本地设备上运行大语言模型的潜力。详情请参阅技术报告。

Installation Requirements: python>=3.10, cmake>=3.22, clang>=18. For Windows users, install Visual Studio 2022. In the installer, toggle on at least the following options: Desktop-development with C++, C++-CMake Tools for Windows, Git for Windows, C++-Clang Compiler for Windows, MS-Build Support for LLVM-Toolset (clang). For Debian/Ubuntu users, you can download with Automatic installation script.

安装要求：python>=3.10, cmake>=3.22, clang>=18。Windows 用户请安装 Visual Studio 2022。在安装程序中，至少勾选以下选项：使用 C++ 的桌面开发、Windows 版 C++-CMake 工具、Windows 版 Git、Windows 版 C++-Clang 编译器、LLVM 工具集 (clang) 的 MS-Build 支持。Debian/Ubuntu 用户可以使用自动安装脚本进行下载。