PrismML hopes its tiny LLM will change how we all use AI
PrismML hopes its tiny LLM will change how we all use AI
PrismML 希望其微型大模型能改变我们使用 AI 的方式
If AI lab PrismML isn’t on your radar yet, it should be — not because it’s raised gobs of money (it hasn’t yet, just a $22.25 million seed round), but because of the technical minds involved and the potentially industry-changing tech it’s developing. 如果 AI 实验室 PrismML 尚未进入你的视野，那么它现在应该引起你的关注了——这并不是因为它筹集了巨额资金（目前还没有，仅完成了 2225 万美元的种子轮融资），而是因为其背后的技术大牛以及它正在开发的、可能改变行业的技术。
PrismML is betting that capable, high-performing, reasoning large language models don’t, in fact, have to be large. It is making reasoning models so small they can fit on PCs and smartphones. (It’s even rumored to be in talks with Apple, though CEO Babak Hassibi declined to comment on that to TechCrunch.) PrismML 押注认为，功能强大、高性能的推理大语言模型实际上并不一定非要“大”。他们正在制造体积极小的推理模型，使其能够运行在个人电脑和智能手机上。（甚至有传言称他们正在与苹果公司进行洽谈，尽管首席执行官 Babak Hassibi 向 TechCrunch 拒绝就此置评。）
On Thursday, PrismML released Bonsai 2 27B, its latest in a family of models, which compresses Qwen3.8 27B, a widely used open source model from Alibaba, down to 5.9 GB. That’s small enough to fit on a PC and, possibly, a high-end smartphone. It’s a 9x to 10x reduction in memory versus the original. 周四，PrismML 发布了其模型家族中的最新成员 Bonsai 2 27B。该模型将阿里巴巴广泛使用的开源模型 Qwen3.8 27B 压缩到了 5.9 GB。这个体积小到足以装进个人电脑，甚至可能运行在高端智能手机上。与原始模型相比，其内存占用减少了 9 到 10 倍。
PrismML was founded by a group of Caltech researchers and is led by Hassibi, a Caltech professor and an expert in compression technologies. The startup also counts Ion Stoica as an adviser. Stoica is a co-founder of Databricks (and other companies) and the director of Berkeley’s famed Sky Computing Lab, which has birthed many technologies and startups, from Letta to SGLang. PrismML 由一群加州理工学院的研究人员创立，并由加州理工学院教授、压缩技术专家 Hassibi 领导。该初创公司还邀请了 Ion Stoica 担任顾问。Stoica 是 Databricks（及其他公司）的联合创始人，也是伯克利著名 Sky Computing Lab 的主任，该实验室孕育了从 Letta 到 SGLang 等许多技术和初创公司。
PrismML is also backed by investors Khosla Ventures, Cerberus Capital, and Caltech. This startup is certainly not the only company working on LLM compression tech. Multiverse Computing, founded by a well-known professor from Spain’s Donostia International Physics Center, is another. (And Multiverse Computing has raised gobs of cash.) PrismML 还获得了 Khosla Ventures、Cerberus Capital 和加州理工学院的投资。当然，这家初创公司并不是唯一一家致力于大模型压缩技术的公司。由西班牙多诺斯蒂亚国际物理中心的一位知名教授创立的 Multiverse Computing 也是其中之一。（而且 Multiverse Computing 已经筹集了大量资金。）
But Hassibi says that PrismML’s compression tech is unique because its LLMs have lost virtually no performance compared with the originals. Bonsai 2 matches 98% of Qwen’s aggregate benchmark scores. That’s up from the first Bonsai, released a couple of months ago in March, that matched 95%. 但 Hassibi 表示，PrismML 的压缩技术是独一无二的，因为其大模型与原始模型相比几乎没有性能损失。Bonsai 2 的综合基准测试得分达到了 Qwen 的 98%。这比几个月前（3 月份）发布的初代 Bonsai（达到了 95%）有所提升。
That original model has already been downloaded over 11 million times, and PrismML’s even smaller models have been downloaded another 2.6 million times, the company says. So this shows that PrismML’s compression results have improved from one release to the next. 该公司表示，初代模型已被下载超过 1100 万次，而 PrismML 更小的模型也被下载了 260 万次。这表明 PrismML 的压缩成果在每一次发布中都在不断进步。
Whether it could ever get to 100% benchmark performance parity is a question that remains to be seen. Compression will likely always have some impact, Hassibi says. Still, perfect benchmark parity is fairly academic anyway. LLMs are not so accurate in their uncompressed form, and benchmarks not so perfectly reflective of actual tasks, that a 2% degradation would likely meaningfully affect how a model performs in actual use. (Plus, the surrounding software — the harness a model runs inside of — matters a lot when it comes to accuracy, too.) 至于能否达到 100% 的基准性能对等，仍有待观察。Hassibi 表示，压缩很可能总是会产生一些影响。不过，完美的基准对等在很大程度上只是学术层面的讨论。大模型在未压缩状态下本身就并非绝对精确，而基准测试也无法完美反映实际任务，因此 2% 的性能下降不太可能对模型的实际使用效果产生重大影响。（此外，周边软件——即模型运行的框架——在准确性方面也至关重要。）
PrismML says it achieves this by shrinking the “weights” that make up a model — weights are, essentially, the information a model learns and stores during training. Normally, each weight requires 16 bits. PrismML’s approach, called “ternary” weights, simplifies that down to three: +1, −1, or 0. With far smaller values to store for each weight, the model takes up dramatically less space. (For a deeper dive on the compression technique, here’s the project’s Hugging Face page.) PrismML 表示，他们通过缩小构成模型的“权重”来实现这一目标——权重本质上是模型在训练过程中学习和存储的信息。通常，每个权重需要 16 位。PrismML 的方法被称为“三元”权重，将其简化为三个值：+1、-1 或 0。由于每个权重存储的值小得多，模型占用的空间大幅减少。（如需深入了解该压缩技术，请访问该项目的 Hugging Face 页面。）
The startup’s next goal is to apply this compression technique to even bigger models. “The next models that we will release, hopefully in the next couple of months, will be in the several-hundred-billion-parameter range, and I expect it will be easier to retain the intelligence there,” Hassibi told TechCrunch. As model size grows, he added, “There is more room to be able to compress them without losing the intelligence. So I would just say, as a general trend, for larger models, it’s easier to get to 100%.” 该初创公司的下一个目标是将这种压缩技术应用于更大的模型。“我们希望在未来几个月内发布的下一批模型将达到数千亿参数规模，我预计在那里保持智能水平会更容易，”Hassibi 告诉 TechCrunch。他补充说，随着模型规模的增长，“在不损失智能的情况下压缩它们的空间更大。所以我只想说，作为一个总体趋势，对于更大的模型，更容易达到 100% 的性能。”
Stoica tells us that he’s excited for this tech because it’s making it possible for advanced models to run on users’ devices. “You are going to have intelligence at your fingertips, and it’s going to be free because it’s going to run on the device you already bought. It’s also going to be private, because you’re not going to send it to the cloud.” Stoica 告诉我们，他对这项技术感到兴奋，因为它使先进模型在用户设备上运行成为可能。“你将触手可及地获得智能，而且它是免费的，因为它运行在你已经购买的设备上。它也将是私密的，因为你不需要将其发送到云端。”