Build specialized multimodal models with LFM3

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

LFM3 is our new generation of Liquid Foundation Models (LFMs), built to be the starting point for specialized intelligence. It brings text, vision, audio, omni, and decision models onto one shared backbone, all in the 2B parameter range. This means you can pick the model for your modality, fine-tune it on your own data on a single GPU, and deploy it where your application runs.

LFM3 是我们新一代的液体基础模型 (LFMs)，旨在成为专业智能的起点。它将文本、视觉、音频、全能和决策模型整合到一个共享的主干网络中，所有模型均处于 2B 参数量级。这意味着您可以根据模态选择模型，在单个 GPU 上使用自己的数据进行微调，并将其部署在您的应用程序运行的地方。

LFM3 brings four main changes compared to our previous LFM2 family: More capability per parameter: a new tri-hybrid architecture and training recipe allow LFM3-1.7B to outperform LFM2.5-2.6B while using 35% fewer parameters. Every modality on one backbone: six 2B-class models (text, vision, audio, omni, and decision) trained from LFM3-1.7B-Base, including our first omni model and video input for our vision model. Built for the edge: the text model fits in about 1 GB at 4-bit precision, new in-house vision and audio encoders speed up processing, and the text model’s 256K context window lets agents run on-device. Made to be specialized: every model fine-tunes on a single GPU, so you can adapt it to your data, terminology, and deployment environment.

与之前的 LFM2 系列相比，LFM3 带来了四个主要变化：单位参数能力更强：全新的三混合架构和训练方案使 LFM3-1.7B 在参数减少 35% 的情况下，性能超越了 LFM2.5-2.6B。所有模态共享一个主干：六个 2B 级别的模型（文本、视觉、音频、全能和决策）均基于 LFM3-1.7B-Base 训练，包括我们的首个全能模型以及视觉模型的视频输入功能。专为边缘设备打造：文本模型在 4-bit 量化下仅需约 1 GB 空间，全新的自研视觉和音频编码器加速了处理速度，且文本模型 256K 的上下文窗口使智能体能够在设备端运行。专为专业化设计：每个模型都可以在单个 GPU 上进行微调，因此您可以根据自己的数据、术语和部署环境进行适配。

How we built a more expressive backbone: LFM2 showed that a hybrid of short convolutions and attention can run fast on-device without sacrificing quality. For LFM3, we used our hardware-in-the-loop architecture search to redesign that architecture for agentic use cases, which need more capability per parameter and fast inference over long contexts. The search now covers 14 device and backend combinations, from phones and a Raspberry Pi 5 to laptops, cloud CPUs, GPUs, and the Qualcomm Hexagon NPU.

我们如何构建更具表现力的主干网络：LFM2 表明，短卷积与注意力机制的混合架构可以在不牺牲质量的前提下在设备端快速运行。对于 LFM3，我们利用硬件在环架构搜索重新设计了该架构，以满足智能体用例的需求，这些用例需要更高的单位参数能力以及长上下文下的快速推理。目前的搜索涵盖了 14 种设备和后端组合，从手机、树莓派 5 到笔记本电脑、云端 CPU、GPU 以及高通 Hexagon NPU。

LFM3-1.7B-Base has 24 layers and four main changes: Tri-hybrid layer mix: 13 gated short convolutions, 5 sliding-window attention (SWA) layers with a 256-token window, and 6 global attention layers. The convolutions use a kernel size of 7, up from 3 in LFM2. Head-wise gated attention: a sigmoid gate scales each head’s output, so a head can switch off when it has nothing to add. This can help suppress activation outliers that hurt quantization. Partial RoPE: global attention layers rotate only the highest-frequency half of each head, so the other half matches on content regardless of distance. SWA layers keep full RoPE. Sandwich norm: each block normalizes both its input and its output to keep activations well-scaled.

LFM3-1.7B-Base 拥有 24 层，并包含四个主要变化：三混合层组合：13 层门控短卷积、5 层具有 256 token 窗口的滑动窗口注意力 (SWA) 层，以及 6 层全局注意力层。卷积核大小从 LFM2 的 3 增加到了 7。头级门控注意力：通过 sigmoid 门控缩放每个头的输出，使某个头在没有信息贡献时可以关闭。这有助于抑制影响量化的激活异常值。部分 RoPE：全局注意力层仅对每个头的高频部分进行旋转，使另一半在内容匹配时不受距离影响。SWA 层保持完整的 RoPE。三明治归一化：每个模块对其输入和输出进行归一化，以保持激活值的良好缩放。

The payoff shows at long context. Per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices, and with 8K tokens in context it is about 25% faster. LFM3-1.7B-Base was pre-trained on 22T tokens with the same 128K vocabulary as LFM2.5-8B-A1B. The final 3.5T tokens are a mid-training stage on curated data with a larger share of agentic traces and tool calls, which also extends the context window to 256K tokens.

其成效在长上下文中显现。在边缘设备上，对于短提示词，LFM3-1.7B 的单位参数性能与 LFM2-1.2B 持平，而在 8K token 上下文中，其速度提升了约 25%。LFM3-1.7B-Base 使用与 LFM2.5-8B-A1B 相同的 128K 词表，在 22T token 上进行了预训练。最后的 3.5T token 是在精选数据上进行的训练阶段，其中包含了更大比例的智能体轨迹和工具调用，这也将上下文窗口扩展到了 256K token。

One backbone for six models: Every LFM3 model branches from LFM3-1.7B-Base. Instead of running one large general-purpose model, you pick the one that fits your inputs, customize it, and run it on-device. The vision and audio models also get two new in-house encoders: Vision encoder: it combines convolution and attention and uses 3x less compute than the SigLIP2 encoder in previous LFM VL models. Our 164M-parameter encoder matches Qwen3.5-2B’s vision encoder in accuracy while requiring 2.2x fewer FLOPs (both measured with the same LFM2.5-230M language backbone). For video, it embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length. Audio encoder and LiquidCodec: the audio encoder mixes sliding-window attention and short convolution.

六个模型共享一个主干：每个 LFM3 模型都源自 LFM3-1.7B-Base。您无需运行一个庞大的通用模型，而是可以选择适合您输入需求的一个模型，进行定制并在设备端运行。视觉和音频模型还配备了两个新的自研编码器：视觉编码器：它结合了卷积和注意力机制，计算量比之前 LFM VL 模型中的 SigLIP2 编码器减少了 3 倍。我们 1.64 亿参数的编码器在准确率上与 Qwen3.5-2B 的视觉编码器相当，但所需的 FLOPs 减少了 2.2 倍（两者均使用相同的 LFM2.5-230M 语言主干进行测量）。对于视频，它将每一对采样帧嵌入到一个时间切片中，使时间序列长度减半。音频编码器和 LiquidCodec：音频编码器混合了滑动窗口注意力和短卷积。