Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS
Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS
构建低延迟多语言语音代理：利用 NVIDIA Magpie TTS 实现开放权重与完全部署控制
Every voice interaction has a latency budget. By the time a user hears your application respond, you’ve already spent precious milliseconds capturing audio, transcribing speech, running an LLM, retrieving context, and generating a response. Text-to-speech (TTS) is the final step — and the one users notice most. If speech generation is slow, the whole experience feels slow. The more of that pipeline you can run and tune yourself, the more of the latency budget you get back.
每一次语音交互都有一个延迟预算。当用户听到应用程序响应时，你已经花费了宝贵的几毫秒来捕获音频、转录语音、运行大语言模型（LLM）、检索上下文并生成响应。文本转语音（TTS）是最后一步，也是用户最能感知的一步。如果语音生成缓慢，整个体验就会显得迟钝。你能够自行运行和调优的流水线环节越多，你所能节省的延迟预算就越多。
Voice AI is moving fast. Integrated speech models offer simplicity — one API call, audio in, audio out — but they trade the ability to fine-tune each component for your domain, swap in better models as they ship, enforce data residency, and understand exactly where latency is coming from. For more control, a cascaded architecture — purpose-built ASR, TTS, and LLM components running together — keeps each layer independently tunable and deployable on infrastructure you own.
语音人工智能发展迅速。集成式语音模型虽然提供了简便性——只需一次 API 调用，即可实现音频输入和输出——但它们牺牲了针对特定领域微调各组件、随模型更新进行替换、强制执行数据驻留以及精确追踪延迟来源的能力。为了获得更多控制权，级联架构（将专用的 ASR、TTS 和 LLM 组件协同运行）使每一层都可以独立调优，并部署在你自己的基础设施上。
NVIDIA Magpie Multilingual TTS is built for that. With open weights, production-ready NVIDIA NIM, and support for 12 languages, you can deploy multilingual speech inside your own infrastructure, optimize latency for your workload, and customize the model for your domain — end to end, in your own environment. The latest release expands multilingual coverage with Modern Standard Arabic, Korean, and Brazilian Portuguese, while improving quality across many existing languages through updated training data and model improvements.
NVIDIA Magpie 多语言 TTS 正是为此而生。凭借开放权重、生产就绪的 NVIDIA NIM 以及对 12 种语言的支持，你可以在自己的基础设施内部署多语言语音功能，针对工作负载优化延迟，并为你的领域定制模型——在自己的环境中实现端到端的控制。最新版本通过增加现代标准阿拉伯语、韩语和巴西葡萄牙语，扩大了多语言覆盖范围，同时通过更新训练数据和模型改进，提升了许多现有语言的质量。
Whether you’re building customer support agents, healthcare assistants, enterprise copilots, translation systems, or conversational AI applications, Magpie provides an open foundation for production voice AI.
无论你是在构建客户支持代理、医疗助手、企业副驾驶（Copilot）、翻译系统还是对话式 AI 应用，Magpie 都为生产级语音 AI 提供了一个开放的基础。
Voice AI Is Becoming Multilingual by Default
语音 AI 正默认走向多语言化
Today’s voice applications don’t serve a single language. Global customer support, enterprise assistants, healthcare documentation, retail automation, and translation workflows increasingly require natural conversations across multiple languages — all while maintaining low latency. Supporting more languages is only part of the challenge. Developers also need the ability to: Deploy where their data lives, meet enterprise privacy requirements, customize pronunciation and voices, predict latency under production workloads, and scale on their own infrastructure. Open models change what’s possible on every one of these.
如今的语音应用不再局限于单一语言。全球客户支持、企业助手、医疗文档、零售自动化和翻译工作流越来越需要跨多种语言的自然对话，同时还要保持低延迟。支持更多语言只是挑战的一部分。开发者还需要具备以下能力：在数据所在地进行部署、满足企业隐私要求、自定义发音和语音、预测生产工作负载下的延迟，以及在自己的基础设施上进行扩展。开放模型改变了在上述每一个环节的可能性。
One Open Model, Twelve Languages
一个开放模型，十二种语言
Magpie TTS Multilingual is a 364M-parameter open-weights model supporting: English, Spanish, French, German, Italian, Vietnamese, Mandarin, Hindi, Japanese, Modern Standard Arabic (new), Korean (new), and Brazilian Portuguese (new). Each language includes male and female speaker voices through a shared multilingual speaker representation. This release also improves multilingual flexibility with expanded code-switching support for Hindi and Japanese, enabled through IPA grapheme-to-phoneme processing and custom pronunciation dictionaries — making it easier to accurately pronounce names, technical terminology, and mixed-language content. Instead of maintaining separate TTS models for different regions, developers can build multilingual applications on a single open foundation.
Magpie TTS Multilingual 是一个拥有 3.64 亿参数的开放权重模型，支持：英语、西班牙语、法语、德语、意大利语、越南语、普通话、印地语、日语、现代标准阿拉伯语（新增）、韩语（新增）和巴西葡萄牙语（新增）。每种语言都通过共享的多语言说话人表示包含男女声。此版本还通过扩展对印地语和日语的语码转换支持，提高了多语言灵活性，这是通过 IPA 字素到音素处理和自定义发音词典实现的——使得准确发音人名、技术术语和混合语言内容变得更加容易。开发者无需为不同地区维护单独的 TTS 模型，而是在一个统一的开放基础上构建多语言应用。
The Latency Your Users Actually Notice
用户真正感知的延迟
In conversational AI, text-to-speech is the final stage before users hear a response. That makes Time to First Audio (TTFA) — the delay between speech generation beginning and the first audio reaching the user — one of the most important latency metrics in a voice pipeline. Because Magpie TTS can be deployed inside your own environment, the latency you measure is the server-side latency you actually control, with no managed-service round-trip in the number.
在对话式 AI 中，文本转语音是用户听到响应前的最后阶段。这使得“首音频时间”（TTFA，即从语音生成开始到用户接收到第一段音频之间的延迟）成为语音流水线中最重要的延迟指标之一。由于 Magpie TTS 可以部署在你自己的环境中，你所测量的延迟就是你实际控制的服务器端延迟，其中不包含托管服务的往返时间。
(Table omitted for brevity, but the text notes:) At 32ms on B200, Magpie’s TTFA leaves the rest of the latency budget for ASR and LLM processing — keeping total end-to-end latency within the sub-200ms window natural conversation requires. Across NVIDIA GPUs, Magpie delivers first audio in 32–79ms on a single stream. At 64 concurrent streams, B200 reaches 239ms TTFA while delivering throughput at 320× real time — generating audio more than 300 times faster than it plays back, even under concurrent load.
（表格略）在 B200 上，Magpie 的 TTFA 仅为 32 毫秒，这为 ASR 和 LLM 处理留出了剩余的延迟预算，使总端到端延迟保持在自然对话所需的 200 毫秒以内。在 NVIDIA GPU 上，Magpie 在单流下可在 32-79 毫秒内输出首段音频。在 64 个并发流下，B200 可达到 239 毫秒的 TTFA，同时提供 320 倍于实时的吞吐量——即使在并发负载下，生成音频的速度也比播放速度快 300 多倍。
Optimized for Real-Time Speech Generation
针对实时语音生成进行优化
Low latency isn’t accidental. Magpie introduces two complementary architectural improvements that reduce inference time while maintaining speech quality. Frame stacking: The decoder predicts two audio frames during each decoding step rather than one. This cuts the number of decoder iterations in half, shortening generation time and improving throughput. Local transformer: Frame stacking alone would reduce audio quality by introducing dependencies between simultaneously generated codebook tokens. The local transformer…
低延迟并非偶然。Magpie 引入了两种互补的架构改进，在保持语音质量的同时缩短了推理时间。帧堆叠（Frame stacking）：解码器在每个解码步骤中预测两个音频帧，而不是一个。这使解码器迭代次数减半，缩短了生成时间并提高了吞吐量。局部 Transformer（Local transformer）：仅靠帧堆叠会通过在同时生成的码本标记之间引入依赖关系而降低音频质量。局部 Transformer……