Voxtral 3
本文为原文前 6,000 字符的节选翻译,完整内容请查看原文。
Voxtral 3
Sure. What’s the reference number? 当然可以。参考编号是多少?
get_referral(ref=“4420 1187”) get_referral(ref=“4420 1187”)
Merci de recevoir ce patient pour un avis dermatologique. Traitement en cours : crème à l’hydrocortisone. Dr Martin, Clinique du Parc. 感谢您接收这位患者进行皮肤科会诊。目前正在进行的治疗:氢化可的松乳膏。Clinique du Parc 的 Martin 医生。
It says, “Merci de recevoir ce patient pour un avis dermatologique.” Your doctor is asking a dermatologist, Dr Martin, to see you. The next slot is Tuesday at 5:45 pm. Shall I book it? 上面写着:“感谢您接收这位患者进行皮肤科会诊。”您的医生正在请求皮肤科医生 Martin 医生为您看诊。下一个空档是周二下午 5:45。需要我为您预约吗?
book_appointment(slot=“Nov 3 17:45”) book_appointment(slot=“Nov 3 17:45”)
Done. Bring the letter, your carte Vitale, and I’ll text you the details in English. 已完成。请带上转诊信和您的医保卡(carte Vitale),我会用英语给您发送详细信息。
Hi, my doctor gave me a referral letter in French. Can you book it for me? 你好,我的医生给了我一封法语的转诊信。你能帮我预约吗?
4402… sorry, 4420 1187. 4402……抱歉,是 4420 1187。
Voxtral 3 introduces a cutting-edge voice agent SDK and state-of-the-art speech models designed for real-time, multi-step conversations across 50+ languages. It combines accurate transcription, natural speech synthesis with emotional adaptability, and deep tool integration for seamless workflow automation. Available via Mistral’s EU-hosted API or self-deployment, it ensures data sovereignty while delivering low-latency, human-like interactions for contact centers, virtual assistants, and hands-free copilots. Voxtral 3 引入了一套尖端的语音代理 SDK 和最先进的语音模型,专为 50 多种语言的实时、多步骤对话而设计。它结合了精准的转录、具备情感适应性的自然语音合成,以及深度工具集成,从而实现无缝的工作流自动化。该产品可通过 Mistral 的欧盟托管 API 或自部署方式获取,在确保数据主权的同时,为呼叫中心、虚拟助手和免提副驾驶提供低延迟、类人的交互体验。
Speech transcription and voice generation have come a long way, but building end-to-end voice agents that can listen, reason, interact, and handle complex workflows remains difficult. 语音转录和语音生成技术已经取得了长足进步,但构建能够倾听、推理、交互并处理复杂工作流的端到端语音代理仍然困难重重。
A reliable voice agent needs to do four things very well (and simultaneously): 一个可靠的语音代理需要非常出色地(且同时)完成四件事:
Listen accurately across languages, accents, speakers, and interruptions, 在跨语言、口音、说话人和中断的情况下准确倾听,
Reason deeply about what users mean in specific contexts, 针对特定语境深入推理用户的意图,
Speak naturally with emotion and expression, and 以情感和表现力自然地说话,以及
Take action over multiple steps through connected systems. 通过连接的系统执行多步骤操作。
Getting all four to work together reliably in real time remains a challenge. 让这四项功能在实时环境下可靠地协同工作仍然是一项挑战。
To address this, we introduce Voxtral 3: a new family of state-of-the-art speech-to-text and text-to-speech models, alongside a voice agent SDK that brings listening, thinking, tool calling, and speaking into a single real-time streaming pipeline. 为了解决这个问题,我们推出了 Voxtral 3:这是一个全新的最先进的语音转文字和文字转语音模型系列,以及一个将倾听、思考、工具调用和说话整合到单一实时流式传输管道中的语音代理 SDK。
Voxtral 3 is built to create natural conversational experiences, from contact-center agents and service desks to virtual assistants and hands-free copilots. With support for dozens of languages, expressive speech, complex reasoning, and deep integration with your own tools and data, agents built with Voxtral can interpret nuanced intent, reason through sophisticated requests, and execute multi-step actions across connected systems. Voxtral 3 旨在创造自然的对话体验,适用于从呼叫中心代理和服务台到虚拟助手和免提副驾驶的各种场景。凭借对数十种语言的支持、富有表现力的语音、复杂的推理能力以及与您自有工具和数据的深度集成,使用 Voxtral 构建的代理能够解读细微的意图,推理复杂的请求,并在连接的系统中执行多步骤操作。
As with other Mistral models, the Voxtral 3 family is designed for sovereign deployment. Available today through Mistral’s EU-hosted API or for self-deployment on your own infrastructure, it gives organizations control over where audio is processed, how data is handled, and whether conversations are retained. 与其他 Mistral 模型一样,Voxtral 3 系列专为自主部署而设计。它现已通过 Mistral 的欧盟托管 API 或在您自己的基础设施上进行自部署,使组织能够控制音频的处理位置、数据的处理方式以及是否保留对话记录。
Accurate transcription across 50+ languages. The speech-to-text model, Voxtral Small Transcribe 3, is state-of-the-art in multilingual transcription, identifying who said what (diarization), using keyphrases to improve its accuracy (context-biasing). It detects when someone has finished speaking so agents can respond without delay (end-of-turn detection). 跨 50 多种语言的精准转录。语音转文字模型 Voxtral Small Transcribe 3 在多语言转录方面处于领先地位,能够识别谁说了什么(说话人日志),并使用关键短语来提高准确性(上下文偏置)。它能检测到说话人何时结束发言,以便代理能够立即响应(轮次结束检测)。
Natural speech that adapts to the conversation. The text-to-speech model, Voxtral Mini TTS 3, introduces new voices that adjust their emotion, pace, and emphasis based on conversational context, with support for 35+ languages and 50+ dialects. 适应对话的自然语音。文字转语音模型 Voxtral Mini TTS 3 引入了新的语音,这些语音会根据对话语境调整情感、语速和强调重点,并支持 35 多种语言和 50 多种方言。
Human-like intelligence at your (spoken) command. The new voice agent SDK combines listening, reasoning, tool calling, and speaking into a real-time experience. Agents can retrieve information, interact with business systems, and complete tasks during live conversations over phone or web. 听候您(口头)调遣的类人智能。新的语音代理 SDK 将倾听、推理、工具调用和说话整合为一种实时体验。代理可以在电话或网络上的实时对话中检索信息、与业务系统交互并完成任务。
Pro Voice for distinctly accurate voice adaptation. For professional use cases where basic voice cloning does not suffice, Voxtral’s Pro Voice adaptation is able to reproduce a speaker’s distinctive character, emotional range, and speaking style across languages, using just 30 minutes of audio. 用于精准语音适配的 Pro Voice。对于基础语音克隆无法满足的专业用例,Voxtral 的 Pro Voice 适配功能仅需 30 分钟的音频,即可跨语言重现说话人独特的个性、情感范围和说话风格。
Your voice AI, on your infrastructure. Use Mistral’s EU-hosted API or deploy Voxtral 3 on your own infrastructure. Keep sensitive conversations within your environment and control whether any audio is retained. 您的语音 AI,部署在您的基础设施上。使用 Mistral 的欧盟托管 API 或在您自己的基础设施上部署 Voxtral 3。将敏感对话保留在您的环境中,并控制是否保留任何音频。
Try Voxtral now | Learn more 立即试用 Voxtral | 了解更多
Voxtral Transcribe 3 represents a significant milestone across transcription, diarization (who spoke when) and context biasing capabilities. The supported list of languages is expanded to 50, including 24 official EU languages and many low-resource Asian and Indic languages. Voxtral Transcribe 3 代表了在转录、说话人日志(谁在何时说话)和上下文偏置能力方面的一个重要里程碑。支持的语言列表已扩展至 50 种,包括 24 种欧盟官方语言以及许多资源匮乏的亚洲和印度语言。
The Small version achieves state-of-the-art results across diarization and multilingual transcription, including in English, French and Spanish. It outperforms Whisper, Scribe v2 and Gemini 3.5 Transcribe. The Mini variant is 4B parameters, enabling fast and accurate transcriptions at a price of $0.0035/min. Small 版本在说话人日志和多语言转录方面取得了最先进的成果,包括英语、法语和西班牙语。它的表现优于 Whisper、Scribe v2 和 Gemini 3.5 Transcribe。Mini 变体拥有 40 亿参数,能够以每分钟 0.0035 美元的价格实现快速且准确的转录。
Both models support context biasing, where key terms and free-form text can be given to bias the transcription and improve recall on technical terms. This enables them to be used in production settings with domain-specific vocabulary without the need for fine-tuning. 两种模型都支持上下文偏置,即可以提供关键术语和自由文本来引导转录,并提高对技术术语的召回率。这使得它们无需微调即可在具有特定领域词汇的生产环境中使用。
Multilingual Transcription 多语言转录
Word Error Rate — lower is better 词错误率 (WER) — 越低越好
Macro-average WER over the intersection of 45 languages in the FLEURS benchmark supported by all seven models. WER is used for all languages except Japanese, Korean, Thai, and Chinese, which use CER. Lower is better. 在所有七个模型支持的 FLEURS 基准测试中,45 种语言交集的宏平均 WER。除日语、韩语、泰语和中文使用 CER 外,所有语言均使用 WER。数值越低越好。
Forced = language specified; Inferred = language auto-detected. Languages: AR, BN, BG, CA, ZH, CS, DA, NL, EN, ET, FI, FR, DE, EL, GU, HI, HU, IS, ID, IT, JA, KN, KO, LV, LT, MK, MS, ML, MR, NO, FA, PL, PT, RO, RU, SK, SL, ES, SV, FIL, TE, TH, TR, UK, VI Forced = 指定语言;Inferred = 自动检测语言。语言包括:阿拉伯语、孟加拉语、保加利亚语、加泰罗尼亚语、中文、捷克语、丹麦语、荷兰语、英语、爱沙尼亚语、芬兰语、法语、德语、希腊语、古吉拉特语、印地语、匈牙利语、冰岛语、印尼语、意大利语、日语、卡纳达语、韩语、拉脱维亚语、立陶宛语、马其顿语、马来语、马拉雅拉姆语、马拉地语、挪威语、波斯语、波兰语、葡萄牙语、罗马尼亚语、俄语、斯洛伐克语、斯洛文尼亚语、西班牙语、瑞典语、菲律宾语、泰卢固语、泰语、土耳其语、乌克兰语、越南语。
Concatenated Minimum Permutation WER — lower is better 级联最小排列词错误率 (cpWER) — 越低越好
Macro-average concatenated minimum permutation WER (cpWER) over public diarization benchmarks: AMI-IHM, AMI-SDM, Switchboard, Switchboard CallHome, AISHELL-4. For maximum compatibility of API baselines, we use 30 minute versions of AMI and AISHELL. Lower is better. 公共说话人日志基准测试(AMI-IHM、AMI-SDM、Switchboard、Switchboard CallHome、AISHELL-4)上的宏平均级联最小排列词错误率 (cpWER)。为了最大限度地兼容 API 基准,我们使用了 30 分钟版本的 AMI 和 AISHELL。数值越低越好。
Diarization benchmark, Concatenated Minimum Permutation Word Error Rate (cpWER). 说话人日志基准测试,级联最小排列词错误率 (cpWER)。
With and without contextual prompting — higher is 有无上下文提示 — 更高是