Build a Voice Assistant with Python and Whisper

查看原文 / View Original 编辑 / Edit

Build a Voice Assistant with Python and Whisper

使用 Python 和 Whisper 构建语音助手

Imagine hearing your computer respond to your voice with the same nuance and clarity as a human, all running locally on your own machine without sending a single byte of data to the cloud. That’s the power of building a Voice Assistant with Python and OpenAI’s Whisper model. 想象一下，你的电脑能像人类一样，以同样的细微差别和清晰度回应你的声音，而且所有处理都在你自己的机器上本地运行，无需向云端发送任何数据。这就是使用 Python 和 OpenAI 的 Whisper 模型构建语音助手的强大之处。

While cloud-based assistants like Siri or Alexa are convenient, they come with privacy trade-offs and latency issues. By combining Whisper’s state-of-the-art speech-to-text capabilities with Python’s flexibility, you can create a private, fast, and customizable voice interface that answers your questions, controls your scripts, or even just chats with you—right from your terminal. Let’s get your hands dirty and build one from scratch today. 虽然 Siri 或 Alexa 等云端助手很方便，但它们伴随着隐私权衡和延迟问题。通过将 Whisper 最先进的语音转文字功能与 Python 的灵活性相结合，你可以创建一个私密、快速且可定制的语音界面，直接在终端中回答你的问题、控制脚本，甚至只是与你聊天。今天，让我们动手从零开始构建一个吧。

Why Whisper and Python?

为什么选择 Whisper 和 Python？

OpenAI’s Whisper is a transformer-based model trained on 680,000 hours of multilingual and multitask supervised data. Unlike older speech recognition tools that struggled with background noise or specific accents, Whisper delivers high accuracy across diverse environments. It’s open-source, meaning you can run it locally, ensuring your conversations stay private. OpenAI 的 Whisper 是一个基于 Transformer 的模型，在 68 万小时的多语言和多任务监督数据上进行了训练。与以往在背景噪音或特定口音下表现不佳的语音识别工具不同，Whisper 在各种环境下都能提供高精度识别。它是开源的，这意味着你可以在本地运行它，确保你的对话保持私密。

Python is the natural partner for this task. With libraries like SpeechRecognition, pyttsx3, and openai-whisper (or its faster variant faster-whisper), you can stitch together the entire pipeline: capturing audio, transcribing it, processing the text with an LLM, and speaking the response back. Python 是这项任务的最佳搭档。借助 SpeechRecognition、pyttsx3 和 openai-whisper（或其更快的变体 faster-whisper）等库，你可以将整个流程串联起来：捕获音频、转录音频、使用大语言模型（LLM）处理文本，并将回复语音播报出来。

Setting Up Your Environment

环境配置

Before writing code, you need the right tools. Start by creating a dedicated project directory and a virtual environment to keep dependencies clean. 在编写代码之前，你需要准备好工具。首先创建一个专门的项目目录和一个虚拟环境，以保持依赖项的整洁。

mkdir voice-assistant && cd voice-assistant
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate

Next, install the core speech processing stack. While openai-whisper works well, faster-whisper is often preferred for production due to its optimized performance and lower memory usage. 接下来，安装核心语音处理栈。虽然 openai-whisper 表现良好，但由于其优化的性能和更低的内存占用，faster-whisper 在生产环境中通常是首选。

pip install SpeechRecognition==3.10.4 \
  faster-whisper \
  pyttsx3==2.98 \
  python-dotenv==1.0.1

You’ll also need an API key if you plan to send the transcribed text to an LLM like OpenAI’s GPT or Anthropic’s Claude. Store this securely in a .env file to avoid hardcoding secrets: 如果你计划将转录后的文本发送给 OpenAI 的 GPT 或 Anthropic 的 Claude 等大语言模型，你还需要一个 API 密钥。请将其安全地存储在 .env 文件中，以避免硬编码密钥：

# .env
ANTHROPIC_API_KEY=sk-ant-api03-your-key-here
OPENAI_API_KEY=your-openai-key-here

The Core Logic: Speech-to-Text and Text-to-Speech

核心逻辑：语音转文字与文字转语音

The heart of your assistant is the loop: Listen → Transcribe → Process → Speak. Let’s break down the first two steps. 助手的核心在于这个循环：监听 → 转录 → 处理 → 说话。让我们拆解前两个步骤。

Capturing and Transcribing Audio

捕获并转录音频

We’ll use SpeechRecognition to capture microphone input and faster-whisper to transcribe it. The recognizer object handles the audio stream, while the whisper model converts audio waves into text. 我们将使用 SpeechRecognition 来捕获麦克风输入，并使用 faster-whisper 进行转录。识别器对象处理音频流，而 Whisper 模型将音频波形转换为文本。

import speech_recognition as sr
from faster_whisper import Whisper
import os
from dotenv import load_dotenv

# Load API keys
load_dotenv()

# Initialize Whisper model (small.en is fast and accurate for English)
model = Whisper("small.en")

# Initialize Speech Recognition
recognizer = sr.Recognizer()

def listen_for_prompt():
    with sr.Microphone() as source:
        print("Listening... (Say something)")
        recognizer.adjust_for_ambient_noise(source, duration=0.5)
        audio = recognizer.listen(source, timeout=5)

    # Convert audio to WAV file for Whisper
    audio_path = "prompt.wav"
    with open(audio_path, "wb") as f:
        f.write(audio.get_wav_data())

    # Transcribe with Whisper
    segments, _ = model.transcribe(audio_path)
    text = "".join([segment.text for segment in segments])
    print(f"You said: {text}")
    return text

This function waits for you to speak, saves the audio to a temporary file, and runs it through Whisper. The small.en model is a great starting point—it’s lightweight (about 50MB) and transcribes 5 seconds of audio in under 0.5 seconds on modern hardware. 此函数会等待你说话，将音频保存到临时文件，并通过 Whisper 进行处理。small.en 模型是一个很好的起点——它很轻量（约 50MB），在现代硬件上可以在 0.5 秒内转录 5 秒的音频。

Speaking the Response Back

语音播报回复

Once you have text, you need to turn it into speech. pyttsx3 is a simple, offline text-to-speech engine that works without external APIs. 一旦有了文本，你需要将其转换为语音。pyttsx3 是一个简单的离线文字转语音引擎，无需外部 API 即可工作。

import pyttsx3

engine = pyttsx3.init()

def speak(text):
    print(f"Assistant: {text}")
    engine.say(text)
    engine.runAndWait()

Putting It All Together: The Assistant Loop

整合：助手循环

Now, let’s combine transcription and speech into a working loop. For this example, we’ll use a simple hardcoded response. In a real project, you’d send the transcribed text to an LLM API for dynamic answers. 现在，让我们将转录和语音功能结合到一个工作循环中。在这个示例中，我们将使用简单的硬编码回复。在实际项目中，你会将转录后的文本发送给 LLM API 以获取动态答案。

def main():
    speak("Hello! I'm your local voice assistant. What can I do for you?")
    while True:
        user_input = listen_for_prompt()
        if not user_input:
            continue
        
        # Simple logic: if user says "stop", exit
        if "stop" in user_input.lower():
            speak("Goodbye!")
            break
            
        # Placeholder for LLM integration
        response = f"I heard you say: '{user_input}'. How can I help with that?"
        speak(response)

if __name__ == "__main__":
    main()

Run this script, and you’ll have a working voice assistant that listens, transcribes, and responds. The beauty here is that everything runs locally—no cloud dependencies for the speech parts, and you can swap the LLM logic to use any provider you prefer. 运行此脚本，你将拥有一个可以监听、转录并回复的语音助手。这里的妙处在于一切都在本地运行——语音部分没有云端依赖，而且你可以替换 LLM 逻辑以使用你喜欢的任何提供商。

Enhancing Your Assistant

增强你的助手

Once the basics are working, you can level up with these practical additions: 一旦基础功能正常运行，你可以通过以下实用功能进行升级：

  • Wake Word Detection: Instead of listening constantly, detect a specific phrase like “Hey Assistant” to activate the loop. This saves CPU and reduces false triggers. 唤醒词检测： 不要一直监听，而是检测像“Hey Assistant”这样的特定短语来激活循环。这可以节省 CPU 并减少误触发。
  • Context Management: Pass previous conversations to your LLM so the assistant remembers context. This creates a more natural, conversational experience. 上下文管理： 将之前的对话传递给你的 LLM，以便助手记住上下文。这能创造更自然、更具对话感的体验。
  • Custom Commands: Map specific phrases to system actions (e.g., “Open browser” → subprocess.run(["google-chrome"])). 自定义命令： 将特定短语映射到系统操作（例如，“Open browser” → subprocess.run(["google-chrome"])）。
  • Faster Models: If you need speed, try tiny.en or base.en models. They’re less accurate but faster for real-time applications. 更快的模型： 如果你需要速度，尝试 tiny.en 或 base.en 模型。它们的精度较低，但对于实时应用来说速度更快。

Privacy and Performance Considerations

隐私与性能考量

Running Whisper locally means your audio never leaves your machine, a critical feature for privacy-conscious users. However, be aware that larger models (like large-v3) require significant RAM and GPU resources. For most desktop use cases, small.en or base.en offers the best balance of speed and accuracy. 在本地运行 Whisper 意味着你的音频永远不会离开你的机器，这对注重隐私的用户来说是一个关键特性。但请注意，较大的模型（如 large-v3）需要大量的内存和 GPU 资源。对于大多数桌面使用场景，small.en 或 base.en 提供了速度与精度的最佳平衡。

← 返回新闻列表 / Back to news list