How to Implement Structured Output with Local LLMs

如何使用本地大模型实现结构化输出

LLM Applications: How to Implement Structured Output with Local LLMs. Why use it? How to implement it? What can we do when it fails? 大模型应用：如何使用本地大模型实现结构化输出？为什么要使用它？如何实现？当它失败时我们该怎么办？

For building LLM applications, local LLMs are an attractive option. They allow us to keep our sensitive data and reduce our dependency on cloud APIs. However, running the model locally is only the first step. In a practical application, the local LLM is usually part of a larger workflow. This means its responses often need to be consumed by another component. 对于构建大模型应用而言，本地大模型是一个极具吸引力的选择。它们允许我们保留敏感数据并减少对云端 API 的依赖。然而，在本地运行模型仅仅是第一步。在实际应用中，本地大模型通常是更大工作流的一部分。这意味着它的响应通常需要被其他组件所调用。

In those situations, free-form text can be very difficult to work with. We want the output to follow some predictable structures. That’s exactly what Structured Output is for. We can achieve that by first defining the expected shape, or schema, in advance. The local serving runtime then constrains the LLM generation to follow that schema. Finally, the LLM would give us a regular Python object that our code can easily parse. 在这种情况下，自由格式的文本往往难以处理。我们希望输出能够遵循某种可预测的结构，而这正是“结构化输出”（Structured Output）的用武之地。我们可以通过预先定义预期的形状（即 Schema）来实现这一点。随后，本地服务运行时会约束大模型的生成过程，使其遵循该 Schema。最终，大模型将返回一个标准的 Python 对象，供我们的代码轻松解析。

In this post, we’ll illustrate this pattern through a concrete case study. We’ll use Gemma 4 as our local LLM, Ollama as the serving runtime, and Pydantic to define and validate the output schema. 在本文中，我们将通过一个具体的案例研究来阐述这一模式。我们将使用 Gemma 4 作为本地大模型，使用 Ollama 作为服务运行时，并使用 Pydantic 来定义和验证输出的 Schema。

1. How Do We Implement Structured Output with a Local LLM?

1. 我们如何使用本地大模型实现结构化输出？

1.1 A Smart-Home Case Study

1.1 智能家居案例研究

Suppose we are building a smart-home application. The user asks a simple question: “Should the dishwasher run now or later?” Before answering, the application needs to extract device information, timing constraints, and electricity tariffs from household notes. Since these notes contain private information, a local LLM is a natural fit as the first step. It can transform the original notes into a structured object that retains only the facts needed for scheduling while removing unnecessary personal details. We can then pass this sanitized object to a more capable cloud LLM for reasoning and scheduling. 假设我们正在构建一个智能家居应用。用户提出了一个简单的问题：“洗碗机现在运行还是稍后运行？”在回答之前，应用需要从家庭笔记中提取设备信息、时间限制和电价。由于这些笔记包含私人信息，本地大模型是执行第一步的理想选择。它可以将原始笔记转换为结构化对象，仅保留调度所需的事实，同时剔除不必要的个人细节。随后，我们可以将这个脱敏后的对象传递给能力更强的云端大模型进行推理和调度。

Here, let’s focus on the local transformation step. The following is the household context we’ll use: 在这里，我们重点关注本地转换步骤。以下是我们使用的家庭背景信息：

USER_QUESTION = "Should the dishwasher run now or later?"
SMART_HOME_CONTEXT = """
It is currently 18:30. The activity log records that the robot vacuum completed today's kitchen pass at 16:10 and returned to its dock. No more vacuuming is needed today. The dishwasher's earliest start is 18:30. A cycle takes 90 minutes and uses about 1.2 kWh. It must be complete before breakfast at 06:30. Because the dishwasher is beside the bedrooms, it must stop running by 22:30. The EV charger's earliest start is 18:30. Charging will take 120 minutes and use about 14 kWh. The car must be charged before its driver leaves at 07:00. The dryer's earliest start is 19:00. Its cycle takes 75 minutes and uses about 3.2 kWh. It contains the football kit, which must be dry by 23:00. The dryer is too loud later in the evening, so it must stop running by 21:30. The washing machine's earliest start is 20:00. Its cycle takes 60 minutes and uses about 0.9 kWh. It contains tomorrow's work clothes and must finish by 05:30. A kitchen pass with the robot vacuum takes 45 minutes and uses about 0.2 kWh. The vacuum's earliest start was 15:00. The home energy controller permits only one flexible load to run at a time. Electricity costs 0.45 per kWh from 17:00 to 20:00, 0.22 from 20:00 to 00:00, 0.12 from 00:00 to 06:00, and 0.25 from 06:00 to 17:00.
""".strip()

The goal of the local LLM is to retain the scheduling facts while leaving those personal details behind. 本地大模型的目标是保留调度事实，同时过滤掉那些个人细节。

1.2 Define the Expected Structure

1.2 定义预期结构

Next, we need to define what the sanitized object should look like. The downstream component needs the current time, the device mentioned in the question, the controller capacity, and the electricity prices. It also needs the devices that still require scheduling, together with their runtime and timing requirements. We can represent this using the following Pydantic models: 接下来，我们需要定义脱敏后的对象应该是什么样子。下游组件需要当前时间、问题中提到的设备、控制器容量以及电价信息。它还需要知道哪些设备仍需调度，以及它们的运行时间和时间要求。我们可以使用以下 Pydantic 模型来表示：

from typing import Annotated
from pydantic import BaseModel, Field

ClockTime = Annotated[
    str,
    Field(
        min_length=5,
        max_length=5,
        description="Clock time in HH:MM format.",
    ),
]

class DeviceToSchedule(BaseModel):
    device_name: str
    duration_minutes: int
    energy_kwh: float
    earliest_start: ClockTime
    finish_by: ClockTime | None

class SchedulingContext(BaseModel):
    current_time: ClockTime
    focus_device: str
    max_concurrent_devices: int
    current_price_per_kwh: float
    off_peak_start: ClockTime
    off_peak_end: ClockTime
    off_peak_price_per_kwh: float
    devices_to_schedule: list[DeviceToSchedule] = Field(
        description=(
            "Devices that have not completed their work "
            "and still need to be scheduled."
        )
    )

Note that we have a nested schema, but the structure is relatively easy to follow. SchedulingContext contains the shared household facts and a list of DeviceToSchedule objects. That’s the shape we want the local LLM to output. 请注意，我们使用了嵌套的 Schema，但结构相对容易理解。SchedulingContext 包含了共享的家庭事实以及一个 DeviceToSchedule 对象列表。这就是我们希望本地大模型输出的形状。

1.3 Setting Ollama and Local LLM

1.3 设置 Ollama 和本地大模型

Before moving forward, make sure that Ollama is installed and running locally. You can install Ollama on Windows: winget install Ollama.Ollama. On macOS or Linux, run: curl -fsSL https://ollama.com/install.sh | sh. 在继续之前，请确保 Ollama 已安装并在本地运行。在 Windows 上，你可以通过 winget install Ollama.Ollama 安装；在 macOS 或 Linux 上，请运行 curl -fsSL https://ollama.com/install.sh | sh。

Once Ollama is installed, we can pull the Gemma 4 model: ollama pull gemma4:e4b. We also need the Ollama Python client and Pydantic: pip install ollama pydantic. Here, we use the compact 4B variant of the Gemma 4 model for our current case study. 安装 Ollama 后，我们可以拉取 Gemma 4 模型：ollama pull gemma4:e4b。我们还需要 Ollama Python 客户端和 Pydantic：pip install ollama pydantic。在此案例研究中，我们使用了 Gemma 4 模型的紧凑型 4B 版本。

1.4 Connect Pydantic to Ollama

1.4 将 Pydantic 连接到 Ollama

Now, we connect the schema to our local model. Here is how we can achieve that: 现在，我们将 Schema 连接到本地模型。实现方式如下：

import ollama

def call_local_llm(schema, instructions, prompt):
    response = ollama.chat(
        model="gemma4:e4b",
        messages=[
            {"role": "system", "content": instructions},
            {"role": "user", "content": prompt},
        ],
        think="medium",
        format=schema.model_json_schema(),
    )
    return schema.model_validate_json(response.message.content)

Two important things worth mentioning here: model_json_schema() converts our Pydantic model into the schema, and then passed into Ollama via the format argument. model_validate_json() parses the response into the same Pydantic model. This allows easy consumption in the downstream steps. 这里有两点值得注意：model_json_schema() 将我们的 Pydantic 模型转换为 Schema，并通过 format 参数传递给 Ollama。model_validate_json() 则将响应解析为相同的 Pydantic 模型。这使得下游步骤可以轻松使用这些数据。