MLOps for Developers: Deploying, Monitoring, and Optimizing Machine Learning Models

面向开发者的 MLOps：机器学习模型的部署、监控与优化

How to deploy, monitor, and optimize ML models in production: GPU selection, VRAM requirements, cloud vs local cost, model drift, and CI/CD pipelines for ML. 如何部署、监控并优化生产环境中的机器学习模型：涵盖 GPU 选择、显存需求、云端与本地成本对比、模型漂移以及机器学习的 CI/CD 流水线。

The Model That Worked in Jupyter and Failed in Production

在 Jupyter 中运行良好却在生产环境中失效的模型

In 2020, a team at a financial services company trained a fraud detection model that achieved 99.2% accuracy in their Jupyter notebooks. They deployed it to production. Within a week, false positives had increased by 300%, and the company was blocking legitimate transactions from thousands of customers. The model wasn’t broken — the data had changed. The pandemic had shifted spending patterns so dramatically that the model’s training data no longer resembled reality. 2020 年，一家金融服务公司的团队训练了一个欺诈检测模型，在 Jupyter Notebook 中达到了 99.2% 的准确率。他们将其部署到生产环境。一周内，误报率激增了 300%，导致公司拦截了数千名客户的合法交易。模型本身没有损坏，而是数据变了。疫情极大地改变了消费模式，使得模型的训练数据不再符合现实情况。

This is the central lesson of MLOps: a model is not a feature you ship once. It’s a system that degrades over time. Unlike traditional software, where a bug either exists or doesn’t, ML models silently degrade as the world changes around them. The data drifts. The distribution shifts. The model that was accurate last month becomes inaccurate this month — and without monitoring, you won’t know until customers complain. 这就是 MLOps 的核心教训：模型不是发布一次就万事大吉的功能，而是一个会随时间推移而退化的系统。与传统软件（Bug 要么存在要么不存在）不同，机器学习模型会随着周围环境的变化而悄无声息地退化。数据会漂移，分布会偏移。上个月准确的模型这个月可能就不准了——如果没有监控，直到客户投诉你才会发现。

MLOps (Machine Learning Operations) is the set of practices that keeps ML models working in production: deploying them reliably, monitoring them continuously, retraining them when they degrade, and doing all of this at scale. This guide covers the fundamentals: choosing the right hardware for inference, optimizing costs (cloud API vs self-hosting), detecting model and data drift, and building CI/CD pipelines that work for ML systems. MLOps（机器学习运维）是一套确保机器学习模型在生产环境中持续运行的实践方法：可靠地部署、持续地监控、在模型退化时进行重训练，并实现这一切的规模化。本指南涵盖了基础知识：为推理选择合适的硬件、优化成本（云端 API 与自托管对比）、检测模型和数据漂移，以及构建适用于机器学习系统的 CI/CD 流水线。

The ML Lifecycle: From Experiment to Production

机器学习生命周期：从实验到生产

The path from a Jupyter notebook to a production ML system has more stages than most developers expect: 从 Jupyter Notebook 到生产级机器学习系统，其路径包含的阶段比大多数开发者预期的要多：

Data collection & preparation — gathering, cleaning, labeling, and splitting data
Experimentation — training models, tuning hyperparameters, evaluating on validation sets
Model validation — testing on holdout sets, checking for bias, verifying performance metrics
Deployment — packaging the model, setting up inference infrastructure, exposing an API
Monitoring — tracking prediction latency, error rates, data drift, and model drift
Retraining — collecting new data, retraining the model, validating, redeploying
数据收集与准备 — 收集、清洗、标注并拆分数据
实验 — 训练模型、调整超参数、在验证集上进行评估
模型验证 — 在留出集（holdout sets）上测试、检查偏差、验证性能指标
部署 — 打包模型、搭建推理基础设施、暴露 API 接口
监控 — 跟踪预测延迟、错误率、数据漂移和模型漂移
重训练 — 收集新数据、重新训练模型、验证并重新部署

Traditional software has steps 1-4 and mostly skips 5-6. ML systems need all six — and steps 5-6 are where most teams fail. A model deployed without monitoring is a model that will fail silently. The key insight: the experiment (steps 1-3) is often less than 20% of the total effort. The production infrastructure (steps 4-6) is where the real engineering happens. 传统软件通常包含 1-4 步，而大多跳过 5-6 步。机器学习系统则需要全部六个步骤——而第 5-6 步正是大多数团队失败的地方。没有监控的部署模型，注定会悄无声息地失效。核心洞察在于：实验阶段（1-3 步）通常只占总工作量的不到 20%，而生产基础设施（4-6 步）才是真正体现工程能力的地方。

Infrastructure: GPU Selection and VRAM Requirements

基础设施：GPU 选择与显存需求

Deploying an ML model requires hardware — and for most modern models, that means GPUs. The central question: how much VRAM do I need? VRAM (Video RAM) is the memory on a GPU where the model weights and activations live during inference. A 7B-parameter model in 16-bit precision requires roughly 14 GB just for the weights (7 billion parameters × 2 bytes each). Add activations, KV cache, and overhead, and a 7B model needs roughly 16-20 GB of VRAM for comfortable inference. A 70B model needs roughly 140-160 GB — which means multiple high-end GPUs. 部署机器学习模型需要硬件支持，对于大多数现代模型而言，这意味着 GPU。核心问题是：我需要多少显存（VRAM）？显存是 GPU 上用于存放推理过程中模型权重和激活值的内存。一个 16 位精度的 7B 参数模型，仅权重就需要约 14 GB（70 亿参数 × 每个 2 字节）。加上激活值、KV 缓存和系统开销，一个 7B 模型通常需要 16-20 GB 显存才能流畅推理。而 70B 模型则需要约 140-160 GB，这意味着需要多张高端 GPU。

(Table omitted for brevity, but the concept remains: Quantization reduces precision to fit models into less VRAM. 8-bit halves it, 4-bit quarters it.) （表格略，但核心概念如下：量化通过降低精度来减少显存占用。8 位量化可减半，4 位量化可减少至四分之一。）

Quantization reduces the precision of model weights to fit more into less VRAM. 8-bit quantization halves the memory requirement (7B model → ~7 GB). 4-bit quantization quarters it (7B model → ~3.5 GB). The cost: a small accuracy degradation that’s often imperceptible for inference tasks. 量化通过降低模型权重的精度，使模型能塞进更小的显存中。8 位量化将内存需求减半（7B 模型 → 约 7 GB），4 位量化则减至四分之一（7B 模型 → 约 3.5 GB）。代价是：会有微小的准确率下降，但在推理任务中通常难以察觉。

Cost Optimization: Cloud API vs Self-Hosting

成本优化：云端 API 与自托管

The most consequential infrastructure decision for ML deployment is: should I call a cloud API or run the model myself? 机器学习部署中最关键的基础设施决策是：我应该调用云端 API，还是自己运行模型？

Cloud APIs (OpenAI, Anthropic, Google) charge per token. The benefit: zero infrastructure management, automatic scaling, and access to the latest models. The cost scales linearly with usage — there’s no way to amortize hardware. 云端 API（如 OpenAI、Anthropic、Google）按 Token 收费。优点是：无需管理基础设施、自动扩缩容，且能使用最新模型。成本随使用量线性增长，无法通过摊销硬件成本来降低费用。

Self-hosting requires buying or renting GPUs but eliminates per-token costs. A $10,000 A100 amortized over 3 years costs roughly $275/month. Electricity and hosting add $100-200/month. Total: ~$400-500/month for unlimited inference. The break-even point depends on your token volume and the specific model — roughly 1-3 million tokens per day for current frontier models. 自托管需要购买或租赁 GPU，但消除了按 Token 收费的成本。一张 10,000 美元的 A100 显卡分摊到 3 年，每月成本约 275 美元。加上电费和托管费，每月约 100-200 美元。总计每月约 400-500 美元即可实现无限量推理。盈亏平衡点取决于你的 Token 使用量和具体模型——对于当前的前沿模型，大约在每天 100 万到 300 万 Token 之间。

The hybrid approach (common in production): use cloud APIs for development, testing, and low-volume tasks; self-host for high-volume, latency-sensitive, or privacy-critical workloads. This gives you the flexibility of cloud APIs with the cost efficiency of self-hosting where it matters. 混合方案（生产环境中很常见）：开发、测试和低流量任务使用云端 API；高流量、对延迟敏感或对隐私要求极高的工作负载则采用自托管。这既保留了云端 API 的灵活性，又在关键场景下实现了自托管的成本效益。