What Actually Breaks During a "Zero-Downtime" Cloud Migration

What Actually Breaks During a “Zero-Downtime” Cloud Migration

“零停机”云迁移过程中,究竟是什么导致了故障?

Every cloud migration plan I’ve seen starts with the same promise: zero downtime. And most of them get pretty close — right up until the one dependency nobody mapped out decides to fall over at 2 a.m. Here are a few of the patterns that tend to cause the most pain, and what’s worked to avoid them.

我所见过的每一个云迁移计划,开篇承诺的都是“零停机”。大多数计划确实能做到接近这一目标——直到某个无人察觉的依赖项在凌晨两点突然崩溃。以下是几种最容易引发问题的模式,以及避免这些问题的有效方法。

1. DNS is never as fast as you think

1. DNS 的生效速度永远比你想象的要慢

TTL settings get set once, early in a project, and then forgotten. If your DNS TTL is sitting at 24 hours when you cut over, your “instant” switch is actually a multi-hour rolling failure for a chunk of your users. Drop the TTL days in advance, not the morning of.

TTL(生存时间)设置通常在项目初期设定一次后就被遗忘了。如果你在切换时 DNS TTL 仍设为 24 小时,那么你所谓的“即时”切换,对一部分用户而言,实际上是一场持续数小时的滚动故障。请提前几天降低 TTL,而不是等到切换当天。

2. Stateful services get left for last — and that’s backwards

2. 把有状态服务留到最后处理——这是本末倒置

Stateless app servers are easy to migrate: spin up new ones, point traffic at them, done. Databases, queues, and anything holding session state are the actual hard part, and they’re usually the thing teams plan for last because it’s the scariest piece. Flip that. Start migration planning with the stateful layer and work outward.

无状态应用服务器很容易迁移:启动新的实例,将流量指向它们,搞定。数据库、队列以及任何保存会话状态的服务才是真正的难点,而团队通常因为畏难而将其留到最后规划。请扭转这种做法:从有状态层开始规划迁移,然后再向外扩展。

3. CI/CD pipelines assume an environment that no longer exists

3. CI/CD 流水线依赖于不再存在的环境

If your pipeline has hardcoded IPs, region-specific credentials, or scripts that assume a particular host’s filesystem layout, migrating infrastructure without touching the pipeline just moves the failure point. A pipeline that’s portable across environments (parameterized configs, environment variables instead of hardcoded values, containerized build steps) survives a migration. One that isn’t will quietly break the first deploy after cutover.

如果你的流水线包含硬编码的 IP、特定区域的凭据,或者假设了特定主机文件系统布局的脚本,那么在不修改流水线的情况下迁移基础设施,只会转移故障点。一个具备跨环境可移植性的流水线(使用参数化配置、环境变量而非硬编码值、容器化构建步骤)才能在迁移中存活。否则,切换后的第一次部署就会悄无声息地失败。

4. Monitoring gaps show up exactly when you need them least

4. 监控漏洞总是在你最需要的时候出现

It’s common to migrate the app and the infrastructure, and forget that your monitoring stack was scraping metrics from the old environment’s internal network. You don’t find out until the first incident happens and the dashboards are just… empty. Stand up monitoring in the new environment before cutover, and run both in parallel for at least a few days.

人们常在迁移应用和基础设施时,忘记监控栈仍在从旧环境的内部网络抓取指标。直到第一次事故发生,你才会发现仪表盘竟然是一片空白。请在切换前建立新环境的监控,并让新旧监控并行运行至少几天。

5. Rollback plans that were never tested aren’t rollback plans

5. 未经测试的回滚计划等于没有计划

A rollback plan that lives in a doc and has never been executed is a hypothesis, not a plan. If you can’t dry-run the rollback in a staging environment, budget time to build that capability before migration day, not after something’s already gone wrong.

只存在于文档中且从未执行过的回滚计划只是一个假设,而非真正的计划。如果你无法在预发布环境中进行回滚演练,请在迁移日之前预留时间来构建这项能力,而不是等到出事之后才去补救。

It comes down to sequencing, not tooling

关键在于顺序,而非工具

None of this is exotic — it’s mostly discipline and sequencing. The teams that get through migrations cleanly aren’t the ones with the fanciest tooling, they’re the ones who mapped their dependencies honestly before they started moving things. If you’re in the middle of planning a migration or a CI/CD overhaul and want a second set of eyes, Web Matrix Lab’s Cloud & DevOps team has written up more detail on how they approach this.

这些都不是什么高深的技术——主要靠的是纪律和顺序。能够顺利完成迁移的团队,并不是拥有最先进工具的团队,而是那些在开始迁移前就诚实地梳理好所有依赖关系的团队。如果你正在规划迁移或 CI/CD 重构并需要第二双眼睛审视,Web Matrix Lab 的云与 DevOps 团队已经详细记录了他们的处理方法。