How Much Should Live Together? Learning to Isolate Services the Hard Way

服务应该如何共存？在“硬核”实践中学习服务隔离

Most of us who get into self-hosting start the same way: start with linux, throw a few apps into Docker, get them running and connectable outside the home network, and call it good for months, maybe even years. Nothing wrong with that approach. A compose file and a spare mini PC gets you further than you think, and if it works and you don’t have to think about it, that’s a perfectly fine place to stop. 大多数涉足自托管（self-hosting）的人起步方式都大同小异：安装 Linux，把几个应用扔进 Docker，让它们运行起来并能从家庭网络外访问，然后就此打住，一用就是几个月甚至几年。这种方法没什么不好。一个 compose 文件加上一台闲置的迷你 PC 能带你走得比想象中更远；如果它运行良好且无需你操心，那这确实是一个完美的终点。

Then there’s the rest of us. The people who get that first setup running, feel the little spark of “wait, I built this”, and immediately start wondering what else is possible. More services. Less babysitting. A real answer to “what happens if this box dies at 2am”. If any of that sounds familiar, this one’s for you. 但还有我们这群人。当我们完成了最初的设置，感受到那种“等等，这是我亲手搭建的”小火花时，便会立刻开始思考还有什么可能性。更多的服务，更少的维护，以及对“如果这台机器在凌晨两点挂了该怎么办”这一问题的真正解答。如果这些听起来很耳熟，那么这篇文章就是为你准备的。

If you keep going, you’ll eventually run into the question every self-hosted setup faces sooner or later, whether you notice it happening or not, “how much should live together, and how much should be kept apart?”. Put everything on one box and you quickly feel the fragility when one bad update takes everything down with it. Or when nightly backups put services on hold longer and longer. Split everything into its own isolated piece and you’ve gained resiliency but now manage a lot of moving parts. Most of the actual learning in running infrastructure happens in the space between those two answers. Where you draw that line is where most of the real infrastructure lessons live. 如果你继续深入，迟早会遇到每个自托管方案都会面临的问题（无论你是否察觉）：到底多少服务应该放在一起，多少应该分开？把所有东西塞进一台机器，你会很快感受到它的脆弱——一次糟糕的更新可能导致全盘崩溃，或者每晚的备份让服务停机的时间越来越长。如果把所有东西都拆分成独立的隔离单元，你确实获得了弹性，但同时也需要管理大量复杂的组件。运行基础设施时，大部分真正的经验都产生于这两个极端答案之间。你划定的那条界线，正是基础设施核心课程所在之处。

Over the years, I’ve lived through a few different answers to that question in my own homelab, and each one taught me something the previous one couldn’t. It started with a large VM, Docker installed, and every service I wanted to self-host running as a container inside. It was the fastest path to “it’s actually working”, and at the time that was the whole goal. I didn’t know yet what I’d eventually want out of this thing, so keeping the infrastructure simple while I figured that out made sense. That setup carried me a long way, and I don’t view it as a mistake. It’s just a different set of tradeoffs than the ones I care about now. 多年来，我在自己的家庭实验室（homelab）中尝试过几种不同的答案，每一种都教会了我前一种无法提供的经验。起初，我使用一个大型虚拟机（VM），安装 Docker，并将所有想托管的服务作为容器运行在里面。这是实现“它真的能跑了”最快的路径，而当时这就是我的全部目标。我那时还不知道自己最终想要什么，所以保持基础设施简单以便摸索是合理的。那个方案支撑了我很久，我不认为那是个错误，它只是与我现在所关注的权衡点不同而已。

Everything sharing one VM means everything shares fate. Patch the kernel and every service reboots together, one misbehaving container eating memory affects everything else in it, and moving a VM to another node means moving everything at once whether you needed to or not. 所有东西共享一个虚拟机意味着“同生共死”。修补内核会导致所有服务一起重启；一个行为异常的容器耗尽内存会影响其中的所有其他服务；将虚拟机迁移到另一个节点意味着必须一次性迁移所有东西，无论你是否需要。

Somewhere along the way, I split things out into a handful of purpose-built VMs instead of one giant one. Plex and the Arr stack in one, one-off applications in another, and a test bench VM for vetting new (potentially insecure) services. That was a big improvement. Smaller blast radius per VM, and coupled with Proxmox HA, I could distribute load and failover at the VM level instead of betting everything on a single box staying up. It still wasn’t the end of the road, and the thing that finally pushed me past it (among other notable tradeoffs) was trying to do something a VM limitation just couldn’t do gracefully, share a GPU. 在某个阶段，我将服务拆分成了几个专用虚拟机，而不是一个巨大的虚拟机。Plex 和 Arr 套件放在一个，一次性应用放在另一个，还有一个用于测试新（可能不安全）服务的测试虚拟机。这是一个巨大的进步。每个虚拟机的“爆炸半径”变小了，配合 Proxmox HA，我可以在虚拟机层面分配负载和故障转移，而不必把所有赌注都压在一台机器上。但这还不是终点，最终促使我进一步突破（除了其他显著的权衡因素外）的原因是，我尝试做一件虚拟机限制无法优雅处理的事情：共享 GPU。

Zoomed out, that’s three different answers to the same question, each with its own breaking point: 从宏观上看，这是对同一个问题的三个不同答案，每一个都有其局限性：

• One big VM — every service as a Docker container, the fastest path to “it’s actually working”
• 一个大型虚拟机——所有服务作为 Docker 容器，实现“它真的能跑了”的最快路径
• A handful of purpose-built VMs — smaller blast radius per VM, with Proxmox HA distributing load and failover
• 几个专用虚拟机——每个虚拟机的爆炸半径更小，通过 Proxmox HA 分配负载和故障转移
• One LXC container per service — real cross-node failover, isolation, and GPU sharing
• 每个服务一个 LXC 容器——真正的跨节点故障转移、隔离和 GPU 共享

Where Categorized VMs Still Ran Into Walls

分类虚拟机依然会碰壁的地方

I had a GPU passed through to the VM running Plex, the classic setup: bind the card to the guest at the PCI level, install drivers inside the VM, done. That grossly oversimplifies the process as anyone who’s attempted this knows, but it gets the point across. It works, but it’s exclusive. The card belongs to the VM and nothing else on the host can touch it, including other services inside the VM. PCI passthrough also doesn’t play well with moving a VM, live migrations with a passed-through device is flatly unsupported, and even offline migration (stopping the VM, moving it, starting it on a different node/server) needs specific per-node resource mapping configured ahead of time. This is because PCI devices have internal registers and memory which need to be migrated exactly as is. 我曾将一个 GPU 直通给运行 Plex 的虚拟机，这是经典配置：在 PCI 层级将显卡绑定到客户机，在虚拟机内安装驱动，搞定。任何尝试过的人都知道，这大大简化了过程，但道理是通的。它能用，但它是排他的。显卡属于该虚拟机，宿主机上的其他任何东西都无法触碰它，包括虚拟机内的其他服务。PCI 直通在迁移虚拟机时表现也不佳，带直通设备的实时迁移完全不支持，甚至离线迁移（停止虚拟机、移动它、在不同节点/服务器上启动）也需要提前配置特定的节点资源映射。这是因为 PCI 设备有内部寄存器和内存，必须原封不动地进行迁移。

That wall and the pursuit of true HA pushed me to the stage I’m on now, one isolated LXC container per service instead of a handful of VMs each running several. I won’t pretend this is the objectively correct way to run a homelab, or that everyone should be doing this, it’s just where my own tradeoffs landed once GPU sharing and real cross-node failover started mattering to me more than simplicity did. Moving to extremely lightweight LXC containers isolated per service and offboarding application and metadata to NVMe storage has been a game changer in my homelab progression. 那堵墙以及对真正高可用性（HA）的追求，将我推向了现在的阶段：每个服务一个独立的 LXC 容器，而不是几个运行多个服务的虚拟机。我不会假装这是运行家庭实验室的唯一正确方式，也不是每个人都应该这样做，这只是当 GPU 共享和真正的跨节点故障转移对我来说比简单性更重要时，我所做出的权衡。转向极其轻量级的、按服务隔离的 LXC 容器，并将应用和元数据卸载到 NVMe 存储上，彻底改变了我的家庭实验室进程。

Small Containers, Not Small Data

小容器，大数据

The shift I’ve been making over the last several months is moving almost all services out of these categorized VMs and into individual LXC containers: one service, one container, each with enough resources to do its job. LXCs share the host kernel instead of virtualizing their own, which means they’re lighter and faster to start, stop, and move than a VM. More importantly for GPU work, a GPU can be shared across multiple containers on the same host, instead of being locked to a single guest. 过去几个月我一直在做的转变是，将几乎所有服务从这些分类虚拟机中移出，放入独立的 LXC 容器中：一个服务，一个容器，每个容器都有足够的资源来完成其工作。LXC 共享宿主机内核，而不是虚拟化自己的内核，这意味着它们比虚拟机更轻量，启动、停止和移动速度更快。对于 GPU 工作负载而言，更重要的是，GPU 可以在同一宿主机上的多个容器之间共享，而不是被锁定在单个客户机中。

Now is great time to bring up Proxmox Helper Scripts and how much time and pain they can save you. Its hundreds of community-built and maintained scripts make managing and deploying services in your homelab a breeze. Applications pre-configured, sensible defaults applied, and in the case of Plex, GPU detection and passthrough handled automatically. It’s a big part of why “one LXC container per service” is realistic to live with instead of just a nice idea on paper. The instinct when you hear “isolated container” is to assume everything about that service, including its data, lives inside it. I’ve landed on the opposite approach for anything with real st 现在是提到 Proxmox Helper Scripts 的好时机，它们能为你节省大量时间和精力。其数百个由社区构建和维护的脚本，让管理和部署家庭实验室服务变得轻而易举。应用预先配置好，应用了合理的默认设置，以 Plex 为例，GPU 检测和直通都是自动处理的。这就是为什么“每个服务一个 LXC 容器”不仅是纸上谈兵，而是切实可行的重要原因。当你听到“隔离容器”时，本能会认为该服务的所有内容（包括数据）都驻留在容器内部。但对于任何有实际存储需求的服务，我采取了相反的方法。