Every secret in my homelab has exactly one home: how I run a self-hosted secrets manager, and the traps that bit me

我的家庭实验室中，每个密钥都有且仅有一个归宿：我是如何运行自托管密钥管理器的，以及我踩过的坑

For a long time my homelab’s secrets lived the way most people’s do: a database password in one docker-compose.yml, an API key in another, a token pasted into a .env that got copied to three machines and diverged on all of them. Nothing was wrong, exactly. It was just that no single place could answer the question “where does this credential live, and what still uses it?” — and that question turns out to be the whole game. 长期以来，我家庭实验室（homelab）的密钥管理方式和大多数人一样：一个数据库密码写在某个 docker-compose.yml 里，一个 API 密钥写在另一个文件里，还有一个令牌（token）被粘贴进 .env 文件，然后被复制到三台机器上，最后导致各处的配置不一致。严格来说，这并没有什么错。只是没有任何一个地方能回答“这个凭据到底存在哪里，还有什么地方在使用它？”这个问题——而这恰恰是整个问题的核心。

So I moved everything behind a self-hosted secrets manager (Infisical, though the shape here applies to Vault, OpenBao, or any of them). One rule, ruthlessly applied: Every secret has exactly one home and one system of record. Nothing keeps its own copy on disk. This is what that actually looks like in practice, what it fixed, and the three traps that cost me real time — including the one where the tool that stores my secrets couldn’t hand one back because of a single $. 所以我把所有东西都迁移到了一个自托管的密钥管理器后面（我用的是 Infisical，不过这里的逻辑同样适用于 Vault、OpenBao 或任何同类工具）。我严格执行一条准则：每个密钥有且仅有一个归宿和唯一的记录系统。任何地方都不允许在磁盘上保留副本。以下是它在实践中的样子、它解决了什么问题，以及让我耗费大量时间的三个坑——其中一个坑是因为一个“$”符号，导致存储密钥的工具无法正常返回密钥。

The principle: one home, pulled at runtime. Before: a secret was wherever I’d last needed it — a compose file, a config, a shell history, a backup of all three. Rotating one meant hunting through the estate hoping I’d find every copy, and knowing I couldn’t be sure I had. After: a service holds no credentials on disk. It boots, authenticates to the secrets manager with a machine identity, pulls exactly the secrets it needs into memory, and runs. The values exist on disk in exactly one system, encrypted, and nowhere else. 原则：唯一的归宿，运行时拉取。以前：密钥散落在任何我需要它的地方——compose 文件、配置文件、shell 历史记录，甚至是这三者的备份中。轮换一个密钥意味着要在整个系统中搜寻，祈祷能找到所有副本，而且心里清楚自己根本无法确定是否找全了。现在：服务在磁盘上不保留任何凭据。它启动时，使用机器身份向密钥管理器进行身份验证，将所需的密钥直接拉取到内存中运行。这些值仅以加密形式存在于唯一的系统中，别无他处。

The test I use to know it’s working is brutal and simple: can I rotate any password in the estate by changing it in exactly one place? When I audited mine, the best thing I could say about the whole setup was yes — because not one docker-compose.yml held a real secret to go stale. That’s the entire payoff, and it’s worth the setup cost on its own. 我用来检验其有效性的测试简单而残酷：我能否通过修改唯一的一个地方，来轮换系统中的任何密码？当我审计自己的系统时，关于这套架构我能给出的最高评价就是“可以”——因为没有任何一个 docker-compose.yml 包含会过期的真实密钥。这就是全部的回报，仅此一点就足以抵消搭建成本。

Machine identities, not humans. The thing I’d most want a new self-hoster to internalise: the things pulling secrets are not you. Each service authenticates as itself — a machine identity scoped to only the folder of secrets it needs. The monitoring stack’s identity can read the monitoring secrets and nothing else. The AI agent’s identity reads its keys and nothing else. 机器身份，而非人类身份。我最希望新手自托管用户理解的一点是：拉取密钥的不是你。每个服务都以其自身的身份进行验证——这是一种仅限于其所需密钥文件夹的机器身份。监控栈的身份只能读取监控相关的密钥，除此之外什么都读不到。AI 代理的身份也只能读取它自己的密钥。

Two consequences fall straight out of that: Blast radius is scoped by default. A compromised service can read its own secrets, which it already had in memory anyway — not the whole vault. The identity is the boundary. Access is auditable and revocable per-service. Retiring a service means revoking one identity, not wondering which shared token it might have known. 由此直接产生两个结果：默认限制了爆炸半径。一个被攻破的服务只能读取它自己的密钥（这些密钥本来就在它的内存里），而无法读取整个保险库。身份就是边界。访问权限可以按服务进行审计和撤销。停用一个服务意味着撤销一个身份，而不用去纠结它可能知道哪些共享令牌。

And one rule I now treat as non-negotiable, because an autonomous agent runs against this vault: the automated identity gets read-only. The agent can read the secret it needs to do its job. It cannot write, rotate, or delete one. Writing secrets is a human action, gated behind a human identity. A clever process with write access to the place all your credentials live is a very bad day waiting for a bug. 我目前有一条不可妥协的规则，因为有自动化代理在访问这个保险库：自动化身份必须是只读的。代理可以读取它工作所需的密钥，但不能写入、轮换或删除密钥。写入密钥是人类的操作，必须通过人类身份进行授权。一个拥有写入权限且能访问所有凭据的智能进程，一旦出现 Bug，后果将不堪设想。

The resolver: degrade in a known direction. Services don’t call the vault’s API directly. They go through a tiny resolver with a fixed precedence: environment variable → secrets manager → built-in default. That order is deliberate, and each step earns its place: Environment first lets me override anything for a one-off — a debug run, a test container — without touching the vault. Secrets manager second is the real source of truth for everything normal. Default last means a non-secret config value still has a sane fallback if the manager is briefly unreachable — but a genuine secret’s “default” is deliberately a value that fails closed, not a working credential. 解析器：按预定方向降级。服务不会直接调用保险库的 API。它们通过一个微小的解析器，遵循固定的优先级：环境变量 → 密钥管理器 → 内置默认值。这个顺序是经过深思熟虑的，每一步都有其存在的意义：环境变量优先，让我在进行一次性操作（如调试运行、测试容器）时可以覆盖任何配置，而无需触碰保险库。密钥管理器排在第二，是所有正常情况下的真实来源。默认值排在最后，意味着如果密钥管理器短暂不可用，非敏感的配置值仍有合理的后备方案——但对于真正的密钥，其“默认值”被刻意设置为一种“关闭失败”（fail-closed）的值，而不是一个可用的凭据。

If the vault is down, the service fails to authenticate loudly; it does not silently fall back to some baked-in key. Degrading toward “broken and obvious” beats degrading toward “working with a secret I forgot was hardcoded.” The resolver is thirty lines. It’s the single most-used piece of code in the estate, precisely because it’s boring. 如果保险库宕机，服务会明确地报错，而不是静默地回退到某个硬编码的密钥。向“显而易见的故障”降级，远好于向“使用我早已忘记的硬编码密钥”降级。这个解析器只有三十行代码。它是整个系统中被使用频率最高的代码，恰恰因为它足够枯燥且稳定。

Trap 1: the bootstrap secret — who guards the guard? The obvious hole in “no secret on disk” is the machine-identity credential itself. That has to live somewhere a service can read at boot, before it can talk to the vault. You cannot store the key to the vault inside the vault. My answer isn’t clever, it’s just honest about the trade: the bootstrap credential lives in one tightly-permissioned environment file per host, owned by root, 600, and it is the only secret on that box. Everything else derives from it at runtime. 坑 1：引导密钥——谁来守护守护者？“磁盘上无密钥”这一理念中明显的漏洞是机器身份凭据本身。它必须存在于服务启动时可读取的地方，以便在连接保险库之前使用。你不能把保险库的钥匙存在保险库里。我的答案并不高明，只是诚实地面对这种权衡：引导凭据存放在每台主机的一个权限严格受限的环境文件中，由 root 用户所有，权限为 600，这是该机器上唯一的密钥。其他所有东西都在运行时从它派生出来。

I’ve accepted that there is exactly one secret I can’t centralise, so I make sure there’s only one, I know exactly where it is, and it’s the first thing I’d rotate if a host were ever touched. The lesson generalises: a secrets manager doesn’t eliminate the on-disk secret, it collapses N of them to one. Pretending it’s zero is how you end up not guarding the one that matters most. 我已经接受了有一个密钥无法被中心化的事实，所以我确保它只有一个，我清楚地知道它在哪里，并且如果主机受到威胁，它将是我第一个轮换的东西。这个教训可以推广：密钥管理器并不能消除磁盘上的密钥，它只是将 N 个密钥压缩成了一个。假装它是零，只会让你最终忽略掉那个最重要的密钥。

Trap 2: a single $ broke the whole stack. This one cost me an evening. A service read its secrets from an env_file in compose, and it restart-looped on boot with an authentication failure — using a password I could see, plainly correct, sitting right there in the file. The password contained a $. Docker Compose performs variable interpolation on values in an env_file, so pa$$word became pa followed by two empty variable expansions. 坑 2：一个“$”符号搞垮了整个栈。这花了我一个晚上。一个服务从 compose 的 env_file 中读取密钥，结果启动时陷入重启循环，报错身份验证失败——但我明明能看到密码，它在文件里写得清清楚楚，完全正确。密码中包含一个“$”符号。Docker Compose 会对 env_file 中的值进行变量插值，所以 pa$$word 变成了 pa 后面跟着两个空的变量扩展。

The service received a truncated password and, of course, the failure didn’t say “your password was mangled by interpolation” — it said “authentication failed,” which sent me hunting in entirely the wrong place. The fix is to escape the $ (double it: $$) or, better, avoid $ in generated secrets entirely. But the lesson is the one that keeps recurring across this whole series: a layer between you and your secret will transform it, and the error will blame the wrong component. When a credential you can literally see is “wrong,” suspect the pipe it travelled through before you suspect the value. 服务收到了被截断的密码，当然，错误信息并没有说“你的密码被插值搞乱了”，而是说“身份验证失败”，这让我完全找错了排查方向。解决方法是转义“$”（写成 $$），或者更好的是，在生成的密钥中完全避免使用“$”。但这个教训在整个系列中反复出现：在你和密钥之间增加一层，它就会对密钥进行转换，而错误信息往往会归咎于错误的组件。当你看到一个凭据明明是“错”的，在怀疑值本身之前，先怀疑它经过的传输管道。

Trap 3: the wrong master password, silently. I keep a tiered rotation worklist — the important credentials, staged to be… 坑 3：错误的管理员密码，静默失败。我维护着一份分级的轮换工作清单——那些重要的凭据，准备被……