Shrinking NixOS Ram footprint and resource requirements to the bare minimum

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Shrinking NixOS Ram footprint and resource requirements to the bare minimum. AnthonySmith AdministratorProviderOG October 30 edited October 30 in Blog 14 min readOriginally written by AnthonySmith · Read the original forum post A few people were surprised by how low you can go with NixOS, so i thought I would do a write up as I did not realise this was anything special, I kept notes when doing it but not for the intention of writing a blog article so i might miss small tweaks I did here and there.

将 NixOS 的内存占用和资源需求缩减至最低限度。AnthonySmith AdministratorProviderOG 10 月 30 日编辑于 10 月 30 日，博客，阅读时长 14 分钟。原文由 AnthonySmith 撰写 · 阅读原始论坛帖子。一些人对 NixOS 能达到的低资源占用感到惊讶，所以我决定写一篇总结，因为我之前没意识到这有什么特别之处。我在操作时记了笔记，但并非为了写博客文章，所以可能会遗漏一些零星的小调整。

I initially set aside half a day for this; it actually took closer to 3 days. This guide is not very focused on saving disk space. If you want a disk space saving guide that has already been done by someone else here: https://nixcademy.com/posts/minimizing-nixos-images/ this has its own set of challenges and drawbacks so if you want to layer them together, be my guest I have decided not to implement the disk-minimising options into the TierHive NixOS template as I feel it is too likely to cause a default that end users are unlikely to expect.

我最初为此预留了半天时间；实际上花费了近 3 天。本指南并不侧重于节省磁盘空间。如果你需要节省磁盘空间的指南，其他人已经写过了：https://nixcademy.com/posts/minimizing-nixos-images/。这有其自身的一系列挑战和缺点，所以如果你想将它们结合起来使用，请自便。我决定不在 TierHive NixOS 模板中实现磁盘精简选项，因为我觉得这很可能会导致最终用户意想不到的默认设置。

Here we go: NixOS 26.05 idling at around 96MB and rebuilding in 512MB Stock deployment before any changes and with cloud-init, it runs on 1024MB RAM, JUST!, it sat at 288MB used with 89 kernel modules loaded, and a plain same-release rebuild needed 896MB (at 768 nix-build got OOM-killed at 505MB anon). It now idles at 96MB used on a 512MB VPS with 54 modules, and a full channel update plus rebuild, including a kernel switch to 7.3 if you want to do that, It finishes at 512MB with a 371MB peak and no OOM.

开始吧：NixOS 26.05 在 512MB 内存下空闲占用约 96MB，并可进行重建。在未做任何更改且使用 cloud-init 的标准部署中，它在 1024MB 内存下勉强运行；空闲占用为 288MB，加载了 89 个内核模块，进行同版本重建需要 896MB（在 768MB 时 nix-build 因内存不足被杀，匿名内存占用 505MB）。现在它在 512MB 的 VPS 上空闲占用仅 96MB，加载 54 个模块，并且可以完成完整的频道更新和重建（如果需要，甚至包括切换到 7.3 内核），最终在 512MB 内存内完成，峰值占用 371MB，且不会出现内存溢出（OOM）。

All numbers are from my own runs sometimes I recorded exact numbers others I left out the KB’s and only recorded the MB’s , measured a few minutes after the second boot with nobody logged in using my own build environment testing tools (I never log in to templates before use so they are clean) Roughly in the order and things done on this template (even the harmless fails): sysctl.vm.mem_profiling=never on the cmdline. The NixOS kernel ships with allocation profiling enabled by default and the tracking tables cost about 6MB (3.9MB vmalloc, 1.9MB slab). the bare mem_profiling=never which is left in isn’t a valid parameter; the kernel juts hands it to userspace and nothing changes, it needs the sysctl. prefix.

所有数据均来自我个人的运行记录，有时我记录了精确数字，有时省略了 KB 只记录了 MB。测量是在第二次启动几分钟后进行的，当时无人登录，使用的是我自己的构建环境测试工具（我在使用模板前从不登录，以保持其纯净）。以下是该模板的大致操作顺序（包括无害的失败尝试）：在命令行中使用 sysctl.vm.mem_profiling=never。NixOS 内核默认开启分配分析，跟踪表占用约 6MB（3.9MB vmalloc，1.9MB slab）。直接使用 mem_profiling=never 不是有效参数；内核会将其传给用户空间，但不会有任何改变，必须加上 sysctl. 前缀。

kfence.sample_interval=0 gets back the 2MB KFENCE pool. damon_stat.enabled=N and audit=0 get rid of the kdamond and kauditd threads. initcall_blacklist=ksm_init,tpm_init,init_tis drops ksmd (1.4MB) and the TPM bits (0.3MB plus a kthread). No TPM in these guests and KSM inside a guest is pointless. (For TierHive anyway) psi=0 ras=cec_disable nowatchdog nmi_watchdog=0 cryptomgr.notests 8250.nr_uarts=1 are all small, mostly fewer kthreads and less noise. On a 2GB test VM the whole cmd line set took non-cache memory from 128.0 to 121MB and kthreads from 64 to 59.

kfence.sample_interval=0 可以回收 2MB 的 KFENCE 池。damon_stat.enabled=N 和 audit=0 可以移除 kdamond 和 kauditd 线程。initcall_blacklist=ksm_init,tpm_init,init_tis 可以丢弃 ksmd (1.4MB) 和 TPM 相关部分（0.3MB 外加一个内核线程）。这些客户机中没有 TPM，且在客户机内部使用 KSM 是毫无意义的（至少对 TierHive 而言）。psi=0、ras=cec_disable、nowatchdog、nmi_watchdog=0、cryptomgr.notests、8250.nr_uarts=1 等设置都很小，主要是为了减少内核线程和系统噪音。在 2GB 的测试虚拟机上，整套命令行设置使非缓存内存占用从 128.0MB 降至 121MB，内核线程从 64 个减至 59 个。

transparent_hugepage=never was one of the biggest single wins. With THP on madvise the kernel bumps vm.min_free_kbytes to 41108 (3710 without THP at 896MB), which is reserved memory you never get to use. Turning it off freed 52MB at 2GB and gave 103MB more available at 896MB, obviously in high end or production there are reasons not to do this, but day to day use, you probably wont notice any difference.

transparent_hugepage=never 是单项收益最大的设置之一。当 THP 设置为 madvise 时，内核会将 vm.min_free_kbytes 提升至 41108（不开启 THP 时在 896MB 内存下仅为 3710），这是你永远无法使用的预留内存。将其关闭后，在 2GB 内存下释放了 52MB，在 896MB 内存下增加了 103MB 可用空间。显然，在高端或生产环境中不建议这样做，但在日常使用中，你可能根本察觉不到任何差异。

Blacklisted the hardware a KVM guest never has or doesn’t need: floppy, intel_agp, i2c_piix4, psmouse, mousedev, joydev, serio_raw, evdev, button (using tiny_power_button already handles ACPI shutdown), the rapl/cstate/uncore lot, kvm, cfg80211 (networkd was pulling it in for nl80211), ata_generic/pata_acpi, qemu_fw_cfg, dmi_sysfs, vga16fb. Blacklisted virtio_console as well, nothing uses the agent channel and it was holding an 8MB kmalloc-4k slab. vsock, vmci and virtiofs get install /bin/false rather than a blacklist. They get requested at switch-root with no device present, and with a plain blacklist vmw_vmci just loaded instead.

禁用了 KVM 客户机从不具备或不需要的硬件驱动：floppy、intel_agp、i2c_piix4、psmouse、mousedev、joydev、serio_raw、evdev、button（使用 tiny_power_button 已能处理 ACPI 关机）、rapl/cstate/uncore 系列、kvm、cfg80211（networkd 因 nl80211 而加载它）、ata_generic/pata_acpi、qemu_fw_cfg、dmi_sysfs、vga16fb。同样禁用了 virtio_console，因为没有任何程序使用该代理通道，且它占用了 8MB 的 kmalloc-4k slab。vsock、vmci 和 virtiofs 使用 install <mod> /bin/false 而非黑名单，因为它们在 switch-root 时会被请求，若仅使用黑名单，vmw_vmci 反而会被加载。

Blacklisted virtio_gpu and let the VNC console run on plain vgacon through virtio-vga. That one was really for a 6.18.59 fix/workaround, it does not follow to 7.3 but thats optional I guess. Initrd cut down to virtio_pci virtio_blk ext4 with includeDefaultModules = false, without importing the qemu-guest profile. Trimmed the kernel module tree with a small derivation that replaces system.modulesTree. It drops sound, GPU, wifi, USB, physical NIC, SCSI/ATA HBA drivers and so on, always keeping the deps of anything it keeps. 162MB down to 45MB on disk, and udev has nothing silly left to autoload. It runs inside nixos-rebuild so kernel updates get the same treatment for free.

禁用了 virtio_gpu，让 VNC 控制台通过 virtio-vga 在普通的 vgacon 上运行。这主要是为了 6.18.59 的修复/变通方案，在 7.3 内核中不再需要，但我猜这可以作为可选方案。Initrd 被精简为仅包含 virtio_pci、virtio_blk 和 ext4，设置 includeDefaultModules = false，且不导入 qemu-guest 配置。通过一个替换 system.modulesTree 的小型派生（derivation）精简了内核模块树。它移除了声音、GPU、WiFi、USB、物理网卡、SCSI/ATA HBA 驱动等，同时保留了必要依赖。磁盘占用从 162MB 降至 45MB，udev 也不再有无用的模块可自动加载。该过程在 nixos-rebuild 内部运行，因此内核更新也能自动获得同样的精简处理。

Disabled systemd-resolved, 8.3MB PSS and it was listening for LLMNR/mDNS on 0.0.0.0:5355. cloud-init writes a plain resolv.conf instead. Swapped systemd-timesyncd (4.6MB) for a oneshot busybox ntpd -n -q on a timer, at boot and hourly, so nothing stays resident (won’t work everywhere but on most KVM hosts it should be fine) services.openssh.startWhenNeeded = true so sshd is socket activated and the 3.9MB listener only exists while someone is connected. (Perfect for NAT but if you are sitting with port 22 listening for connections on a public IP you might want to skip this) Disabled systemd-oomd (1.8MB).

禁用了 systemd-resolved（占用 8.3MB PSS，且在 0.0.0.0:5355 上监听 LLMNR/mDNS）。改由 cloud-init 写入简单的 resolv.conf。将 systemd-timesyncd (4.6MB) 替换为通过定时器运行的 busybox ntpd -n -q（在启动时和每小时运行），这样就不会有常驻进程（虽然不适用于所有环境，但在大多数 KVM 主机上应该没问题）。设置 services.openssh.startWhenNeeded = true，使 sshd 通过 socket 激活，3.9MB 的监听进程仅在有人连接时存在（这对 NAT 环境很完美，但如果你在公网 IP 上监听 22 端口，可能需要跳过此项）。禁用了 systemd-oomd (1.8MB)。

A tiny oneshot drops LLDP=no into the cloud-init generated .network files, which stops the networkd LLDP socket and af_packet loading at idle. What’s left resident is systemd, udevd, nsncd, networkd, journald, logind, dbus-broker and one getty, about 40MB of userland PSS. zram with zstd at memoryPercent = 100 capped at 512MB, no disk swap. This, plus the next one, is what makes rebuilding in 512MB possible. vm.swappiness = 180 so the kernel would rather compress cold anon pages into zram than throw away page cache, vm.page-cluster = 0 since read-ahead on zram is wasted memory, and vm.watermark_scale_factor = 125 with vm.watermark_boost_factor = 0 so kswapd starts earlier and the rebuild doesn’t stall in direct reclaim. During a rebuild at 512MB zram peaked at 116MB of real RAM. Tuned the Boehm GC

一个微小的 oneshot 脚本将 LLDP=no 写入 cloud-init 生成的 .network 文件中，这阻止了 networkd 在空闲时加载 LLDP socket 和 af_packet。剩下的常驻进程包括 systemd、udevd、nsncd、networkd、journald、logind、dbus-broker 和一个 getty，用户空间 PSS 占用约 40MB。使用 zstd 压缩的 zram，memoryPercent = 100，上限 512MB，无磁盘交换分区。这与下一项设置共同使得在 512MB 内存下进行重建成为可能。vm.swappiness = 180，让内核优先将冷匿名页压缩到 zram 而非丢弃页缓存；vm.page-cluster = 0，因为 zram 的预读是内存浪费；vm.watermark_scale_factor = 125 配合 vm.watermark_boost_factor = 0，使 kswapd 更早启动，从而避免重建过程因直接回收（direct reclaim）而卡顿。在 512MB 内存下进行重建时，zram 的实际内存占用峰值为 116MB。调整了 Boehm GC。