Stop Thrashing Under Memory Pressure: Practical zram + systemd-oomd on Linux
Stop Thrashing Under Memory Pressure: Practical zram + systemd-oomd on Linux
Stop Thrashing Under Memory Pressure: Practical zram + systemd-oomd on Linux 当家庭实验室服务器或小型 VPS 的可用内存耗尽时,其故障模式很少是干净利落的进程终止。更常见的情况是机器会陷入长达数分钟的“抖动”(thrashing):匿名页面被挤压到缓慢的磁盘交换分区,页面缓存崩溃,SSH 连接变得卡顿,最后内核的 OOM Killer 才会介入。
Two complementary tools fix different halves of that story: 有两个互补的工具可以分别解决这一问题的一半:
- zram — a compressed block device in RAM, usually used as high-priority swap so reclaim stays in memory instead of on disk. zram — 一个位于内存中的压缩块设备,通常用作高优先级的交换空间,使内存回收留在内存中而不是写入磁盘。
- systemd-oomd — a userspace OOM daemon that watches cgroup v2 pressure stall information (PSI) and kills a descendant cgroup before the whole host livelocks. systemd-oomd — 一个用户空间的 OOM 守护进程,它监控 cgroup v2 的压力停顿信息(PSI),并在整个主机陷入死锁之前终止某个子 cgroup。
This post is a practical setup for both, with verification and a clean rollback. It is not a hibernation guide, not a zswap deep dive, and not another MemoryMax= sandbox recipe. 本文提供了两者的实用配置方案,并包含验证和干净的回滚方法。这不是一份休眠指南,也不是对 zswap 的深度剖析,更不是另一份 MemoryMax= 沙盒配置教程。
What you are actually fixing
你到底在解决什么问题
Swap is not “emergency RAM.” Chris Down’s well-known write-up makes the real point: swap exists so rarely used anonymous pages can be reclaimed the same way clean file pages can. Without any swap, those anonymous pages stay pinned, reclaim is less egalitarian, and under pressure you thrash the page cache instead of a swap device. 交换空间(Swap)并不是“紧急内存”。Chris Down 的著名文章指出了核心观点:交换空间的存在是为了让那些很少使用的匿名页面能够像干净的文件页面一样被回收。如果没有交换空间,这些匿名页面就会被锁定,内存回收机制就不那么公平,当压力增大时,你最终会抖动页面缓存,而不是交换设备。
Disk-backed swap still works — but on a busy SSD or a slow VPS volume it can turn moderate pressure into multi-second stalls. zram keeps that swap path in RAM with compression (kernel docs note a rough ~2:1 expectation; real ratios vary by workload). systemd-oomd then uses the breathing room that swap creates so it can react on PSI instead of waiting for the global kernel OOM path. 基于磁盘的交换空间依然有效,但在繁忙的 SSD 或缓慢的 VPS 存储上,它可能将中等压力转化为长达数秒的系统停顿。zram 通过压缩将交换路径保留在内存中(内核文档指出压缩比预期约为 2:1;实际比例取决于工作负载)。随后,systemd-oomd 利用交换空间创造的缓冲余地,根据 PSI 做出反应,而不是等待全局内核 OOM 路径触发。
Memory pressure 内存压力 │ ├─► reclaim cold anon pages → zram swap (compressed RAM) │ ├─► 回收冷匿名页面 → zram 交换(压缩内存) │ └─► sustained PSI / swap exhaustion → systemd-oomd SIGKILL of a leaf cgroup │ └─► 持续的 PSI / 交换空间耗尽 → systemd-oomd 发送 SIGKILL 终止某个叶子 cgroup
Prerequisites
前置条件
- Linux with the zram module (common on modern distros). 带有 zram 模块的 Linux(现代发行版中很常见)。
- cgroup v2 unified hierarchy (systemd default on current Debian/Ubuntu/Fedora/Arch). cgroup v2 统一层级(当前 Debian/Ubuntu/Fedora/Arch 中 systemd 的默认设置)。
- Kernel PSI support (mainline since 4.20):
/proc/pressure/memorymust exist. 内核 PSI 支持(4.20 版本后进入主线):必须存在/proc/pressure/memory。 - Packages:
zram-generator(Fedora ships it by default on many spins; Arch packagezram-generator; Debian/Ubuntu package name is typicallyzram-toolsor install upstream/distrosystemd-zram-generator/zram-generatordepending on release — confirm with your package manager). 软件包:zram-generator(Fedora 的许多版本默认提供;Arch 下为zram-generator;Debian/Ubuntu 下包名通常为zram-tools,或者根据发行版安装上游/发行版提供的systemd-zram-generator/zram-generator—— 请通过你的包管理器确认)。 - systemd-oomd (package
systemd-oomdon Debian/Ubuntu; often already present on Fedora). systemd-oomd(Debian/Ubuntu 下包名为systemd-oomd;Fedora 通常已预装)。
Check the basics: 检查基础环境:
# cgroup v2?
mount | grep -E 'cgroup2|type cgroup2'
# PSI present?
cat /proc/pressure/memory
# zram module available?
modinfo zram | head
Part 1 — Compressed swap with zram-generator
第一部分 — 使用 zram-generator 进行压缩交换
Why the generator instead of a one-shot script 为什么使用生成器而不是一次性脚本
zram-generator is a systemd unit generator. You drop a small conf file; at boot it creates systemd-zram-setup@zramN.service, formats the device (swap by default), and activates it. No fragile rc scripts, no hand-rolled mkswap in rc.local.
zram-generator 是一个 systemd 单元生成器。你只需放置一个小的配置文件;在启动时,它会自动创建 systemd-zram-setup@zramN.service,格式化设备(默认为 swap),并激活它。无需脆弱的 rc 脚本,也不需要在 rc.local 中手动编写 mkswap。
Config path precedence (lowest to highest override style matching systemd norms): 配置路径优先级(从低到高,符合 systemd 的覆盖规范):
/usr/lib/systemd/zram-generator.conf/etc/systemd/zram-generator.conf← administrator file drop-ins under*.conf.d//etc/systemd/zram-generator.conf← 管理员在*.conf.d/下的配置文件
Kernel cmdline: systemd.zram=0 disables generator devices; systemd.zram=1 forces zram0 with defaults.
内核命令行: systemd.zram=0 禁用生成器设备;systemd.zram=1 强制使用默认配置创建 zram0。
Minimal working config 最小化工作配置
sudo tee /etc/systemd/zram-generator.conf >/dev/null <<'EOF'
[zram0]
# Uncompressed capacity as a function of MemTotal (MiB variable: ram).
# Default if omitted: min(ram / 2, 4096)
zram-size = min(ram / 2, 8192)
# Prefer a fast modern compressor when the kernel offers it.
compression-algorithm = zstd
# Higher than typical disk swap so zram is chosen first.
swap-priority = 100
# Default options already include discard; keep it explicit.
options = discard
EOF
After writing the conf: 写入配置后:
sudo systemctl daemon-reload
sudo systemctl start systemd-zram-setup@zram0.service
systemctl status systemd-zram-setup@zram0.service --no-pager
Verify zram swap 验证 zram 交换空间
zramctl
swapon --show
cat /proc/swaps
# Live compression stats
cat /sys/block/zram0/mm_stat
cat /sys/block/zram0/comp_algorithm
Disable zswap when zram is the primary swap
当 zram 作为主要交换空间时,禁用 zswap
If the kernel’s zswap pool is enabled, it sits in front of swap devices as a compressed cache. On many stock kernels it is on by default. Arch’s zram page is explicit: leaving zswap enabled can intercept pages before they reach zram and waste the setup. 如果内核的 zswap 池已启用,它会作为压缩缓存位于交换设备之前。在许多原生内核中,它是默认开启的。Arch 的 zram 文档明确指出:保持 zswap 开启可能会在页面到达 zram 之前将其拦截,从而浪费了 zram 的配置。
Runtime check and disable: 运行时检查与禁用:
# Current state
cat /sys/module/zswap/parameters/enabled 2>/dev/null || echo 'zswap module params not present'
# Temporary disable
echo 0 | sudo tee /sys/module/zswap/parameters/enabled
Persist with a kernel parameter (bootloader-specific): zswap.enabled=0
通过内核参数持久化(取决于引导加载程序):zswap.enabled=0
VM sysctls that make sense for in-memory swap
适用于内存交换的 VM sysctl 设置
Disk swap wants conservative swappiness. Compressed RAM swap is different: reclaiming into zram is often cheaper than dropping hot file cache. 磁盘交换需要保守的 swappiness 设置。压缩内存交换则不同:回收内存到 zram 通常比丢弃热文件缓存代价更小。
A widely copied starting point (Pop!_OS defaults / community zram benchmarks summarized on the ArchWiki): 一个被广泛采用的起点(Pop!_OS 默认值 / ArchWiki 上总结的社区 zram 基准测试):
sudo tee /etc/sysctl.d/99-vm-zram-parameters.conf >/dev/null <<'EOF'
# Bias reclaim toward anon pages
vm.swappiness = 100
EOF