ratatop day 2: the memory box, and the lie in `free -h`

Ratatop Day 2: The Memory Box, and the Lie in free -h

Ratatop 第二天:内存框与 free -h 中的谎言

Hello, I’m Maneshwar. I’m building git-lrc, a Micro AI code reviewer that runs on every commit. It is free and source-available on Github. Star git-lrc to help devs discover the project. Do give it a try and share your feedback. 你好,我是 Maneshwar。我正在开发 git-lrc,这是一个可以在每次提交时运行的微型 AI 代码审查工具。它在 Github 上免费且源码可见。请为 git-lrc 点个星标,帮助更多开发者发现这个项目。欢迎试用并分享你的反馈。

Yesterday I got the CPU box working in ratatui and wrote up my first impressions coming from Bubble Tea. Today: memory. I assumed this would be the easy one. CPU needed delta tracking between samples, per-core bookkeeping, braille bit twiddling. Memory is just a file with numbers in it. Read it, print it, go home. Reader, it was not the easy one. 昨天我在 ratatui 中完成了 CPU 监控框,并写下了我从 Bubble Tea 转过来后的第一印象。今天的主题是:内存。我原以为这会很简单。CPU 监控需要样本间的增量跟踪、每个核心的记账以及盲文位操作。而内存只是一个包含数字的文件。读出来,打印出来,收工。读者们,这其实一点也不简单。

Free memory is not free memory. Look at that box again. Total 13.5 GiB. Free: 271 MiB. Two percent free. That sounds like a machine about to fall over. It is completely fine. I have 7.10 GiB available right now. This is the thing that trips up everyone reading /proc/meminfo for the first time, and it tripped me up too. “空闲内存”并非真正的空闲内存。再看看那个监控框:总计 13.5 GiB,空闲:271 MiB。仅剩 2% 的空闲空间。这听起来像是机器快要崩溃了,但实际上它完全正常。我现在有 7.10 GiB 可用内存。这是每个第一次阅读 /proc/meminfo 的人都会踩的坑,我也没能幸免。

MemFree means memory the kernel is doing absolutely nothing with. Not caching a file, not buffering a write, nothing. Just sitting there. And the kernel hates that. Unused RAM is wasted RAM. So it fills spare memory with page cache, and hands it back the instant a process actually needs it. So MemFree on a healthy machine trends toward zero, by design. It is a measure of how much RAM your kernel is failing to put to work. MemFree 指的是内核完全没有使用的内存。它没有缓存文件,没有缓冲写入,什么都没做,只是闲置在那里。而内核讨厌这样。未使用的内存就是浪费的内存。因此,内核会用页面缓存填满空闲内存,并在进程真正需要时立即将其收回。所以,在健康的机器上,MemFree 的值趋向于零是设计使然。它衡量的是内核有多少内存没有被利用起来。

The number you actually want is MemAvailable, added to the kernel in 3.14 precisely because everyone was misreading MemFree. It is an estimate of how much you can allocate right now without swapping, and it accounts for the fact that most of that cache is reclaimable. Note that Used and Available add up to Total, but Cached and Free do not slot in neatly beside them. Cached memory is counted inside available. They overlap. Trying to make all four sum to total is a trap, and I definitely tried for a bit. 你真正需要关注的数字是 MemAvailable。内核在 3.14 版本中引入它,正是因为每个人都误读了 MemFree。它估算了你当前在不使用交换分区(swap)的情况下可以分配多少内存,并考虑到了大部分缓存是可以回收的事实。请注意,Used(已用)和 Available(可用)相加等于 Total(总计),但 Cached(缓存)和 Free(空闲)并不能简单地与它们对应。缓存内存被计入可用内存中,它们是重叠的。试图让这四个数值相加等于总数是一个陷阱,而我确实尝试过一段时间。

So how do you compute “used”? Here is where I went looking at how btop does it instead of inventing my own. The obvious approach is to add up the parts. Total minus free minus cached minus buffers, something like that. This is what a lot of tools did historically, and it is where the old “Linux ate my RAM” confusion comes from. btop does the simple thing instead: used = MemTotal - MemAvailable. That is it. Let the kernel do the hard estimate, because it is the only thing that actually knows what is reclaimable. 那么,如何计算“已用”内存呢?我没有自己发明算法,而是去参考了 btop 的做法。显而易见的方法是将各部分相加:总数减去空闲、缓存和缓冲区,诸如此类。这是历史上许多工具的做法,也是“Linux 吃掉了我的内存”这一困惑的来源。btop 采取了更简单的做法:used = MemTotal - MemAvailable。就是这样。让内核去做复杂的估算,因为只有它真正知道哪些内存是可以回收的。

In Rust, my collector boils down to this: 在 Rust 中,我的收集器代码归结为:

pub struct Memory {
    pub total: u64,
    pub used: u64,
    pub available: u64,
    pub cached: u64,
    pub free: u64,
}

Plain u64 bytes, no units baked in. Formatting is the UI’s problem, not the collector’s. 纯粹的 u64 字节,不包含单位。格式化是 UI 的问题,而不是收集器的问题。

The bug that a substring gave me: for /proc/meminfo looks like this: 子字符串匹配给我带来的 Bug:/proc/meminfo 的内容如下:

MemTotal: 14177796 kB
MemFree: 497404 kB
MemAvailable: 7490900 kB
Buffers: 1235656 kB
Cached: 5146132 kB
SwapCached: 81068 kB

Spot the landmine. SwapCached contains the substring Cached. If you parse this with a contains or a loose prefix match, SwapCached arrives four lines later and quietly overwrites your real cached value with something roughly 60 times smaller. Your box renders. No error. No panic. Just a wrong number sitting there looking plausible. 发现地雷了吗?SwapCached 包含了 Cached 这个子字符串。如果你使用 contains 或松散的前缀匹配来解析,SwapCached 会在四行之后悄悄地用一个大约小 60 倍的值覆盖你真实的缓存值。你的监控框会正常渲染,没有错误,没有崩溃,只是显示了一个看起来还算合理的错误数字。

The fix is boring: split on the colon, match the label exactly. 修复方法很枯燥:按冒号分割,精确匹配标签。

let Some((label, rest)) = line.split_once(':') else { continue; };
match label {
    "MemTotal" => mem.total = bytes,
    "MemFree" => mem.free = bytes,
    "MemAvailable" => available = Some(bytes),
    "Cached" => mem.cached = bytes,
    _ => {}
}

I wrote a test named swap_cached_does_not_get_mistaken_for_cached purely so the next person who “simplifies” this parser gets caught. Also worth knowing: the kB suffix in that file is a lie too. Those are KiB, 1024 bytes, not 1000. So the multiply is << 10, not * 1000. 我写了一个名为 swap_cached_does_not_get_mistaken_for_cached 的测试,纯粹是为了让下一个试图“简化”这个解析器的人被抓个正着。另外值得一提的是:该文件中的 kB 后缀也是个谎言。它们实际上是 KiB(1024 字节),而不是 1000。所以乘法应该是 << 10,而不是 * 1000

Three significant figures, and why. Look at the value column again: 13.5 GiB, 6.42 GiB, 271 MiB. Different decimal counts. That is deliberate, and it is what btop does. The rule is three significant figures, always. Under 10, you get two decimals. Between 10 and 100, one. Above 100, none. The number is always exactly three digits wide, so the column stays aligned no matter what the values do. 关于三位有效数字及其原因。再看看数值列:13.5 GiB、6.42 GiB、271 MiB。小数位数不同。这是故意的,也是 btop 的做法。规则始终是三位有效数字。小于 10 时,保留两位小数;10 到 100 之间,保留一位;大于 100,不保留小数。数字始终保持三位宽度,因此无论数值如何变化,列都能保持对齐。

let decimals = if unit == 0 || value >= 100.0 { 0 } 
               else if value >= 10.0 { 1 } 
               else { 2 };

Small thing. But a column that jitters between 6.4 GiB and 13.52 GiB looks amateurish in a way you feel before you can name it. 这只是小事一桩。但如果一列数字在 6.4 GiB 和 13.52 GiB 之间跳动,会显得非常业余,这种感觉在你意识到原因之前就能体会到。

One thing I got wrong on the first pass: my loop divided by 1024 while the value was too large, with no guard on the unit table. Feed that a u64::MAX and it walks straight off the end of the array. Now it stops at TiB, with a test that passes u64::MAX specifically. 第一版中我犯了一个错误:我的循环在数值过大时不断除以 1024,但没有对单位数组进行边界检查。如果传入 u64::MAX,它会直接越界。现在它在 TiB 处停止,并且我专门添加了一个传入 u64::MAX 的测试。

Where ratatui made this pleasant: Two things stood out building a second box. Each field gets its own gradient. btop’s theme gives used, available, cached and free their own three-stop colour ramps, and I lifted the exact hex values out of btop_theme.cpp. Because a Gradient is just a value in my code, colouring a meter is passing a different one in. Ratatui 让开发变得愉悦的地方:在构建第二个监控框时,有两点非常突出。每个字段都有自己的渐变色。btop 的主题为已用、可用、缓存和空闲内存提供了各自的三色渐变,我直接从 btop_theme.cpp 中提取了这些十六进制值。因为在我的代码中,渐变(Gradient)只是一个值,所以给仪表上色只需传入不同的渐变对象即可。

Immediate mode really pays off here. There is no widget to construct, register, style and later mutate. You are in a loop, you have a Rect, you draw. Four fields, four iterations, done. The meter widget was already written. I built it for the CPU bar on day one, and reusing it for memory took zero changes. That is the nice consequence of ratatui’s Widget trait being tiny: a widget is a struct plus one render method, so a generic one you wrote yesterday just works today. 即时模式(Immediate mode)在这里发挥了巨大作用。无需构建、注册、设置样式然后再修改组件。你处于循环中,拥有一个矩形区域,然后直接绘制。四个字段,四次迭代,搞定。仪表组件(meter widget)之前已经写好了。我第一天为 CPU 条构建了它,重用它来显示内存时无需任何修改。这就是 ratatui 的 Widget trait 非常精简带来的好处:一个组件就是一个结构体加上一个渲染方法,所以你昨天写的通用组件今天就能直接使用。