I hate packaging my software for Linux

I hate packaging my software for Linux

我讨厌为 Linux 打包我的软件

I hate packaging my software for Linux. All I wanted was to make Fresh easy to install, for everybody, everywhere - how hard can it be?! Windows and macOS aren’t so bad, because they’re relatively uniform. For Windows, I use winget - ugly, but works (once I got past the initial headache). For macOS, I’m using homebrew for now, which isn’t ideal, but it wasn’t hard to set up. I’ll probably move to a more native signed mac app once I take the time to package it correctly. Both solutions work ok for modern Windows and macOS users. 我讨厌为 Linux 打包我的软件。我最初的想法很简单:让 Fresh 在任何地方、对任何人来说都易于安装——这能有多难?!Windows 和 macOS 没那么糟糕,因为它们相对统一。对于 Windows,我使用 winget——虽然界面丑陋,但确实有效(克服了最初的头痛之后)。对于 macOS,我目前使用 homebrew,虽然不够理想,但设置起来并不难。一旦我有时间正确地进行打包,我可能会转向更原生的签名 Mac 应用。这两种方案对于现代 Windows 和 macOS 用户来说都运行良好。

Linux is a different story. I started by releasing Fresh as an npm package. This felt ugly and annoyed some people on hn but also has real problems: Linux 的情况则完全不同。我最初将 Fresh 作为 npm 包发布。这感觉很糟糕,也惹恼了 Hacker News 上的一些人,而且它确实存在实际问题:

  • Security - npmjs has had multiple breaches and I want to sleep better at night.
  • Not universal - many users don’t have npm installed, why would they install npm just to install Fresh?
  • Weird installer - the npm install flow, it’s actually a script that you get from npmjs which goes to github and downloads the correct binary artifact for your machine. You have to do this every time you want to update.
  • 安全性 - npmjs 曾多次发生安全漏洞,我希望能睡个安稳觉。
  • 非通用性 - 许多用户并没有安装 npm,他们为什么要为了安装 Fresh 而专门去安装 npm 呢?
  • 奇怪的安装程序 - npm 的安装流程实际上是一个从 npmjs 获取的脚本,它会跳转到 GitHub 并为你的机器下载正确的二进制文件。每次更新时你都必须重复这个过程。

So I listened to the annoyed people’s feedback, and (with the help of several gracious contributors) we went ahead and created packages for “EVERYTHING”: 所以我听取了那些感到困扰的用户的反馈,并在几位热心贡献者的帮助下,为“所有平台”创建了安装包:

  • rust’s cargo on crates.io (and also cargo-binstall)
  • un-distros like AppImage and Flatpak
  • deb
  • rpm
  • AUR (Arch Linux), two variants - source build and pre-built binary
  • bin package
  • nix
  • mise
  • homebrew for linux (wtf?)
  • npm / npx
  • Terra
  • Gentoo GURU
  • Plus I’m releasing pre-built binaries as a tarball.
  • Rust 的 cargo (crates.io 以及 cargo-binstall)
  • 通用发行版格式,如 AppImage 和 Flatpak
  • deb
  • rpm
  • AUR (Arch Linux),包含源码构建和预编译二进制两个版本
  • bin 包
  • nix
  • mise
  • Linux 版 homebrew(什么鬼?)
  • npm / npx
  • Terra
  • Gentoo GURU
  • 此外,我还以 tarball 形式发布预编译的二进制文件。

You can imagine how fragile all of this can be. After each release I wonder if I’ll hit some problem or another with one of the many channels. Every one of those solutions works for some users but none of them work for all users. And they all have drawbacks. 你可以想象这一切有多么脆弱。每次发布后,我都在担心是否会在众多渠道中的某一个遇到问题。这些解决方案中的每一个都适用于部分用户,但没有一个能覆盖所有用户。而且它们都有各自的缺点。

“Why not use nix?!” Because many people DON’T HAVE NIX installed and don’t want to install it just to use my app. I support nix as a method but it doesn’t work for everybody. “为什么不用 nix?!”因为很多人根本没有安装 nix,也不想为了使用我的应用而去安装它。我支持 nix 作为一种分发方式,但它并不适用于所有人。

“Just use Flatpak!” I do release a Flatpak but I’m actually working against it. It’s designed for self-contained sandboxed desktop GUI applications and I’m releasing a terminal-based TUI that allows you to rampage around your machine and network. I’m passing some horrible “bad practice” flags to get it to work. Flatpak sandboxing is not a good fit. “直接用 Flatpak 吧!”我确实发布了 Flatpak 版本,但我实际上是在违背它的设计初衷。它旨在用于自包含、沙盒化的桌面 GUI 应用,而我发布的是一个基于终端的 TUI(终端用户界面),它允许你在机器和网络上进行各种操作。为了让它运行,我不得不传入一些糟糕的“坏习惯”标志。Flatpak 的沙盒机制并不适合它。

“Just use AppImage!” I do release an AppImage, but it’s extremely slow to run because of the squashfs FUSE-mount on-demand, which makes bringup time unacceptably slow. I want my program to start instantly, as quickly as technically possible. To make bringup times reasonable, my installer script bakes in a horrible hack to extract the squashfs contents somewhere and drop the AppImage. You can still just run the AppImage if you prefer, but it’ll be annoyingly slow to start. And in both cases it requires FUSE to be installed. Why does my app need FUSE?! “直接用 AppImage 吧!”我确实发布了 AppImage,但由于 squashfs FUSE 按需挂载的机制,它的运行速度极慢,导致启动时间长得令人无法接受。我希望我的程序能瞬间启动,尽可能快。为了让启动时间合理,我的安装脚本内置了一个糟糕的 hack,将 squashfs 内容提取到某个地方并放弃 AppImage 格式。如果你愿意,仍然可以直接运行 AppImage,但启动速度会慢得令人恼火。而且这两种情况都需要安装 FUSE。为什么我的应用需要 FUSE?!

Also, since my binary requires some minimal libc version, it isn’t actually universally portable (so for a 2-3 year old distro it might not work), hence I might as well use static linking and also just drop AppImage. For some reason I imagined that using AppImage alone solves this portability problem. 此外,由于我的二进制文件需要最低版本的 libc,它实际上并不是完全通用的(对于两三年前的发行版可能无法运行),所以我还不如直接使用静态链接,并放弃 AppImage。不知为何,我曾天真地以为仅靠 AppImage 就能解决这种可移植性问题。

Lastly, for various reasons, many people have a bad view of AppImage and Flatpak (and Snap) and they would just refuse to use it. I lose. 最后,由于各种原因,许多人对 AppImage 和 Flatpak(以及 Snap)印象不佳,他们会直接拒绝使用。我输了。

Debian family pain

Debian 系列的痛苦

As explained in this other hn discussion I hadn’t gotten around to pushing my .deb as a Debian (and Ubuntu) official package, because it requires all the (many) rust dependencies to also be Debian packages. There are many good reasons for this policy - fully reproducible and self-contained builds for any package, reduces supply chain hell, etc. - but it’s a lot of work to do. And how will I ever keep up - every one of my direct dependencies will need to be re-updated on Debian on every security issue etc. I don’t have time for that. I can’t just vendor all my deps as sources into my deb source package, that goes against the policy. 正如在另一篇 Hacker News 讨论中所解释的那样,我一直没能将我的 .deb 推送为 Debian(和 Ubuntu)的官方包,因为它要求所有(大量的)Rust 依赖项也必须是 Debian 包。这一政策有很多充分的理由——确保任何包都能实现完全可复现和自包含的构建,减少供应链地狱等——但这需要做大量工作。而且我怎么可能跟得上呢?每当出现安全问题时,我的每一个直接依赖项都需要在 Debian 上重新更新。我没有时间做这些。我不能仅仅把所有依赖项作为源码打包进我的 deb 源码包中,这违反了政策。

Another fun anecdote is that I do want to support older machines running e.g. older Ubuntu (and I guess any older distro) - but these have older libc, meaning my build for newer ubuntu can’t be installed on the older ones as they fail on link error at binary load time. So I’d have to build in an old Ubuntu container image and inherit its unpatched packages into my build - or drop those users entirely. Neither option holds up. 另一个有趣的插曲是,我确实想支持运行旧版 Ubuntu(以及其他旧版发行版)的旧机器,但它们使用的是旧版 libc,这意味着我为新版 Ubuntu 构建的版本无法在旧系统上安装,因为它们会在二进制加载时出现链接错误。所以我必须在旧版 Ubuntu 容器镜像中进行构建,并将其中未修补的包继承到我的构建中——或者干脆放弃这些用户。这两个选项都行不通。

No automatic updates for .deb / .rpm

.deb / .rpm 没有自动更新

Because it’s such a headache, I didn’t get my package .deb (or .rpm) accepted into the official sources. Therefore, people who install these packages don’t get automatic updates when they run their system’s native package update mechanism, apt-get upgrade or equivalent. It would’ve been nice if there was a quick serverless solution where you could just say a URL to look for newer versions of a single package, and both apt and dnf would remember that and update a package. 因为这太让人头疼了,我没能让我的 .deb(或 .rpm)包被官方源收录。因此,安装这些包的用户在运行系统原生的包更新机制(如 apt-get upgrade 或同类工具)时,无法获得自动更新。如果有一个快速的无服务器解决方案,只需指定一个 URL 来查找单个包的新版本,并且 apt 和 dnf 都能记住并更新该包,那该多好。

I’m well aware that the “correct” thing to do here is to get my package into the official channels. I opened a bugzilla thing for Fedora and someone started a whatever thing for Ubuntu and there are votes etc, but I just don’t have the time and energy to push these requests through. I just want to GIVE AWAY MY SOFTWARE. I guess my point is: since people use all kinds of distros, the compounded effort of doing this is very high. 我很清楚,这里“正确”的做法是将我的包推送到官方渠道。我为 Fedora 开了一个 Bugzilla 工单,有人为 Ubuntu 发起了一些流程,也有投票等等,但我实在没有时间和精力去推动这些请求。我只是想免费提供我的软件。我想表达的是:由于人们使用各种各样的发行版,完成这些工作的累积成本实在太高了。

Mise worked but then it didn’t

Mise 曾经有效,但后来不行了

Turns out some people use mise as their local (user-level) package manager. Someone got it to work for Fresh, which is great! But one day it stopped working with no change on my end. Turns out GitHub rotated the certificate of its build attestation system, and mise had pinned an outdated trust root instead of fetching the current one. I didn’t know anything was broken until someone complained. I ended up fixing it myself but why am I spending my time on that kind of problem? Do I even know what I’m doing? I mean mise is cool and I also use it for some projects but as an author of some software tool - there are just too many of these things. 事实证明,有些人使用 mise 作为他们的本地(用户级)包管理器。有人让它在 Fresh 上运行起来了,这很棒!但有一天,在我的端没有任何改动的情况下,它突然停止工作了。原来是 GitHub 轮换了其构建认证系统的证书,而 mise 锁定了一个过期的信任根,而不是获取当前的证书。直到有人投诉,我才知道出了问题。我最终自己修复了它,但为什么我要把时间花在这种问题上?我到底在做什么?我的意思是,mise 很酷,我也在一些项目中使用它,但作为一个软件工具的作者——这类琐事实在太多了。