Any Nix package, live in your browser
Any Nix package, live in your browser
任何 Nix 软件包,直接在浏览器中运行
tl;dr Try it at https://trynix.dev. Click hello, or python 3.6.2 from 2017, or two eras of hello at once, or a package that exists in no public cache. A Linux machine boots in the tab and you get a shell with those Nix packages on PATH. This is my magnum opus of Nix work. 简而言之,请访问 https://trynix.dev 体验。点击 hello,或者 2017 年的 python 3.6.2,或者同时运行两个不同时代的 hello,甚至是一个在任何公共缓存中都不存在的软件包。一个 Linux 机器会在标签页中启动,你将获得一个包含这些 Nix 软件包的 shell 环境。这是我在 Nix 领域最得意的作品。
I knew all the ideas I have been creating were building blogs for something greater: nixpkgs-multiverse indexed every version of every package nixpkgs ever shipped, grail taught it version ranges and omniflake allowed adding over sixteen thousand flakes from a single input. The crazy insight I had lately was the craziness of the “fast-mode” of the nixmultiverse.com, which lets you skip evaluation and go straight to the store path. This lets you leverage the amazingness of Nix without having to deal with the complexity of evaluation and building. 我深知我所创造的所有想法都在为更宏大的目标铺路:nixpkgs-multiverse 索引了 nixpkgs 曾经发布过的每一个软件包的所有版本,grail 为其引入了版本范围,而 omniflake 则允许从单个输入添加超过 16,000 个 flake。我最近产生的一个疯狂灵感来自于 nixmultiverse.com 的“快速模式”,它让你跳过评估阶段,直接定位到存储路径(store path)。这让你无需处理评估和构建的复杂性,就能利用 Nix 的强大功能。
You can just ask for a package and get the exact store path that Hydra built for it, at any version it ever had. If we have the produced binaries, we can run them. And if we can run them, we can run any of them, in a browser tab, with nothing installed on the host machine. Welcome to trynix, a browser-based Nix package runner. 你只需请求一个软件包,就能获得 Hydra 为其构建的确切存储路径,涵盖其曾经发布过的任何版本。如果我们拥有生成的二进制文件,我们就能运行它们。如果我们能运行它们,我们就能在浏览器标签页中运行其中的任何一个,而无需在宿主机上安装任何东西。欢迎来到 trynix,一个基于浏览器的 Nix 软件包运行器。
You can browse the complete history of nixpkgs, over 310,083 package versions, and run any of them in a Linux machine that boots in your tab. It is a Nix store in memory, a Linux kernel in WebAssembly, and a terminal emulator in the page. 你可以浏览 nixpkgs 的完整历史记录(超过 310,083 个软件包版本),并在标签页中启动的 Linux 机器里运行其中的任何一个。它是一个内存中的 Nix 存储库、一个 WebAssembly 格式的 Linux 内核,以及页面中的终端模拟器。
Screencast: searching nixpkgs for pfetch, picking a version, and booting it to a shell in the browser This is bonkers! 🤯 We can boot the VM with the store-path closure within seconds. Nothing is pre-installed: search, pick a version, boot, run it. The craziest part? We are not restricted to the public cache. You can share a store path you built yourself, and anyone can boot it in their browser tab. 演示视频:在 nixpkgs 中搜索 pfetch,选择一个版本,并在浏览器中启动 shell。这太疯狂了!🤯 我们可以在几秒钟内启动带有存储路径闭包的虚拟机。无需预装任何东西:搜索、选择版本、启动、运行。最疯狂的部分是什么?我们不局限于公共缓存。你可以分享你自己构建的存储路径,任何人都能在他们的浏览器标签页中启动它。
The only requirement is that the cache is served with access-control-allow-origin: *, which GitHub Pages does for free, so does Cachix and obviously cache.nixos.org as well. In a unbelievable twist of fate, I had actually requested 5 years ago for cache.nixos.org to serve access-control-allow-origin: * via issue#156 to make it possible to query the cache from an OpenAPI specification I had implemented. Thank you universe. 🙏
唯一的条件是缓存必须设置 access-control-allow-origin: *,GitHub Pages 可以免费做到这一点,Cachix 和 cache.nixos.org 当然也可以。命运的巧合是,我曾在 5 年前通过 issue#156 请求 cache.nixos.org 开启 access-control-allow-origin: *,以便从我实现的 OpenAPI 规范中查询缓存。感谢宇宙。🙏
This link boots a VM with a store-path served from Github Pages of a modified GNU hello. This is a store path that does not exist on cache.nixos.org and yet it boots in your browser tab. 这个链接启动了一个虚拟机,其存储路径由 GitHub Pages 提供,运行的是一个修改版的 GNU hello。这是一个在 cache.nixos.org 上不存在的存储路径,但它依然能在你的浏览器标签页中启动。
Making the pieces fit
让各个部分各就各位
Since we can access store-paths from caches that serve access-control-allow-origin: *, that makes the browser a legitimate Nix client. The missing piece the browser lacked was somewhere to run the binaries since they store-paths are either x86-64 or aarch64 ELF executables. Standing on the shoulders of giants, we can run a Linux kernel in WebAssembly. This means we can boot a real x86_64 kernel inside our browser tab. Give that kernel a filesystem containing a Nix store. All that’s left knowing which store-paths to fetch, which we beautifully solved with nixpkgs-multiverse. 🤌
既然我们可以从支持 access-control-allow-origin: * 的缓存中访问存储路径,这就使浏览器成为了一个合法的 Nix 客户端。浏览器缺失的部分是运行二进制文件的环境,因为这些存储路径中的文件要么是 x86-64,要么是 aarch64 的 ELF 可执行文件。站在巨人的肩膀上,我们可以在 WebAssembly 中运行 Linux 内核。这意味着我们可以在浏览器标签页中启动一个真实的 x86_64 内核。为该内核提供一个包含 Nix 存储库的文件系统,剩下的工作就是确定要获取哪些存储路径,而我们通过 nixpkgs-multiverse 完美地解决了这个问题。🤌
I have to keep reminding myself: there is no server in the above picture, the web-page is purely static files and everything else is a publicly accessible cache. It is a virtual machine that exists only inside your tab. The ultimate embodiment of “Erase your darlings.” 我必须不断提醒自己:上述场景中没有服务器,网页完全是静态文件,其他一切都是可公开访问的缓存。这是一个仅存在于你标签页中的虚拟机。这是“Erase your darlings”(抛弃你所珍视的)理念的终极体现。
Since this is Nix, we get the simplicity of managing multiple versions of the same package. You can boot two versions of hello in one machine, and they will not conflict because each binary names its own dependencies by absolute path (RUNPATH) down to the loader and libc. Once the VM is already started, you can add more store-paths to it while it’s running. This is no different than adding more paths to your own /nix/store on your laptop. No reboot, or dnf install, or apt-get install, the site fetches the closure and adds it to the store.
因为这是 Nix,我们获得了管理同一软件包多个版本的简便性。你可以在一台机器中启动两个版本的 hello,它们不会冲突,因为每个二进制文件都通过绝对路径(RUNPATH)指定了其依赖项,一直到加载器和 libc。一旦虚拟机启动,你可以在运行过程中向其添加更多的存储路径。这和你笔记本电脑上向 /nix/store 添加路径没有什么不同。无需重启,无需 dnf install 或 apt-get install,网站会自动获取闭包并将其添加到存储库中。
It has to feel instant
必须感觉即时
Booting a kernel under emulation is slow, and despite the amazingness of the idea, no one would use it if it took 30 seconds to get a shell. The site employs some neat tricks to make it feel instant. The site pre-fetches the engine and the VM snapshot in the background, so by the time you click a link, you have already downloaded it. The site also never boots the VM from scratch. It resumes. 在模拟环境下启动内核很慢,尽管这个想法很棒,但如果需要 30 秒才能进入 shell,没人会使用它。该网站采用了一些巧妙的技巧来使其感觉即时。网站会在后台预取引擎和虚拟机快照,因此当你点击链接时,它已经下载完毕了。此外,网站从不从零开始启动虚拟机,而是进行恢复。
A machine is booted once, ahead of time, on a native build of the same QEMU, and paused at the moment before it mounts the store which is then saved to a snapshot. Subsequent visits to the site have the engine and snapshot already in the browser cache, so the only thing that has to be fetched is the closure of the store-path you asked for. This makes each subsequent visit feel much faster. 机器会提前在相同 QEMU 的原生构建版本上启动一次,并在挂载存储库之前暂停,然后保存为快照。后续访问网站时,引擎和快照已经存在于浏览器缓存中,因此唯一需要获取的就是你所请求的存储路径的闭包。这使得后续的访问感觉快得多。
I have to give a lot of credit to LLMs here for helping find a lot of the performance opportunities and bottlenecks. What first started as a “neat idea” turned into an incredibly usable project with their help. Despite all the performance work, it is still not instant. It is fast enough to be usable, but it is not instant. Execution of a binary is still slow, because it is running under emulation. The first time you run a binary, it is translated from x86_64 to WebAssembly and that takes time. Subsequent runs are faster, because the translation is cached in memory. 我必须高度赞扬 LLM 在此过程中帮助发现了许多性能优化机会和瓶颈。在它们的帮助下,最初的一个“好点子”变成了一个极其好用的项目。尽管做了所有的性能优化,它仍然不是即时的。它足够快以至于可用,但并非即时。二进制文件的执行仍然很慢,因为它是在模拟下运行的。第一次运行二进制文件时,它会从 x86_64 转换为 WebAssembly,这需要时间。后续运行会更快,因为转换结果已缓存在内存中。
Lastly, we have an upper-bound on the size of the closure we can fetch. The whole closure has to fit in tab memory, which is set to a hard limit of ~1.5GiB as of now and WebAssembly has a hard limit of 4GiB as it is a 32-bit address space. 最后,我们对可以获取的闭包大小有一个上限。整个闭包必须放入标签页内存中,目前硬性限制约为 1.5GiB,而 WebAssembly 由于是 32 位地址空间,其硬性限制为 4GiB。
More than a parlor trick
不仅仅是杂耍
The demo is clearly fun and impressive, but is it more than a parlor trick? I have been thinking of endless ideas of ways in which this… 这个演示显然既有趣又令人印象深刻,但它不仅仅是杂耍吗?我一直在思考无数种方法,让这个……