Solo: a .so loader for static Linux binaries

Solo: a .so loader for static Linux binaries

Solo:用于静态 Linux 二进制文件的 .so 加载器

SoLo — a .so loader for static Linux binaries. Ship one musl-linked executable. At runtime, load the user’s existing glibc-linked GPU driver. No container, no AppImage, and no second libc in the process. SoLo 是一个用于静态 Linux 二进制文件的 .so 加载器。它允许你发布一个基于 musl 链接的可执行文件,并在运行时加载用户系统中现有的基于 glibc 链接的 GPU 驱动程序。无需容器,无需 AppImage,进程中也不需要第二个 libc。

Static binaries are a wonderfully boring way to deploy software on Linux: one file, no dependencies, nothing to break. We build ours with IX, a source-first build system for producing fully static Linux binaries. The boredom ends the moment the application needs the GPU: Vulkan and OpenGL drivers are supplied by the host as shared objects, usually built against glibc, and a fully static musl binary cannot normally dlopen() them. 静态二进制文件是在 Linux 上部署软件的一种极其“无聊”(指稳定可靠)的方式:单个文件,无依赖,不会出错。我们使用 IX(一个以源码为先的构建系统)来构建完全静态的 Linux 二进制文件。但当应用程序需要使用 GPU 时,这种“无聊”就结束了:Vulkan 和 OpenGL 驱动程序通常由宿主机以共享对象(shared objects)的形式提供,且通常是针对 glibc 构建的,而完全静态的 musl 二进制文件通常无法通过 dlopen() 加载它们。

SoLo crosses that boundary. It provides a dlfcn-style source API backed by its own ELF loader (x86-64 and aarch64) and a glibc ABI bridge implemented on top of musl. The result is still one ordinary static executable, but it can use the graphics driver already installed on the machine. SoLo 跨越了这一界限。它提供了一个 dlfcn 风格的源码 API,由其自身的 ELF 加载器(支持 x86-64 和 aarch64)以及在 musl 之上实现的 glibc ABI 桥接器提供支持。最终结果依然是一个普通的静态可执行文件,但它能够使用机器上已安装的图形驱动程序。

The repository includes an end-to-end Vulkan proof: a fully static executable loads the host’s unmodified Vulkan driver, runs a compute shader, and writes the result to a PNG. Tested on AMD radv, radeonsi, Intel, and NVIDIA GPUs under Linux, and on Apple M1 under Asahi Linux. The host keeps the hardware-specific code. You ship everything else. 该仓库包含一个端到端的 Vulkan 验证示例:一个完全静态的可执行文件加载宿主机未经修改的 Vulkan 驱动程序,运行计算着色器,并将结果写入 PNG 文件。该方案已在 Linux 下的 AMD radv、radeonsi、Intel 和 NVIDIA GPU 上,以及 Asahi Linux 下的 Apple M1 上进行了测试。宿主机保留硬件特定的代码,而你只需发布其余部分。

And not on a demo’s word alone: on every commit, CI loads the shared libraries of the 1,000 most-installed Debian packages — over 2,100 host objects — through SoLo, on both x86-64 and aarch64. 这不仅仅是一个演示:在每次提交时,CI 都会通过 SoLo 加载 Debian 中安装量最大的 1,000 个软件包的共享库(超过 2,100 个宿主机对象),并同时在 x86-64 和 aarch64 架构上进行测试。

See it work

查看运行效果

Grab the prebuilt binary — no clone, no toolchain, any Linux with a Vulkan driver installed (mesa-vulkan-drivers is enough): 获取预构建的二进制文件——无需克隆代码,无需工具链,任何安装了 Vulkan 驱动程序的 Linux 系统均可(安装 mesa-vulkan-drivers 即可):

curl -LO https://github.com/pg83/solo/releases/latest/download/vulkan-x86_64
chmod +x vulkan-x86_64
./vulkan-x86_64 hello.png

vulkan-aarch64 is the same demo for arm64 machines. The command discovers the distro-installed Vulkan ICD in the usual way and produces a 512×512 RGBA image. This is how we build the Shitty release binaries—a blazingly fast terminal emulator, BTW! vulkan-aarch64 是针对 arm64 机器的相同演示。该命令会以常规方式发现发行版安装的 Vulkan ICD,并生成一张 512×512 的 RGBA 图像。顺便提一下,我们就是用这种方式构建 Shitty(一个速度极快的终端模拟器)的发布版二进制文件的!

To force a particular driver: 强制指定特定驱动程序:

./vulkan-x86_64 --driver /usr/share/vulkan/icd.d/radeon_icd.x86_64.json radeon.png
./vulkan-x86_64 --driver /usr/share/vulkan/icd.d/lvp_icd.json lavapipe.png

ICD manifest names vary slightly between distributions. Passing no --driver lets the embedded Khronos loader perform its normal discovery. You can verify that the executable itself is not dynamically linked: 不同发行版之间的 ICD 清单名称略有不同。不传递 --driver 参数时,嵌入的 Khronos 加载器会执行正常的发现过程。你可以验证该可执行文件本身是否未进行动态链接:

readelf -lW ./vulkan-x86_64 | grep INTERP # 无输出
readelf -dW ./vulkan-x86_64 # "There is no dynamic section"

Or build the same demo from source, with Python 3 and a C/C++ compiler in PATH: 或者从源码构建相同的演示,前提是 PATH 中包含 Python 3 和 C/C++ 编译器:

git clone https://github.com/pg83/solo.git
cd solo
./build vulkan
./vulkan hello.png

This is not a toy call to vkCreateInstance. The demo: enters the statically linked Khronos Vulkan loader; loads the host’s Vulkan ICD and its non-glibc dependencies through SoLo; creates a device, storage buffer, descriptor set, and compute pipeline; dispatches a checked-in SPIR-V shader; maps the result and writes it through statically linked libpng. The complete example is in bin/vulkan, and the Vulkan program itself is in main.cpp. 这不仅仅是一个简单的 vkCreateInstance 调用。该演示过程如下:进入静态链接的 Khronos Vulkan 加载器;通过 SoLo 加载宿主机的 Vulkan ICD 及其非 glibc 依赖项;创建设备、存储缓冲区、描述符集和计算管线;分发一个内置的 SPIR-V 着色器;映射结果并通过静态链接的 libpng 写入文件。完整示例位于 bin/vulkan,Vulkan 程序本身位于 main.cpp

How it works

工作原理

┌──────────────────── fully static executable ────────────────────┐
│                                                                 │
│ application → embedded Vulkan loader → SoLo dlopen/dlsym        │
│                                                                 │
│ ├─ x86-64 ELF mapper                                            │
│ └─ glibc ABI → musl                                             │
│                                                                 │
└───────────────────────────────────────────┬─────────────────────┘
                                            │ maps at runtime

                                   system Mesa/Vulkan ICD.so + DSOs

elf_loader.cpp maps ELF segments, walks DT_NEEDED, resolves versioned symbols, applies x86-64 relocations, supports ELF TLS and TLSDESC, materializes IFUNCs, applies RELRO, and runs initializers. Dependencies that are themselves ELF DSOs are loaded recursively. elf_loader.cpp 负责映射 ELF 段、遍历 DT_NEEDED、解析版本化符号、应用 x86-64 重定位、支持 ELF TLS 和 TLSDESC、实现 IFUNC、应用 RELRO 并运行初始化程序。作为 ELF DSO 的依赖项会被递归加载。

glibc is deliberately not loaded. Imports such as malloc@GLIBC_2.2.5 are resolved by glibc_shim.cpp to ABI-correct adapters over the process’s existing musl runtime. Unsupported glibc functions have unique generated stubs that fail loudly with the exact symbol and version if they are ever called, instead of silently corrupting the process. glibc 被刻意排除在外。诸如 malloc@GLIBC_2.2.5 之类的导入由 glibc_shim.cpp 解析为针对进程现有 musl 运行时的 ABI 正确适配器。不支持的 glibc 函数具有独特的生成存根(stubs),如果被调用,它们会明确报错并指出确切的符号和版本,而不是静默地破坏进程。

Because musl sizes its synchronization objects to the glibc ABI of each architecture, the bridge does not shadow them: a pthread_mutex_t a driver creates is used in place. A lock is therefore one lock for both the loaded DSO and the static executable that may share it, and glibc’s static recursive and error-check initializers are adopted on first use. 由于 musl 将其同步对象的大小调整为各架构的 glibc ABI,因此桥接器不会对其进行遮蔽:驱动程序创建的 pthread_mutex_t 会被直接使用。因此,对于加载的 DSO 和可能共享它的静态可执行文件来说,锁是同一个,glibc 的静态递归和错误检查初始化程序会在首次使用时被采用。

Before loading a DSO from disk, SoLo checks its static provider registry. This lets an application satisfy a dependency—Wayland, for example—with functions already linked into the executable. LD_LIBRARY_PATH and DL_ELF_LIBRARY_PATH are honored for libraries outside the standard system directories. 在从磁盘加载 DSO 之前,SoLo 会检查其静态提供程序注册表。这允许应用程序使用已链接到可执行文件中的函数来满足依赖关系(例如 Wayland)。对于标准系统目录之外的库,LD_LIBRARY_PATHDL_ELF_LIBRARY_PATH 均会被遵循。

The interesting pieces are small enough to read: 核心代码非常简洁,易于阅读:

  • lib/dlfcn.cppdlopen, dlsym, errors, and static providers
  • lib/elf_loader.cpp — ELF mapping, symbols, relocations, and TLS
  • lib/glibc_shim.cpp — implemented glibc ABI adapters
  • lib/glibc_stubs.cpp — explicit fallbacks for the rest of the ABI

Use it as a library

作为库使用

The default target builds the standalone archive: ./build. The published ./dlfcn symlink points to the resulting libdlfcn.a. Include lib/dlfcn.h, link the archive into a musl-static application, and ordinary dlopen()/dlsym() calls are redirected to SoLo. The source tree is intentionally self-contained and suitable for copying into another static build graph. 默认目标会构建独立归档文件:./build。发布的 ./dlfcn 符号链接指向生成的 libdlfcn.a。包含 lib/dlfcn.h,将该归档文件链接到 musl 静态应用程序中,普通的 dlopen()/dlsym() 调用就会被重定向到 SoLo。该源码树被刻意设计为自包含,适合复制到其他静态构建图中。

Reproduce the experiment

复现实验

./build test # load an Arch glibc DSO closure in the smoke test
./build vulkan_test # build the static demo and verify a native Lavapipe PNG

CI performs the native build and test on Alpine/musl with GCC, Fedora with GCC, and Ubuntu with Clang. The Vulkan test installs each distribution’s own Lavapipe package; it does not run the driver from an Arch sysroot. Every build input for the standalone Vulkan executable is vendored under bin/vulkan. build.py compiles those sources directly: upstream CMake, Meson, configure, and Make build systems are not invoked. CI 在 Alpine/musl(使用 GCC)、Fedora(使用 GCC)和 Ubuntu(使用 Clang)上执行原生构建和测试。Vulkan 测试会安装各发行版自带的 Lavapipe 软件包;它不会运行来自 Arch sysroot 的驱动程序。独立 Vulkan 可执行文件的所有构建输入都已 vendored 在 bin/vulkan 下。build.py 直接编译这些源码:不会调用上游的 CMake、Meson、configure 和 Make 构建系统。