The Fedora 45 Sausage Factory

Fedora 45 “香肠工厂”

This is a walkthrough of how Fedora turns source code and packages into the artifacts you download and install. It follows a package from a packager’s git push to a composed release: ISOs, cloud images, container images, and OSTree deployments. 本文将带您了解 Fedora 是如何将源代码和软件包转化为您下载并安装的制品的。我们将追踪一个软件包从打包者推送 Git 代码到最终发布（包括 ISO、云镜像、容器镜像和 OSTree 部署）的全过程。

The walkthrough describes how the Fedora ‘sausage’ is created as of Fedora 45, things change all the time; I hope to have time to update this document every cycle or every few cycles of Fedora releases so there’s both history and people can find up to date information. 本文描述了截至 Fedora 45 版本时 Fedora “香肠”的制作过程。由于技术细节时刻在变，我希望能在每个或每几个 Fedora 发布周期更新本文，以便既能保留历史记录，又能让人们找到最新的信息。

In fact this document is currently still living as Fedora 45 has not been released yet so this post likely will get updated until the release. There’s a few change proposals in flight that change some of the bits and bobs here. Specifically how the boot.iso is produced which is a decently large part of this post. 事实上，由于 Fedora 45 尚未正式发布，本文档目前仍处于动态更新状态，因此在正式发布前，这篇文章很可能会持续更新。目前有几个正在进行中的变更提案可能会影响文中的部分细节，特别是关于 boot.iso 的制作方式，这在本文中占有相当大的篇幅。

Thus now is also the right time to let me know if I got anything wrong, please e-mail me at cmdr@supakeen.com if you find any errors. 因此，现在也是您指出文中错误的好时机，如果您发现任何问题，请发送电子邮件至 cmdr@supakeen.com 联系我。

Starting Line: dist-git

起跑线：dist-git

Things start with a packager pushing a commit to a package. 一切始于打包者向软件包推送一次提交。

Fedora stores the source definitions of every package in individual Git repositories at src.fedoraproject.org. Each repo contains an RPM spec file, any downstream patches, and a sources file that points to upstream tarballs stored in a separate lookaside cache. The large binary files stay out of Git; the other files exist inside of Git and thus have full version control. Fedora 将每个软件包的源代码定义存储在 src.fedoraproject.org 的独立 Git 仓库中。每个仓库包含一个 RPM spec 文件、下游补丁以及一个指向存储在独立“旁路缓存”（lookaside cache）中上游压缩包的 sources 文件。大型二进制文件不会存入 Git，而其他文件则存在于 Git 中，从而实现了完整的版本控制。

Packagers usually interact with these repos through fedpkg, a CLI that wraps the common operations: cloning repos, uploading source tarballs, submitting builds, and creating updates. The important thing about fedpkg build is what it actually does: it constructs a URL pointing at a specific commit in the Git repo and hands that to Koji, the build system. The build is fully reproducible from that commit hash. 打包者通常通过 fedpkg 与这些仓库交互，这是一个封装了常见操作（如克隆仓库、上传源码包、提交构建和创建更新）的命令行工具。fedpkg build 的关键之处在于其实际操作：它会构建一个指向 Git 仓库中特定提交的 URL，并将其交给构建系统 Koji。基于该提交哈希值，构建过程是完全可复现的。

Branches map to releases: rawhide for development, f44 for Fedora 44. The hosting is split between Pagure (src.fedoraproject.org) and Forgejo (forge.fedoraproject.org), with an ongoing migration from the former to the latter. 分支对应着不同的发布版本：rawhide 用于开发，f44 用于 Fedora 44。托管服务目前分散在 Pagure (src.fedoraproject.org) 和 Forgejo (forge.fedoraproject.org) 之间，且正在从前者向后者迁移。

Packagers can do work without using fedpkg, but for the purposes of this post it’s easier to assume fedpkg is in use since it abstracts away some of the bits that are irrelevant to the reader. 打包者也可以不使用 fedpkg 进行工作，但为了方便本文叙述，我们假设使用了 fedpkg，因为它屏蔽了一些与读者无关的细节。

Building Packages: Koji

构建软件包：Koji

When fedpkg build submits that Git URL, Koji takes over. Koji is Fedora’s build system. It has been around since Fedora 7, and it builds essentially everything. 当 fedpkg build 提交那个 Git URL 后，Koji 便接管了任务。Koji 是 Fedora 的构建系统，自 Fedora 7 以来一直沿用至今，它负责构建几乎所有的东西。

Koji follows a hub-and-spoke architecture. The hub is a passive XML-RPC server that sits in front of a PostgreSQL database. Builder daemons poll the hub for work, create a fresh Mock chroot environment for each build, run the build, and upload the results. Every build starts from a clean room. You can never get a different result because someone installed something on the builder last week. Koji 采用“中心辐射型”（hub-and-spoke）架构。中心是一个位于 PostgreSQL 数据库前端的被动式 XML-RPC 服务器。构建守护进程会轮询中心以获取任务，为每次构建创建一个全新的 Mock chroot 环境，运行构建，并上传结果。每次构建都从“洁净室”开始，你绝不会因为有人上周在构建机上安装了什么东西而得到不同的结果。

The organizational model is built around tags. A tag is a named collection of builds. Build targets map an incoming build request to two tags: a build tag (which defines the buildroot, the packages available during the build) and a destination tag (where the finished build lands). Tags support multiple inheritance, so you can layer a Fedora 44 tag on top of a base tag without duplicating everything. 其组织模型围绕“标签”（tags）构建。标签是构建的命名集合。构建目标将传入的构建请求映射到两个标签：构建标签（定义构建根目录，即构建期间可用的软件包）和目标标签（完成的构建存放的位置）。标签支持多重继承，因此你可以在基础标签之上叠加 Fedora 44 标签，而无需重复所有内容。

Koji doesn’t just build RPMs. Through its plugin system and content generators, it also orchestrates image builds: Kiwi images via the kiwiBuild task type, Image Builder artifacts via imageBuilderBuild, and OSTree composes via runroot tasks. We’ll get to those. Koji 不仅仅构建 RPM。通过其插件系统和内容生成器，它还协调镜像构建：通过 kiwiBuild 任务类型构建 Kiwi 镜像，通过 imageBuilderBuild 构建 Image Builder 制品，以及通过 runroot 任务构建 OSTree。我们稍后会详细介绍这些。

Gating Updates: Bodhi

更新门控：Bodhi

A fresh RPM build sitting in Koji doesn’t automatically reach users. For branched releases (anything that isn’t Rawhide), it goes through Bodhi, Fedora’s update management system. 存放在 Koji 中的新鲜 RPM 构建并不会自动推送给用户。对于分支版本（非 Rawhide 的任何版本），它必须经过 Fedora 的更新管理系统 Bodhi。

Bodhi gates the release of updates through a feedback and testing cycle. A packager submits an update containing one or more builds. That update goes through a sequence of states: pending, testing, stable. Users and automated tests provide karma (+1 / -1). When an update hits +3 karma or spends enough days in testing, it will be automatically pushed to stable. If an update hits +1 karma the maintainer can manually push to stable (+2 for critical path). Bodhi 通过反馈和测试周期来控制更新的发布。打包者提交包含一个或多个构建的更新。该更新会经历一系列状态：待定（pending）、测试（testing）、稳定（stable）。用户和自动化测试提供 Karma 值（+1 / -1）。当更新达到 +3 Karma 或在测试中停留足够天数后，它会自动推送到稳定版。如果更新达到 +1 Karma，维护者可以手动将其推送到稳定版（关键路径更新则需要 +2）。

Behind the scenes, Bodhi manages all of this through Koji tags. When an update moves from testing to stable, Bodhi moves the build from the f44-updates-testing tag to the f44-updates tag. It then invokes Pungi to compose the update repository, the actual yum/dnf repo that users pull from when they run dnf upgrade. 在幕后，Bodhi 通过 Koji 标签管理这一切。当更新从测试状态转为稳定状态时，Bodhi 会将构建从 f44-updates-testing 标签移动到 f44-updates 标签。随后，它会调用 Pungi 来合成更新仓库，即用户运行 dnf upgrade 时所拉取的实际 yum/dnf 仓库。

Critical path packages, the ones your system needs to boot and function, get stricter requirements: 14 days in testing instead of 7, and more karma needed. Bodhi also integrates with Greenwave and ResultsDB for CI test gating, so automated test failures can block an update from reaching stable. 关键路径软件包（系统启动和运行所必需的包）有更严格的要求：在测试中停留 14 天而非 7 天，且需要更多的 Karma 值。Bodhi 还集成了 Greenwave 和 ResultsDB 用于 CI 测试门控，因此自动化测试失败可以阻止更新进入稳定版。

For Rawhide, these processes are configured differently. Non-critical path packages go stable immediately unless there’s a gating policy (either because the package is in the critical path or because the package itself has one) it still applies and an update can be held back until things pass. 对于 Rawhide，这些流程的配置有所不同。非关键路径软件包会立即进入稳定版，除非存在门控策略（因为软件包处于关键路径或软件包自身有门控要求），在这种情况下，更新会被搁置直到通过测试。

Composing a Release: Pungi

合成发布：Pungi

Individual RPMs, even with Bodhi’s gating, are just packages. Turning them into something you can download and install, an ISO, a cloud image, a repository, is the job of Pungi. 单个 RPM 即使经过了 Bodhi 的门控，也仅仅是软件包。将它们转化为你可以下载和安装的制品（如 ISO、云镜像、仓库），这就是 Pungi 的工作。

Pungi is the compose orchestrator. It doesn’t do much of the heavy lifting itself. Instead, it coordinates the tools that do, ensuring everything is built from the same consistent set of packages. Pungi 是合成编排器。它本身并不承担繁重的任务，而是协调执行这些任务的工具，确保所有内容都基于同一套一致的软件包构建。

The name is a reference to the pungi, a snake-charming instrument. It “charms” Anaconda, Fedora’s installer. The pun has survived since 2006. 这个名字源自“pungi”，一种弄蛇人使用的乐器。它负责“诱导”（charm）Fedora 的安装程序 Anaconda。这个双关语自 2006 年以来一直沿用至今。

A compose starts when pungi-koji runs, either triggered by cron for nightly Rawhide composes, or manually for milestone releases. It loads a configuration file (for Fedora, that’s fedora.conf in the pungi-fedora repository) and runs through a sequence of phases. 合成过程始于 pungi-koji 运行，无论是通过 cron 触发的 Rawhide 每日合成，还是里程碑版本的手动触发。它会加载配置文件（对于 Fedora，即 pungi-fedora 仓库中的 fedora.conf）并按顺序执行各个阶段。

Freezing the Package Set 冻结软件包集

The first real work Pungi does is snapshot the set of packages from a Koji tag. This is the Pkgset phase. Pungi 执行的第一个实际工作是从 Koji 标签中对软件包集进行快照。这就是 Pkgset 阶段。