Diskless Linux boot using ZFS, iSCSI and PXE

使用 ZFS、iSCSI 和 PXE 实现无盘 Linux 启动

Motivation

动机

I wanted to test out the new Unsloth models for Qwen3.6 and Gemma4 on my gaming PC. llama.cpp on Windows is tedious to compile, and I have littered my Windows installation with too many toolchains already. Python venvs, Mingw, Cuda, UCRT64 & WSL to name a few. Windows still does not feel developer friendly to me. I think I’m ok with it being a frontend for Steam’s Big Picture mode. I didn’t want to disturb my Windows setup that I use for gaming. Windows has a nasty habit of breaking GRUB on updates. UEFI fixes that to some extent, but it’s a pain to maintain the UEFI entries manually and change them every time the kernel updates. One of the best benefits of using the method described here is that GRUB is also on the remote drive. I have a couple of NVME drives in the PC, both contain a few games that I play frequently. I didn’t want to get into the hassle of repartitioning everything that the boot loader works with both Linux & Windows. Sure I can use a USB drive and in the past I have done so, but I tend to misplace my USB drives everywhere and when I urgently need one, I tend to pick the USB that’s readily available e.g. for some FedEx printing or as backup drive for photos when on vacation. I end up wiping the Linux USBs more often than not. I already have a NAS, so why not use remote boot ? I always wanted to know how PXE worked over iSCSI.

我想在我的游戏 PC 上测试 Qwen3.6 和 Gemma4 的新 Unsloth 模型。在 Windows 上编译 llama.cpp 非常繁琐，而且我的 Windows 系统里已经堆满了各种工具链，比如 Python venvs、Mingw、Cuda、UCRT64 和 WSL 等等。对我来说，Windows 依然不够开发者友好。我觉得它作为 Steam 大屏幕模式（Big Picture mode）的前端还凑合，但我不想破坏我用于玩游戏的 Windows 环境。Windows 有个坏习惯，更新时经常会破坏 GRUB。虽然 UEFI 在一定程度上解决了这个问题，但手动维护 UEFI 条目并在每次内核更新时进行更改非常麻烦。使用本文所述方法的一大好处是，GRUB 也存放在远程驱动器上。我的电脑里有几块 NVMe 硬盘，里面存着我常玩的游戏。我不想为了让引导加载程序同时兼容 Linux 和 Windows 而去折腾重新分区。当然，我也可以使用 USB 驱动器，过去我也确实这么做过，但我总是随手乱放 USB，急用时往往会顺手抓起手边现成的那个（比如用来打印快递单，或者度假时备份照片），结果就是 Linux 启动盘经常被我格式化掉。既然我已经有了 NAS，为什么不试试远程启动呢？而且我一直想了解 PXE 是如何通过 iSCSI 工作的。

Limitations

局限性

Installing Debian on a network drive will indeed be noticeably slower than a native install. Since I’m going to use some portion of my local NVMe drive to store & load the models, I didn’t really care about the OS performance as I have enough RAM to run everything smoothly once the OS has booted up. I won’t be using this for browsing stuff using Firefox.

在网络驱动器上安装 Debian 的速度确实会比本地安装明显慢一些。由于我打算利用本地 NVMe 硬盘的一部分来存储和加载模型，我并不太在意操作系统的性能，因为一旦系统启动，我有足够的内存来流畅运行一切。我不会用这个系统来通过 Firefox 浏览网页。

Assumptions

前提条件

A single Debian 13 based server is used for Netboot.xyz, tftpd, iSCSI Target & ZFS ZVol. My Proxmox install works perfectly fine for this. I used my Asus Router with the Merlin firmware for DNSMasq. The post is broken down into the following sections: Install & Configure Netboot.xyz, Configure TFTP, Configure DNSMasq on Router, ZFS ZVol Creation, iSCSI Configuration, Install Debian.

使用一台基于 Debian 13 的服务器来运行 Netboot.xyz、tftpd、iSCSI Target 和 ZFS ZVol。我的 Proxmox 环境完全可以胜任。我使用刷了 Merlin 固件的华硕路由器来运行 DNSMasq。本文分为以下几个部分：安装与配置 Netboot.xyz、配置 TFTP、在路由器上配置 DNSMasq、创建 ZFS ZVol、配置 iSCSI、安装 Debian。

Install & configure Netboot.xyz

安装与配置 Netboot.xyz

I’m using my Proxmox host to export my iSCSI targets. Install the required packages: apt install apache2 git ansible tftpd-hpa targetcli-fb. Clone & compile netboot. One can use netboot directly without compiling, but then it downloads all the assets at runtime which, although handy, is not something that I would recommend.

我使用 Proxmox 主机来导出 iSCSI 目标。首先安装必要的软件包：apt install apache2 git ansible tftpd-hpa targetcli-fb。克隆并编译 netboot。虽然可以直接使用 netboot 而无需编译，但那样会在运行时下载所有资源，虽然方便，但我不推荐这样做。

cd /opt
git clone https://github.com/netbootxyz/netboot.xyz.git
cd netboot.xyz

We edit a few config files to tailor our netboot install. Edit /opt/netboot.xyz/user_overrides.yml with the below contents:

我们需要编辑几个配置文件来定制 netboot 安装。编辑 /opt/netboot.xyz/user_overrides.yml，填入以下内容：

generate_menus: true
generate_disks: true
generate_checksums: true
generate_local_vars: false
make_num_jobs: 1
site_name: 192.168.50.167
boot_domain: 192.168.50.167

Ensure site_name & boot_domain points to the netboot host. It is the same as the Proxmox host in my case. Now we fix up some netboot templates so we can boot our installer & iSCSI.

确保 site_name 和 boot_domain 指向 netboot 主机。在我的案例中，它与 Proxmox 主机是同一台。现在我们修改一些 netboot 模板，以便引导安装程序和 iSCSI。

Edit /opt/netboot.xyz/roles/netbootxyz/templates/menu/boot.cfg.j2 — find the :end section and change it to: 编辑 /opt/netboot.xyz/roles/netbootxyz/templates/menu/boot.cfg.j2，找到 :end 部分并修改为：

:end
chain local-vars.ipxe || exit

Edit /opt/netboot.xyz/roles/netbootxyz/templates/local-vars.ipxe.j2 and change it to: 编辑 /opt/netboot.xyz/roles/netbootxyz/templates/local-vars.ipxe.j2 并修改为：

#!ipxe
set custom_url http://192.168.50.167

Use ansible to install netbootxyz to /var/www/html. This can take a while… 使用 ansible 将 netbootxyz 安装到 /var/www/html。这可能需要一些时间……

ansible-playbook -i inventory site.yml

Now we need to add a custom menu to boot from our disks. If the disk does not have an OS, it will start the Debian installer. Create /var/www/html/debian13-iscsi.ipxe and change it as below. Make sure the IP addresses & IQNs are correct.

现在我们需要添加一个自定义菜单来从磁盘启动。如果磁盘没有操作系统，它将启动 Debian 安装程序。创建 /var/www/html/debian13-iscsi.ipxe 并按如下修改。请确保 IP 地址和 IQN 正确。

#!ipxe
set iscsi-server 192.168.50.167
set iscsi-target iqn.2026-05.xyz.716697.pve-vt:tank-debian-disk-12700k
set initiator-iqn iqn.2026-05.xyz.716697.pve-vt:12700k
set username myuser
set password mypassword
set reverse-username targetuser
set reverse-password targetpassword
sanboot iscsi:${iscsi-server}::::${iscsi-target} || goto installer

:installer
imgfree
kernel http://${iscsi-server}/assets/debian13/linux
initrd http://${iscsi-server}/assets/debian13/initrd.gz
imgargs linux root=/dev/ram0 initrd=initrd.gz vga=normal boot

Create the custom netboot.xyz entry. Create a new file /var/www/html/custom.ipxe: 创建自定义 netboot.xyz 条目。新建文件 /var/www/html/custom.ipxe：

#!ipxe
menu Local Custom Menu
item --gap Local iSCSI Installs:
item debian13-iscsi Debian 13 iSCSI Boot (192.168.50.167)
item --gap --
item back Back to main menu
choose menu || goto back
goto ${menu}

:debian13-iscsi
chain http://192.168.50.167/debian13-iscsi.ipxe || goto back

:back
chain http://192.168.50.167/menu.ipxe

Download the Debian initrd installer: 下载 Debian initrd 安装程序：

mkdir -p /var/www/html/assets/debian13
cd /var/www/html/assets/debian13
wget http://ftp.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/initrd.gz
wget http://ftp.debian.org/debian/dists/trixie/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux

Configure TFTP

配置 TFTP

Configure in /etc/default/tftpd-hpa: 在 /etc/default/tftpd-hpa 中进行配置：

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/srv/tftp"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure"

Copy the netboot.xyz binaries we compiled into tftp/ipxe: 将我们编译好的 netboot.xyz 二进制文件复制到 tftp/ipxe：

mkdir -p /srv/tftp/ipxe
cp /var/www/html/ipxe/netboot.xyz-undionly.kpxe /srv/tftp/ipxe/
cp /var/www/html/ipxe/netboot.xyz-snp.efi /srv/tftp/ipxe/
cp /var/www/html/ipxe/netboot.xyz.efi /srv/tftp/ipxe/
chown -R tftp:tftp /srv/tftp/ipxe
service tftpd-hpa restart