Migrating to root on ZFS in a nutshell

^2022-01-20

There are plenty of guides on how to set up root on ZFS but they are all long-winded and none seem to explain how to migrate an existing system.

When using ZFS as our root filesystem we have to ensure that:

The hardware can find the bootloader.
The bootloader (e.g. GRUB) can read the ZFS filesystem.

1. Creating the pool

Obviously, we first need to create the pool where we will put our system on:

mkdir /mnt/rpool
zpool create rpool /dev/sdX -O compression=lz4 -R /mnt/rpool

Create any datasets now. I don’t know if this is actually required but I like having a separation between “user” and “system” data:

# mountpoint=/ isn't strictly necessary but is useful
zfs create -o mountpoint=/ -o canmount=noauto rpool/system
zfs create -o mountpoint=/home rpool/user

Set the system dataset as the bootfs so the bootloader knows which set to mount at the root:

zpool set bootfs=rpool/system rpool

Unmount user and then mount system and user so system is mounted first:

zfs unmount rpool/user
zfs mount rpool/system
zfs mount rpool/user

Now is a good time to check if the mountpoints are correct:

$ df -h
...
rpool/system          899G  128K  899G   1% /mnt/rpool
rpool/user            899G  128K  899G   1% /mnt/rpool/home

2. Copy data

Copy all data on your system to the pool. Be careful not to include any special filesystems though!

rsync -rav \
    /bin \
    /boot \
    /etc \
    /home \
    /initrd.img \
    /initrd.img.old \
    /lib \
    /lib32 \
    /lib64 \
    /libx32 \
    /opt \
    /root \
    /sbin \
    /usr \
    /var \
    /vmlinuz \
    /vmlinuz.old \
    /mnt/rpool
(cd /mnt/rpool && mkdir tmp dev proc sys run mnt)

In the meantime, grab a coffee.

3. Configure boot loader

We need to make sure the boot loader can actually read the ZFS partition.

First, chroot to the pool:

mount --bind /dev /mnt/rpool/dev
mount --bind /proc /mnt/rpool/proc
mount --bind /sys /mnt/rpool/sys
chroot /mnt/rpool

If you’re using Debian, install zfs-initramfs first:

apt install zfs-initramfs

Install GRUB (for UEFI systems!):

mkdosfs -F 32 -s 1 -n EFI /dev/sdX9
mkdir /boot/efi
echo UUID=`blkid -s UUID -o value /dev/sdX9` /boot/efi vfat defaults 0 0 >> /etc/fstab
mount /boot/efi
apt reinstall grub-efi-amd64 shim-signed

Check if grub recognizes the boot filesystem:

$ grub-probe /boot
zfs

Edit GRUB configuration (`/etc/default/grub´):

GRUB_CMDLINE="root=ZFS=rpool/system"

Update initrd and GRUB:

update-initramfs -c -k all
update-grub

Install GRUB to ESP:

grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=debian --recheck --no-floppy

4. Reboot

The pool should now be bootable. Reboot and run df -h:

...
rpool/system          788G   13G  775G   2% /
...
/dev/sdX9             7.9M  3.4M  4.5M  43% /boot/efi
rpool/user            887G  112G  775G  13% /home
...

5. Resources + more things to do

OpenZFS documentation

Now that ZFS is installed, you can make snapshots & backups of the entire system with Sanoid / Syncoid.