Root on ZFS

I’ve been using ZFS for data partitions for several years now. ZFS has a lot going for it. You can read all about ZFS’s features here. The key features I use it for include snapshots (made fast and “cheap” by way of being a copy-on-write (COW) FS), and block level checksumming. The former makes backups easy and consistent. The latter feature means I can be sure my data doesn’t die from bit rot and has had the happy side effect of giving early indications of pending disk failure.

ZFS on Linus (ZoL) has come a long way over the years since I started using it. Recently I’ve seen a lot of people use ZFS as their root filesystem on Linux. The stability of this feature, and the general benefits of ZFS, have finally pushed me over the edge into upgrading my home server to use ZFS as its root filesystem.

Unfortunately there is no standard installer support for installing Linux (Ubuntu is my preferred distribution) with a ZFS root filesystem. Fortunately there are some really great instructions out there. I used these instructions for my install, and barely deviated from them.

Here’s the [short] list of those deviations:

• I created a dedicated swap partition on the root device in addition to the EFI, ZFS boot pool, and ZFS root pool partitions. This is because I did not want to have to deal with this issue. That resulted in this partitioning scheme for my root device:

  Device           Start      End  Sectors  Size Type
  /dev/nvme0n1p1     256   131327   131072  512M EFI System
  /dev/nvme0n1p2  131328   393471   262144    1G Solaris /usr & Apple ZFS
  /dev/nvme0n1p3  393472  4587775  4194304   16G Linux swap
  /dev/nvme0n1p4 4587776 62514768 57926993  221G Solaris /usr & Apple ZFS
  

• I added an rpool/docker dataset (since added as an option to the instructions). As a heavy docker user, I’m looking forward to experimenting with docker’s ZFS storage driver.
• I chose to create /tmp on a tmpfs partition rather than a ZFS dataset.
• I used slightly different GRUB options, though these were entirely personal preference with no real impact on the final result.
• I configured an encrypted swap partition; this turned out to be a bit of an adventure as I had issues with systemd’s early crypto disk setup.

The install, all in all, went surprisingly smoothly. The only issue I encountered before getting my system entirely up and running was that my data zpool wasn’t being imported on startup. There was a lot of conflicting information about this problem online, and it’s apparently an issue that has plagued ZoL for some time. The solution I found worked for me was to create the file /etc/modprobe.d/zfs.conf with the contents¹:

options zfs zfs_autoimport_disable=0

Then a quick run of update-initramfs -u -k all to include that setting when loading the ZFS module during init. It took me a while to discover this solution, as similar problem are seen when the settings in /etc/default/zfs are misconfigured or when there are problems with the zpool cache (located at /etc/zfs/zpool.cache). I am not clear why autoimport is not enabled by default.

In the end though, I have ZFS running on my root filesystem. A quick installation of zfs-auto-snapshot and installing my custom backup script into cron, and “new” server is ready to go.