Arch Linux installed on ZFS

Posted on Fri 25 September 2020 in arch

ZFS is both a filesystem and a volume manager. If you have extra RAM in your system it will increase the read speed of your system dramatically by caching the system's most-used files in RAM.

How to use ZFS:

  1. The easiest way to use ZFS with Arch Linux is to install and use ZFS. Arch Linux is installed outside of the ZFS pool and is used to manage the disks.
  2. Install your operating system on top of ZFS to get all of the goodies that come with ZFS on your root install.


This guide will focus on #2 above: installing Arch Linux root on ZFS. There is a little bit of inception going with this setup. We're going to use the Arch Linux Live CD to create a ZFS pool and then we are going to install Arch Linux on that ZFS pool.

Let's do this!

To continue make sure that you have created an Arch Linux iso with ZFS installed. Check out Creating Arch Linux iso with ZFS installed to see how this is done.

The only detail about the hardware on the system that is relevant here is we are using two 120GB drives. The pool mirrored to allow for a fail-over in the case of a drive failure. Both of the drives will have a 550 MB partition for the boot loader. This only needs to be on one of the drives but just for the sake of keeping the size of the partitions the same, I added it to both disks. This boot partition is how the ZFS Pool will get mounted on startup.

Use sgdisk to partition the drives.

From here the guide assumes you are booted into the Arch Linux Live ISO

# Clear the disk and create 2 partitions on each disk
sgdisk --zap-all /dev/disk/by-id/<disk0>
sgdisk -n1:0:+550M -t1:ef00 /dev/disk/by-id/<disk0>
sgdisk -n2:0:0 -t2:bf00 /dev/disk/by-id/<disk0>

sgdisk --zap-all /dev/disk/by-id/<disk1>
sgdisk -n1:0:+550M -t1:ef00 /dev/disk/by-id/<disk1>
sgdisk -n2:0:0 -t2:bf00 /dev/disk/by-id/<disk1>

Format the boot partition

mkfs.vfat /dev/disk/by-id/<disk0>-part1

Configure the ZFS Pool

Here the pool is created and set up in a mirror configuration. ashift=12 is used in this configuration but this may be different in your use case. More information can be found at Performance and Tuning.

# Create the pool and configure as a mirror
zpool create -m none -O canmount=off -o ashift=12 zroot mirror /dev/disk/by-id/<disk0>-part2 /dev/disk/by-id/<disk1>-part2

# Create the container for the datasets
zfs create -o canmount=off -o mountpoint=none zroot/ROOT
# Create the root dataset
zfs create -o canmount=noauto -o mountpoint=/ zroot/ROOT/default

# Configure disk options and compression
zfs set compression=on zroot
zfs set atime=off zroot
zfs set xattr=sa zroot
zfs set acltype=posixacl zroot

Here are some additional datasets that you might add

# Create a container for other datasets
zfs create -o mountpoint=none zroot/data

# Create datasets and examples of how to use children datasets
zfs create -o mountpoint=/home zroot/data/home
zfs create -o mountpoint=/root zroot/data/home/root
zfs create -o mountpoint=/var -o canmount=off     zroot/var
zfs create                                        zroot/var/log
zfs create -o mountpoint=/var/lib -o canmount=off zroot/var/lib
zfs create                                        zroot/var/lib/libvirt
zfs create                                        zroot/var/lib/docker

Install Arch Linux on the ZPool

zfs umount -a
zpool export zroot
zpool import -d /dev/disk/by-id -R /mnt zroot

## Note: if you have an issue mounting the drive you may need to remove the contents of /mnt
zfs mount /zroot/ROOT/default
zpool set bootfs=zroot/ROOT/default zroot
mkdir /mnt/boot
mount /dev/disk/by-id/<disk0>-part1 /mnt/boot
mkdir /mnt/etc

genfstab -U /mnt >> /mnt/etc/fstab

pacstrap /mnt base base-devel linux linux-firmware vim-minimal

Add ZFS as a kernel module

Add zfs as a kernel module by editing /mnt/etc/mkinitcpio.conf

# /mnt/etc/mkinitcpio.conf
HOOKS=(base udev autodetect modconf block keyboard zfs usr filesystems shutdown)

Let's dive in!

Now that the base Arch OS is installed, we can chroot into the system.

arch-chroot /mnt

Add archzfs source

Install wget with pacman.

$ pacman -S wget

Add the archzfs repository source to /etc/pacman.conf

# /etc/pacman.conf

Server =$repo/x86_64

This was the most frustrating part about the install. The process that I used may change in the future so if the next step does not work for you check out the following link to get the correct keys and update the system.

Take a look at the archzfs page to get the correct keys to import and then update the system:

pacman-key -a archzfs.gpg
pacman-key -r F75D9D76
pacman-key --lsign-key F75D9D76

check the fingerprint and verify it matches the one on the archzfs page

pacman-key -f F75D9D76

Additional debugging for this step can be found in the archzfs github issues:

Configure the Arch Linux install

From here you can follow the Arch installation guide to configure your system to your liking. Make sure to install the linux-headers and zfs-dkms packages along with any other necessary packages for your configuration.

pacman -Syu linux-headers zfs-dkms openssh

The home stretch!

Create the zpool cache file

zpool set cachefile=/etc/zfs/zpool.cache zroot

install the boot loader

bootctl install

Create a file at /boot/loader/entries/arch.conf 

# */boot/loader/entries/arch.conf*

title Arch Linux
linux vmlinuz-linux
initrd amd-ucode.img
initrd initramfs-linux.img
options zfs=zroot/ROOT/default rw

Configure the bootloader to use the entry we just created by editing */boot/loader/loader.conf* and setting default arch .

# /boot/loader/loader.conf
default arch
timeout 3

Whew, we now have our setup built. Let's exit the chroot and reboot.

umount -R /mnt
zfs umount -a
zpool export zroot

Enable ZFS services and generate a hostid

systemctl enable
systemctl enable zfs-import-cache
systemctl enable zfs-mount
zgenhostid $(hostid)
mkinitcpio -p linux

Whew, we did it. Now we have Arch Linux installed and configured on our ZPool. Enjoy the benefits of ZFS! Use snapshots as a way to experiment and feel confident you can revert to a working state easily.

Additional Resources: