Setting up ZFS with Arch root install

Posted on Fri 10 March 2017 in Linux

Configuration overview

The last setup of this machine was an UNRAID install with virtual machines using PCI passthrough. For this setup I am going to run Arch Linux with a root install on ZFS. The root install will allow snapshots of the entire operating system.

Array hardware

Boot Drive - 32 GB USB flash drive
Array Storage - 3x 3TB HDD
Cache - 2x 128GB SSD

Final configuration

The completed setup will have the following zpool configuration and mount points

$ zpool status

  pool: vault
 state: ONLINE
  scan: none requested
config:

    NAME                                             STATE     READ WRITE CKSUM
    vault                                            ONLINE       0     0     0
      mirror-0                                       ONLINE       0     0     0
        ata-ST3000DM001-1CH144_Z1F5W372              ONLINE       0     0     0
        ata-ST3000DM001-1CH144_Z1F5YJ5C              ONLINE       0     0     0
        ata-ST3000DM001-1C6144_Z1F5KYV4              ONLINE       0     0     0
    cache
      ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A  ONLINE       0     0     0
      ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B  ONLINE       0     0     0

errors: No known data errors

$ zfs mount

vault/ROOT/default              /
vault/home                      /home
vault                           /vault

Create EFI partition on boot drive

Boot into archlive in UEFI mode and create a 512MB EFI partition on the USB drive

# enter into fdisk for the required device
$ fdisk /dev/sdX
$ g # create a new GPT partition table
$ n # create a new partition
$ enter # select partition 1 as default
$ enter # select default start location
$ +512M # end location of 512M after start location

$ t # change the partition type to EFI
$ 0 # in fdisk the EFI system partition is 0
$ w # write changes to disk

Create and configure the pool

You do not need to partition the disks that will be used in the pool as zfs will partition the drives automatically when the pool is created. Use disk id to specify the disks in the pool. You can find disk ids with the command ls /dev/disk/by-id. This install is going to use a 3 way mirror with the SSDs acting as the cache.

Load the kernel module

$ modprobe zfs

Note the -o ashift=12 will use 4096 byte sectors which is what you want for modern disks.

$ zpool create -f -o ashift=12 vault mirror \
ata-ST3000DM001-1CH144_Z1F5W372 \
ata-ST3000DM001-1CH144_Z1F5YJ5C \
ata-ST3000DM001-1C6144_Z1F5KYV4 \
cache \
ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A \
ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B

Check the status of the pool to make sure it was created correctly.

$ zpool status

  pool: vault
 state: ONLINE
  scan: none requested
config:

    NAME                                             STATE     READ WRITE CKSUM
    vault                                            ONLINE       0     0     0
      mirror-0                                       ONLINE       0     0     0
        ata-ST3000DM001-1CH144_Z1F5W372              ONLINE       0     0     0
        ata-ST3000DM001-1CH144_Z1F5YJ5C              ONLINE       0     0     0
        ata-ST3000DM001-1C6144_Z1F5KYV4              ONLINE       0     0     0
    cache
      ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A  ONLINE       0     0     0
      ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B  ONLINE       0     0     0

errors: No known data errors

Turn on compression and optimize writes to disk with relatime

#turn on compression
$ zfs set compression=on vault
# optimize writes to disk
$ zfs set relatime=on vault

Create datasets for / and home

$ zfs create -o mountpoint=/ vault/ROOT/default
$ zfs create -o mountpoint=/home vault/home

Specify the dataset that will be used to boot from

$ zpool set bootfs=vault/ROOT/default vault

Unmount zfs volumes and export before installing arch

$ zfs umount -a
$ zpool export vault

Install the base Arch system

Create directories for mountpoints

$ mkdir /mnt/{boot,home}

Mount zfs and boot volumes

$ mount /dev/sdX1 /mnt/boot # boot disk
$ zpool import -d /dev/disk/by-id -R /mnt vault

$ pacstrap -i /mnt base base-devel

Copy the zpool.cache file to the arch install

$ cp /etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache

Generate the fstab and make sure /boot is mounted

$ genfstab -U -p /mnt >> /mnt/etc/fstab

Add hooks to mkinitcpio.conf and regenerate it

$ vim /mnt/etc/mkinitcpio.conf

# Add zfs after keyboard but before filesystems
HOOKS="base udev autodetect modconf block keyboard zfs filesystems"


# regenerate mkinitcpio
$ mkinitcpio -p linux

Chroot into the Arch install and configure the system

$ arch-chroot /mnt /bin/bash

Add the archzfs repo to /etc/pacman.conf

$ vim /etc/pacman.conf

# add the following in the repository section
[archzfs]
Server = http://archzfs.com/$repo/x86_64

Sign the repository key with the key from AUR

$ pacman-key -r 5E1ABF240EE7A126
$ pacman-key --lsign-key 5E1ABF240EE7A126

Update the system and install zfs

$ pacman -Syyu
$ pacman -S zfs-linux

Enable zfs services

$ systemctl enable zfs.target
$ systemctl enable zfs-import-cache.service

Install the EFI bootloader

bootctl --path=/boot install

Create an entry for Arch in the bootloader

$ vim /boot/loader/entries/arch.conf

# add the following
title     Arch Linux
linux     /vmlinuz-linux
initrd    /initramfs-linux.img
options   zfs=vault/ROOT/default rw

Exit chroot unmount drives and restart

# exit chroot before unmounting
$ umount /mnt/boot
$ zfs umount -a
$ zpool export vault
$ reboot

Apparently when the computer is booting the host id is not available to the system. To fix this, create a hostid file and regenerate mkinitcpio. After rebooting:

$ hostid > /etc/hostid
$ mkinitcpio -p linux

Previous Post Next Post