Adventures with ZFS

As part of plan to improve my backup strategy, I was testing out different zfs configurations to see what might be the best option.

The Problem

I was having trouble testing out using native zfs encryption

• sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase test-pool/test-encrypt
• invalid property 'encryption'

Checking zfs version

So first I checked the zfs version:

• sudo modinfo zfs | grep version
• version:        0.8.3-1ubuntu12.15
• zpool version
• returned an error, which I found weird
• zpool upgrade -v
• returned a maximum version of 23
• expected a maximum version of 28
• Did some googling and found out that I may have the older zfs-fuse installed
• dpkg -s zfs-fuse 
• confirmed my suspicion and told me that I had 0.7.0 installed

Replace zfs-fuse with zfsutils-linux

• * Unmount all zfs datasets*
• I didn't do this step, but you definitely should to prevent data errors
• sudo zfs unmount <zpool>/<dataset>
• sudo apt remove zfs-fuse
• sudo apt install zfsutils-linux
• Here I rebooted the system
• sudo zpool status
• no zpools available
• sudo zpool import -a
• listed my zpool with its last use
• sudo zpool import -f <zpool>
• sudo zpool status
• now correctly showed my zpool

Test setup

• I am using 2 disks that have a raw speed of 150 MB/s (benchmarked using dd on a single disk)
• Creating the test pool
• sudo zpool create test-pool mirror /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number> /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number>
• or
• sudo zpool create test-pool -o ashift=9 mirror /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number> /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number>
• Creating test dataset
• sudo zfs create test-pool/test
• sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase test-pool/test-encrypt
• Disable caching
• sudo zfs set primarycache=none test-pool/test
• sudo zfs set secondarycache=none test-pool/test
• 10 back-to-back copies of a 5GB file and waiting for the numbers to stabilize
• sudo rsync --progress Downloads/Win11_22H2_English_x64v1.iso /test-pool/test/
• Deleting the zpool when done testing
• sudo zpool destroy test-pool

Results of the upgrade and test

The good news is that I am seeing a 2-3X speed up on my existing zpool setup with ashift=9 from doing the upgrade. It also appears that native encryption has a minimal impact (as long as your processor has AES-NI). Here are the numbers that I was seeing

• zfs-fuse 0.7.0:
• ashift=9, no encryption: 35-45 MB/s
• ashift=12, no encryption:  70-72 MB/s
• zfsutils-linux 0.8.3:
• ashift=9, no encryption: 108-117 MB/s
• ashift=9, native encryption: 105-117 MB/s
• ashift=12, no encryption: 134-141 MB/s
• ashift=12, native encryption: 133-139 MB/s

Appendix

Article on setting up native zfs encryption:

• https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/

checking zfs version:

• https://www.reddit.com/r/zfs/comments/8bxxhu/how_do_i_check_which_version_of_zfs_i_have/
• https://askubuntu.com/questions/423355/how-do-i-check-if-a-package-is-installed-on-my-server

zfs-fuse being super outdated:

• https://github.com/openzfs/zfs/issues/10329
• https://github.com/pscedu/zfs-fuse

Importing missing zpool:

• https://serverfault.com/questions/585228/zfs-import-unable-to-find-any-pools

More about ashift:

• https://jro.io/truenas/openzfs/#tuning
• https://louwrentius.com/zfs-performance-and-capacity-impact-of-ashift9-on-4k-sector-drives.html

rsync stats:

• https://unix.stackexchange.com/questions/65077/is-it-possible-to-see-cp-speed-and-percent-copied