ZFS Backups

Here is the process that I use to create local zfs backups.

Manual Backup Procedure

1. Create Snapshots
• zfs snapshot -r storage/containers@backup-20240115
• -r: for recursive
2. Create a RaidZ1 Pool as our backup target
• zpool create backup-pool raidz1 [DISK1] [DISK2] [DISK3] [DISK4]
3. Send the initial snapshot
• zfs send -R --raw storage/containers@backup-20240115 | zfs receive -o readonly=on backup-pool/containers
• -R: for recursive
• --raw: so that encrypted data is not decrypted (not encrypted data will remain not encrypted)
• -o readonly=on: so that the backups are not editable
4. Send incremental snapshots
• zfs send -R --raw -I storage/containers@backup-20240115 storage/containers@backup-20240210 | zfs receive backup-pool/containers
• -I [prev-snapshot]: incremental data since specified snapshot

Automating Backup Procedure

1. Created the below scripts to help automate in /root/backup_scripts/
• snapshot.sh
• snapshot-all.sh
• backup.sh
• backup-all.sh
2. Set the snapshot-all.sh script on crontab to run monthly
• crontab -e
• 05 8 1 * * sh /root/backup_scripts/snapshot-all.sh
• remember that the cron time is UTC
3. Manually push the backups
• nohup sh backup-all.sh >>all.log 2>&1 &

Scripts

snapshot.sh

#!/bin/sh

DATASET=$1

zfs snapshot -r ${DATASET}@backup-$(date "+%Y%m%d")

snapshot-all.sh

#!/bin/sh

sh /root/backup_scripts/snapshot.sh rpool
sh /root/backup_scripts/snapshot.sh storage

backup.sh

#!/bin/sh

if [ $# -ne 2 ] ; then

  echo "Requires 2 args source and destination"

  exit 1

fi

src=$1

dest=$2

snaplist=$(zfs list -t snapshot $src)

if [ $? -ne 0 ] ; then

  echo "No snapshots found for $src"

  exit 1

fi

bkpsnap=$(zfs list -t snapshot $dest 2>&1)

incremental=$?

if [ $incremental -eq 0 ] ; then

  incsnap=$(echo "$bkpsnap" | tail -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  snap=$(echo "$snaplist" | grep "$incsnap" -A 1 | tail -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  if [ "$incsnap" = "$snap" ] ; then

    echo "Backup is already up to date"

    exit 1

  fi

  echo "Sending incremental snap $src@$snap on top of $dest@$incsnap"

  echo "zfs send -R --raw -I $src@$incsnap $src@$snap | zfs receive $dest"

  zfs send -R --raw -I $src@$incsnap $src@$snap | zfs receive $dest

else

  snap=$(echo "$snaplist" | tail -n +2 | head -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  echo "Sending initial snap $src@$snap to $dest"

  echo "zfs send -R --raw $src@$snap | zfs receive -o readonly=on $dest"

  zfs send -R --raw $src@$snap | zfs receive -o readonly=on $dest

fi

echo "Complete\n"

backup-all.sh

#!/bin/sh

echo "========================================"

echo "Backup on $(date +"%Y-%m-%d %H:%M:%S")"

echo ""

zpool import bpool

sh /root/backup_scripts/backup.sh rpool/data backup-pool/data

sh /root/backup_scripts/backup.sh storage/containers backup-pool/containers

sh /root/backup_scripts/backup.sh storage/encrypted backup-pool/encrypted

sh /root/backup_scripts/backup.sh storage/lists backup-pool/lists

echo "Complete at $(date +"%Y-%m-%d %h:%M:%S")"

echo "Performing scrub"

zpool scrub -w backup-pool

echo "Complete at $(date +"%Y-%m-%d %h:%M:%S")"

zpool status backup-pool

zpool export backup-pool

echo "Spinning down hard drives"

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFPX-68C5ZN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFPX-68C5ZN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX

echo "Hard drives spun down"

echo "========================================"

Appendix

Sources

• https://www.jeffgeerling.com/blog/2021/htgwa-create-zfs-raidz1-zpool-on-raspberry-pi
• https://forum.proxmox.com/threads/solved-help-needed-on-a-script-for-automating-zfs-snapshots-to-removable-media.81784/
• https://mtlynch.io/zfs-encrypted-backups/
• https://unix.stackexchange.com/questions/263677/how-to-one-way-mirror-an-entire-zfs-pool-to-another-zfs-pool
• https://www.reddit.com/r/zfs/comments/1c3vnkm/cannot_receive_incremental_stream_destination_has/
• https://openzfs.github.io/openzfs-docs/man/master/8/zfs-recv.8.html
• https://docs.oracle.com/cd/E36784_01/html/E36835/gnheq.html
• https://openzfs.github.io/openzfs-docs/man/master/8/zpool-scrub.8.html