ZFS Backups

Here is the process that I use to create local zfs backups.

Manual Backup Procedure

1. Create Snapshots
• zfs snapshot -r storage/containers@backup-20240115
• -r: for recursive
2. Create a RaidZ1 Pool as our backup target
• zpool create backup-pool raidz1 [DISK1] [DISK2] [DISK3] [DISK4]
3. Send the initial snapshot
• zfs send -R --raw storage/containers@backup-20240115 | zfs receive -o readonly=on backup-pool/containers
• -R: for recursive
• --raw: so that encrypted data is not decrypted (not encrypted data will remain not encrypted)
• -o readonly=on: so that the backups are not editable
4. Send incremental snapshots
• zfs send -R --raw -I storage/containers@backup-20240115 storage/containers@backup-20240210 | zfs receive backup-pool/containers
• -I [prev-snapshot]: incremental data since specified snapshot

Automating Backup Procedure

1. Created the below scripts to help automate in /root/backup_scripts/
• snapshot.sh
• snapshot-all.sh
• backup.sh
• backup-all.sh
2. Set the snapshot-all.sh script on crontab to run monthly
• crontab -e
• 05 8 1 * * sh /root/backup_scripts/snapshot-all.sh
• remember that the cron time is UTC
3. Manually push the backups
• nohup sh backup-all.sh >>all.log 2>&1 &

Scripts

snapshot.sh

#!/bin/sh

DATASET=$1

zfs snapshot -r ${DATASET}@backup-$(date "+%Y%m%d")

snapshot-all.sh

#!/bin/sh

sh /root/backup_scripts/snapshot.sh rpool
sh /root/backup_scripts/snapshot.sh storage

backup.sh

#!/bin/sh

if [ $# -ne 2 ] ; then

  echo "Requires 2 args source and destination"

  exit 1

fi

src=$1

dest=$2

snaplist=$(zfs list -t snapshot $src)

if [ $? -ne 0 ] ; then

  echo "No snapshots found for $src"

  exit 1

fi

bkpsnap=$(zfs list -t snapshot $dest 2>&1)

incremental=$?

if [ $incremental -eq 0 ] ; then

  incsnap=$(echo "$bkpsnap" | tail -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  snap=$(echo "$snaplist" | grep "$incsnap" -A 1 | tail -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  if [ "$incsnap" = "$snap" ] ; then

    echo "Backup is already up to date"

    exit 1

  fi

  echo "Sending incremental snap $src@$snap on top of $dest@$incsnap"

  echo "zfs send -R --raw -I $src@$incsnap $src@$snap | zfs receive $dest"

  zfs send -R --raw -I $src@$incsnap $src@$snap | zfs receive $dest

else

  snap=$(echo "$snaplist" | tail -n +2 | head -n 1 | cut -f1 -d" " | cut -f2 -d"@")

  echo "Sending initial snap $src@$snap to $dest"

  echo "zfs send -R --raw $src@$snap | zfs receive -o readonly=on $dest"

  zfs send -R --raw $src@$snap | zfs receive -o readonly=on $dest

fi

echo "Complete\n"

backup-all.sh

#!/bin/sh

echo "========================================"

echo "Backup on $(date +"%Y-%m-%d %H:%M:%S")"

echo ""

zpool import bpool

sh /root/backup_scripts/backup.sh rpool/data backup-pool/data

sh /root/backup_scripts/backup.sh storage/containers backup-pool/containers

sh /root/backup_scripts/backup.sh storage/encrypted backup-pool/encrypted

sh /root/backup_scripts/backup.sh storage/lists backup-pool/lists

echo "Complete at $(date +"%Y-%m-%d %h:%M:%S")"

echo "Performing scrub"

zpool scrub -w backup-pool

echo "Complete at $(date +"%Y-%m-%d %h:%M:%S")"

zpool status backup-pool

zpool export backup-pool

echo "Spinning down hard drives"

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFPX-68C5ZN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFPX-68C5ZN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX

hdparm -y /dev/disk/by-id/ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX

echo "Hard drives spun down"

echo "========================================"

Appendix

Sources

• https://www.jeffgeerling.com/blog/2021/htgwa-create-zfs-raidz1-zpool-on-raspberry-pi
• https://forum.proxmox.com/threads/solved-help-needed-on-a-script-for-automating-zfs-snapshots-to-removable-media.81784/
• https://mtlynch.io/zfs-encrypted-backups/
• https://unix.stackexchange.com/questions/263677/how-to-one-way-mirror-an-entire-zfs-pool-to-another-zfs-pool
• https://www.reddit.com/r/zfs/comments/1c3vnkm/cannot_receive_incremental_stream_destination_has/
• https://openzfs.github.io/openzfs-docs/man/master/8/zfs-recv.8.html
• https://docs.oracle.com/cd/E36784_01/html/E36835/gnheq.html
• https://openzfs.github.io/openzfs-docs/man/master/8/zpool-scrub.8.html

GiGaPlus GP-S25-0802P Review

My Netgear JGS516PE PoE switch died. It was still providing PoE, however, it was not routing and all the lights were lit up amber. As a temporary solution, I was able to set up one AP on a PoE injector and plug in the Switch Flex Mini using USB-C. This allowed me to get the network up and running while looking for a replacement.

Since newer wireless access points are coming with 2.5Gbps port and requiring PoE+, I wanted the replacement switch to support those 2 things as well as preferring a rack mountable solution.

Options

• Unifi Pro Max 16 PoE
• https://store.ui.com/us/en/category/all-switching/products/usw-pro-max-16-poe
• $399 + $49 rack mount kit
• 2 x 10 Gbps SFP+
• 4 x 2.5 Gbps PoE++
• 12 x 1 Gbps PoE+
• PoE max: 180W
• Unifi Flex 2.5G PoE
• https://store.ui.com/us/en/category/all-switching/products/usw-flex-2-5g-8-poe
• $199 + $79 power adapter
• 1 x 10 Gbps SFP+/RJ45 combo port
• 8 x 2.5 Gbps PoE++
• PoE max: 196W
• Not rack mountable
• Sodola SL-SWTGW0108P-D
• https://www.amazon.com/dp/B0D5QFMZW3/?th=1
• $95.99 - $119.99
• 1 x 10 Gbps SFP+
• 1 x 2.5 Gbps PoE++
• 7 x 2.5 Gbps PoE+
• PoE max: 120W
• GiGaPlus GP-S25-0802P
• https://www.amazon.com/dp/B0DDB3LHPX?th=1
• $86.89 - $109.99
• 2 x 10 Gbps SFP+
• 8 x 2.5 Gbps PoE+
• PoE max: 130W
• Not managed
• Used Netgear JGS516PE
• $40 - $70 on eBay
• 8 x 1 Gbps PoE
• 8 x 1 Gbps Non-PoE
• PoE max: 85W

Decision

I went with the GiGaPlus because it wasn't much more expensive than the used Netgear and came with significant upgrades. It also was not managed so it has less of an attack vector and you don't have to worry about firmware updates from a smaller company.

My Experience

My original unit worked well and as expected until I tried switching the uplink to sfp+ using a 1 meter Sodola DAC cable. Then I started getting weird behavior like slack taking forever to load. I then tried speed tests and was not getting my expected speed and having large swings in speed during the test. I thought it might have to do with flow control, so I tried switching that on, but it did not help. I then tested the other sfp+ and these issues all went away.

Armed with this information, I contacted support. They asked me to test with iperf instead of an external speed test. I did this and sent them the results which were even worse than the external speed test. They then confirmed it was defective and asked if I wanted my money back. I then replied asking if they wanted their unit back and if they would provide a shipping label. I never received a response. I waited a few days and then went through Amazon returns as I was still in the 30 day return window.

Review

Pros:

• Low cost
• Provides everything I need now and future expandability
• 2 x 10 Gbps SFP+ ports
• One for Uplink
• One to Daisy Chain or connect to server
• Plug and play with no configuration necessary
• Forwards VLAN tagged traffic (No ability to add/modify tags though)

Cons:

• No management interface
• cannot see how close to PoE budget max you are
• no ability to cut power to a port without unplugging
• LED Status Lights are rather lacking
• Power LED and Ports 9 and 10 activity are on the far right
• No light to indicate which ports are supplying PoE
• Green light indicates 2.5 Gbps
• Orange light indicates 1 Gbps / 100 Mbps / 10 Mbps
• No light indicates no connection
• The original unit only had one functional sfp+ port, the replacement works perfectly

Other Thoughts:

• Support is only via email and only responded once a day to my emails likely due to the time difference
• I would make sure that everything works in the 30 day Amazon return period so that you can go through Amazon returns instead of support.

Appendix

Research

• Non-PoE version: https://www.servethehome.com/gigaplus-gp-s25-0802-8-port-2-5gbe-and-2-port-10g-review/
• https://www.servethehome.com/gigaplus-gp-s25-0802p-poe-8-port-2-5gbe-and-2-port-10g-review/
• https://www.servethehome.com/nicgiga-s25-0802p-8-port-2-5gbe-and-2-port-10g-poe-switch-review/

Flow Control

• https://www.reddit.com/r/Ubiquiti/comments/1iclm8l/sfp_to_rj45_adapter_causing_traffic_dropouts_to/

Replacing HDHomeRun tuner

Since my HDHomeRun Connect died in a lightning storm possibly due to static build-up. I bought a replacement HDHomeRun Flex Duo as well as a lightning arrester/suppressor with a ground.

Here are the ones that I went with:

• $109.99 HDHomeRun Flex Duo: Link
• $17.95 Proxicast Coaxial Lightning Arrester/Suppressor: Link

I had considered the 4 tuner with ATSC 3.0 support, but with the threat of the channels becoming encrypted and the increased price made me go with the duo.

Device Setup

• Add the lightning arrester/suppressor between the antenna and TV tuner and attach a ground wire
• Plug in antenna, ethernet, and power into the HDHomeRun
• Assign the HDHomeRun a static IP in your router
• Reboot the HDHomeRun
• Update the firmware: http://hdhomerun.local

Preparing Docker

setup.yml should be a copy of the mythtv docker-compose.yml, but update the image to dheaps/mythbackend:setup and add VNC_PASS=<super_secret_pass> to the environment section. Some key portions are:

• network_mode: host
• otherwise mythtv-setup will not be able to detect your HDHomeRun
• hostname: <your_hostname>
• otherwise mythtv-setup will use whatever generated hostname docker feeds it
• environment variable VNC_PASS
• the password to connect to VNC

Starting Docker

First and most importantly make sure your mythbackend is NOT running. I did this with:

• docker compose down
• docker ps

Next is starting the docker containers for mythtv-setup:

• docker compose --file setup.yml up

Running Setup

Connect vnc to <ip_address>:5900. Then you will get prompted with several questions.

• Add user to mythtv group? NO
• Ignore the error and continue? NO
• Would you like to start the mythtv backend? NO
• Would you like to run mythfilldatabase? NO

After answering the questions,  I got dumped to the terninal.

On the command line run:

• mythtv-setup.real

When done with setup exit with ESC and make sure to save changes if prompted. Then close vnc and press ctrl-C on the terninal running the docker container.

Appendix

Troubleshooting

• mythweb: Unable to connect to 127.0.0.1:6543
• in setup make sure that under General -> IP v4 address is set to the correct IP

Sources

• https://info.hdhomerun.com/info/update#updating_from_the_web_interface
• https://hub.docker.com/r/dheaps/mythbackend
• https://mizar.mythtv.org/wiki/Configuring_MythTV
• https://mizar.mythtv.org/wiki/Setup_General

Kodi and Jellyfin Intro Skipper

 I wanted to be able to skip the longer intro sequences like you can on many streaming services. In my search, I found a plugin and addon to accomplish the task.

Installation

• Add Intro-Skip Plugin to Jellyfin
• Add the Repository
• Jellyfin Web Interface -> Hamburger menu -> Dashboard
• Click on "Catalog" under Plugins
• Click on the settings wheel at the top
• Click on the "+"
• https://intro-skipper.org/manifest.json
• Install the Addon
• On the Catalog page scroll down to the Intro-Skipper section and install "Intro Skipper"
• Restart Jellyfin to finish the installation
• Add the Jellyskip Addon to Kodi
• Download https://github.com/SgtJalau/service.jellyskip/archive/refs/heads/main.zip
• Copy to Kodi
• adb connect <IP of your Kodi box>
• adb push service.jellyfin-main.zip /sdcard/Download
• Install From zipfile
• Settings -> Addons -> Install from zip file -> Download
• Select service.jellyfin-main.zip

Appendix

Sources

• https://github.com/intro-skipper/intro-skipper/wiki/Installation
• https://github.com/SgtJalau/service.jellyskip/

Reverse Proxy for Jellyfin

 As I wanted to be able to access Jellyfin on the go, I needed to put it behind my reverse proxy.

Jellyfin Changes

Tell Jellyfin that it is behind a proxy so it will ignore the proxy ip and log the correct ip

• https://jellyfin.home.arpa:8920
• hamburger menu -> Networking
• Add the IP of your reverse proxy to "Known Proxies"
• Restart Jellyfin

Reverse Proxy

• Add the below to your NGINX config:

    location = /jellyfin {

        return 301 $scheme://$http_host/jellyfin/ ;

    }

    location /jellyfin/ {

        auth_basic off;

        proxy_pass https://jellyfin.home.arpa:8920/;

        proxy_buffering off;

        proxy_set_header X-Real-IP $remote_addr;

        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_set_header X-Forwarded-Host $http_host;

        proxy_set_header X-Forwarded-Port $server_port;

    }

    location /jellyfin/socket {

        # Proxy Jellyfin Websockets traffic

        auth_basic off;

        proxy_pass https://jellyfin.home.arpa:8920/socket;

        proxy_http_version 1.1;

        proxy_set_header Upgrade $http_upgrade;

        proxy_set_header Connection "upgrade";

        # proxy_set_header Host $host;

        proxy_set_header X-Real-IP $remote_addr;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_set_header X-Forwarded-Protocol $scheme;

        proxy_set_header X-Forwarded-Host $http_host;

        proxy_set_header X-Forwarded-Port $server_port;

    }

• Restart NGINX

Appendix

Sources

• https://jellyfin.org/docs/general/post-install/networking/advanced/nginx/
• https://jellyfin.org/docs/general/post-install/networking/#known-proxies

Lenovo Yoga 7 2-in-1 14" Touchscreen Laptop Review

 As my child had been getting into video games, we decided to get them a laptop of their own so they wouldn't need to borrow my wife's.

Here were my list of requirements:

• Priced less than $700
• 11-14" Screen
• IPS or OLED
• at least 300 nits
• at least 45% NTSC coverage
• A recent processor with great integrated graphics
• AMD Ryzen 7 7840U with Radeon 780M
• AMD Ryzen 7 8840U with Radeon 780M
• Intel Core Ultra 7 155U
• or newer
• RAM >= 16GB 
• Storage >= 512GB 
• USB C charging
• WiFi 7/6E with 2x2 antennas
• Backlit keyboard (optional)
• USB C charging on both sides (optional)

Best Buy had a sale for the Lenovo Yoga 2-in-1 14" Touchscreen Laptop (83DK000AUS / 12IAHP9), which we bought as it met most of my criteria.

• Price: $600
• 14" Screen 300 nits 45% NTSC coverage (~60% sRGB)
• AMD Ryzen 7 8840U with Radeon 780M
• RAM: 16GB
• Storage: 1TB
• USB C charging
• WiFi 6E with 2x2 antennas
• Backlit keyboard
• USB C only on a single side :-(
• Left ports:
• 2 x USB C (one USB4 40Gbps)
• 1 x HDMI 2.1
• 1 x 3.5mm headphone/microphone combo jack
• Right ports:
• 1 x USB A
• 1 x microSD
• 3.55 lbs
• Charger is a 65W brick style with 3 pin power cord

Compared to my wife's Lenovo Yoga 2-in-1 16" Touchscreen Laptop (82QG0001US / 16IAP7) which we bought in 2022:

• Price: $600
• 16" Screen 400 nits 72% NTSC coverage (100% sRGB)
• Intel Core i5-1240P
• RAM: 8GB
• Storage: 256GB
• USB C charging
• WiFi 6E with 2x2 antennas
• Backlit keyboard
• USB C only on a single side :-(
• Left ports:
• 2 x USB C (thunderbolt)
• 1 x USB A
• 1 x HDMI 2.0
• 1 x SD card reader
• Right ports:
• 1 x USB A
• 1 x 3.5mm headphone/microphone combo jack
• 4.37 lbs
• Charger is a 65W wall-wort style with 2 folding pins

My thoughts:

• The laptop feels sturdy and premium
• While the screen is not for color accurate work, or as good as my wife's, it looks plenty good to my eyes when not looking side-by-side
• The memory and storage should be sufficient for many years to come
• GPU should be about 2x more performant than my wife's, which should be plenty to play the types of games that my child likes
• The included 65W charger works well. Unfortunately, the laptop charges extremely slowly on our existing 45W chargers.

Child acceptance factor, in general they like it except:

• it is too heavy (would prefer a ligher 11")
• sometimes when starting screen share on Zoom, Zoom will stop responding

Appendix

Sources:

• https://www.bestbuy.com/site/lenovo-yoga-7-2-in-1-14-2k-touchscreen-laptop-amd-ryzen-7-8840hs-with-16gb-memory-1-tb-ssd-artic-grey/6571366.p?skuId=6571366
• https://psref.lenovo.com/Detail/Yoga_7_2_in_1_14AHP9?M=83DK000AUS
• https://psref.lenovo.com/Detail/Yoga/Yoga_7_16IAP7?M=82QG0001US

Other Reviews:

• https://www.notebookcheck.net/AMD-Radeon-780M-iGPU-analysis-AMD-s-new-RDNA-3-GPU-takes-on-its-competitors.714019.0.html
• https://www.rtings.com/laptop/reviews/lenovo/yoga-7-2-in-1-14-2024
• https://www.cnet.com/tech/computing/best-lenovo-laptop/

Ensuring USB DVD order

As my USB dvd drives were being detected into a seemingly random order, I wanted to be able to control what device each was being assigned.

Adding udev rules

• Find serials to add
• sudo udevadm info /dev/sr0 | grep ID_SERIAL_SHORT
• sudo udevadm info /dev/sr1 | grep ID_SERIAL_SHORT
• sudo udevadm info /dev/sr2 | grep ID_SERIAL_SHORT
• Create /etc/udev/rules.d/99-dvd.rules

ACTION="add", KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="first drive serial", SYMLINK+="dvd0"

ACTION="add", KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="second drive serial", SYMLINK+="dvd1"

ACTION="add", KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="third drive serial", SYMLINK+="dvd2"

• Reboot or force rerunning the rules
• sudo udevadm control --reload
• sudo udevadm trigger
• However, this did not work
• Edit /etc/udev/rules.d/99-dvd.rules and remove the ACTION

KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="first drive serial", SYMLINK+="dvd0"

KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="second drive serial", SYMLINK+="dvd1"

KERNEL="sr[0-9]", ENV{ID_SERIAL_SHORT}="third drive serial", SYMLINK+="dvd2"

• Reboot or force rerunning the rules
• sudo udevadm control --reload
• sudo udevadm trigger

Outcome

Success, it now creates the /dev/dvd* in a consistent order. Unfortunately, Handbrake still tries to use the /dev/sr* labels.

Appendix

Sources

• https://www.reddit.com/r/linuxquestions/comments/1ci1nsj/fixed_identity_location_for_multiple_optical/
• https://reactivated.net/writing_udev_rules.html#basic
• https://dev.to/enbis/how-udev-rules-can-help-us-to-recognize-a-usb-to-serial-device-over-dev-tty-interface-pbk