Tech Tips

02 February 2024

New VNC client for ChromeOS

As RealVNC discontinued their ChromeOS version and it was giving me issues with disconnections, I decided to look for a replacement.

The search

xtightvncviewer

Bad connection dialog
Does not handle ChromeOS scaling properly
No client window scaling without changing host resolution

ssvnc

Bad connection dialog
Does not handle ChromeOS scaling properly
No client window scaling without changing host resolution
Puts 2 icons on the dock
Supports ssh/ssl encryption

tigervnc-viewer

Acceptable connection dialog
Does not handle ChromeOS scaling properly
No client window scaling without changing host resolution
Support TLS encryption

vinagre

Acceptable connection dialog
Handles ChromeOS scaling properly
Suppports client window scaling without changing host resolution
Dock icon didn't load properly
Touchpad scrolling does not work
No longer maintained, superseded by Gnome Connections

Gnome Connections

Slick looking connections page
Handles ChromeOS scaling properly
Supports client window scaling without changing host resolution
Supports TLS encryption
Touchpad scrolling does not work
Does not remember/resize window when connecting

After my search I decided to go with TigerVNC viewer, but will keep Gnome Connections installed as it may eventually overtake it. Below is how I installed each

Installing TigerVNC viewer

Launch terminal

sudo apt install tigervnc-viewer

Configure it to scale properly

First determine your Chromebooks scaling

Settings -> Displays -> Display size

Test to make sure it is what you like, where .8 == 80% from above

/usr/bin/sommelier -X --scale=.8 /usr/bin/xtigervncviewer

Edit xtightvncviewer.desktop

cp /usr/share/applications/xtigervncviewer.desktop ${HOME}/.local/share/applications/
vi ${HOME}/.local/share/applications/xtigervncviewer.desktop
find "Exec"
and set it to the command that you tested

Launch the App

Search Key -> TigerVNC

Connect

192.168.1.XXX:1

Installing Gnome Connections

Resize your linux storage size

Settings -> Advanced -> Linux development environment -> Disk size "Change"
I set it to 16 GB

Launch terminal

Make sure apt is up to date

sudo apt update
sudo apt upgrade

Install flatpak

sudo apt install flatpak
flatpak --user remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
Restart the linux container by right clicking on your terminal icon and select "Shut down Linux"

Install and start Gnome Connections

flatpak install flathub org.gnome.Connections
flatpak run org.gnome.Connections

Add your VNC server

192.168.1.XXX:5901
Make sure to select VNC

Appendix

Sources:

26 January 2024

Miscellaneous Minecraft Matters

I wanted to host a Minecraft server for me and my daughter to be able to play on. After some research, using a docker container surfaced as the easiest way to do this. Also I wanted to use Steam Link to be able to play, so I needed to add a shortcut to Minecraft in Steam.

Below are the steps that I used to accomplish this

Minecraft Bedrock Docker Server

https://github.com/itzg/docker-minecraft-bedrock-server
find your players XUID

I did this by starting the server connecting and looking at the server output

create a docker-compose.yml file where OPS has your players XUIDs in a comma separated list and ALLOW_LIST_USERS has the player names and XUIDs that you want to be able to login

version: '3.4'

services:
bds:
image: itzg/minecraft-bedrock-server
environment:
EULA: "TRUE"
GAMEMODE: creative
DIFFICULTY: peaceful
SERVER_NAME: "Our World"
OPS: "1234,5678"
ALLOW_CHEATS: "true"
ALLOW_LIST: "true"
ALLOW_LIST_USERS: "player1:1234,player with spaces:5678"
ports:
- "19132:19132/udp"
volumes:
- /storage/containers/minecraft/world1:/data
stdin_open: true
tty: true

start the container

docker-compose up

A permissions.json file will be created giving the specified players ops powers
Note: even though your server is local the Playstation/Xbox/Switch version will not be able to connect without a PS Plus/Xbox Live/Nintendo Online subscription

https://bugs.mojang.com/browse/BDS-1634

Adding a shortcut to Minecraft in Steam

Find out where Minecraft was stored

Paste the following into an explorer window:

%LocalAppData%\Packages\

Find the folder like:

Microsoft.MinecraftUWP_<seemingly_random_letters_and_numbers>

The seemingly random letters and numbers are the app id, we will need them for later

In Steam go to your Library and click "ADD A GAME" and then "Add a Non-Steam Game..."
Navigate to C:\Windows and select explorer.exe
You will see a new entry in your library called explorer
Right click on it -> Properties
Choose an appropriate icon
Rename it
Click "SET LAUNCH OPTIONS"
type/paste in the following:

shell:appsFolder\<your-app-id>!App

Click "OK"
Click "CLOSE"
You should now be able to launch Minecraft from Steam

Appendix

Sources:

https://linustechtips.com/topic/547779-add-universal-apps-and-purchased-windows-store-games-to-steam-library-and-how-to-pin-steam-games-to-windows-taskbar-and-start-menu/

04 January 2024

Changing a zpool from ashift=9 to ashift=12

I wanted the additional write speed on my nas drives that come from aligning the ashift value with physical sector size of my hard drives (ashift=9 is 512 bytes and ashift=12 is 4KB). Unfortunately, you cannot change ashift on an existing zpool, so you will have to backup the data, destroy the pool, recreate it, and then restore the data.

Prereq

pv (to monitor the process/speed)

sudo apt-get install pv

encrypted zfs data with the "wrong" ashift value that you want to migrate

Process to move

Stop any process that writes to your storage that you want to move
Setup temporary storage location

sudo zpool create external-storage mirror /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-XXXXXXX /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-XXXXXXX
sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase external-storage/encrypted

Create a snapshot

sudo zfs snapshot storage/encrypted@migrate-20231229
sudo zfs list -t snapshot

Copy snapshot over

sudo bash -c 'zfs send storage/encrypted@migrate-20231229 | pv | zfs recv external-storage/encrypted/backup'

Ensure that all files have been backed up
Unmount the datasets

sudo zfs unmount storage/encrypted

Destroy the old zpool

sudo zpool destroy storage

Create the new zpool

sudo zpool create storage mirror /dev/disk/by-id/ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX ata-WDC_WD60EFZX-68B3FN0_WD-XXXXXXX

Ensure is setup with the correct ashift value

sudo zdb -C storage | grep ashift

Create a temporary file to contain your passphrase because since zfs recv is using stdin to pull in the data it cannot prompt for it

echo "super-secret" > /home/example/passphrase.txt

Copy snapshot back

sudo bash -c 'zfs send external-storage/encrypted/backup@migrate-20231229 | pv | zfs recv -o encryption=aes-256-gcm -o keylocation=file:///home/example/passphrase.txt -o keyformat=passphrase storage/encrypted'

Change from a file to prompt for password

sudo zfs change-key -o keylocation=prompt storage/encrypted

Remove the temp passphrase

rm /home/example/passphrase.txt

Check that all your files are back in place
Now if you want you can destroy the backup or export it and keep the backup

sudo zpool destroy external-storage
OR
sudo zpool export external-storage

Appendix

If you see an error like:

cannot receive new filesystem stream: zfs receive -F cannot be used to destroy an encrypted filesystem or overwrite an unencrypted one with an encrypted one
That means that you cannot copy to the encrypted dataset. What I did to get around this was to instead copy to a child of the encrypted dataset.

Sources:

01 January 2024

Migrating from eCryptFS to native zfs encryption

I wanted to move from eCryptFS on top of a zfs dataset to a more standard and speedier encryption approach which is native zfs encryption. Here is the process that I went through.

Process

Ensure your backups are up to date!
Upgrade the zpool

ensure you are on a recent version of zfs and not zfs-fuse (see previous post)
sudo zpool upgrade storage

Create the destination dataset

sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase storage/new-encrypt

Set/change the mount point (optional)

sudo zfs set mountpoint=/storage/new-encrypt storage/new-encrypt

Move the files over

sudo rsync -avh --progress --remove-source-files /storage/encrypted/* /storage/new-encrypt/
-z / --compress is not needed and would slow down a local transfer

Remove the left over directories

sudo find /storage/encrypted/ -type d -empty -delete

Verify no files are left:

ls -al /storage/encrypted
if any files exist then repeat the rsync

Unmount the encryptfs

sudo umount /storage/encrypted

Remove/comment the entry from /etc/fstab

sudo vi /etc/fstab

Unmount the zfs dataset

sudo zfs unmount storage/.encrypted

Test destroying the zfs dataset

sudo zfs destroy -n storage/.encrypted

Destroy the zfs dataset

sudo zfs destroy storage/.encrypted

Change the name of new-encrypt

sudo zfs rename storage/new-encrypt storage/encrypted

Update mountpoint (if required)

sudo zfs set mountpoint=/storage/encrypted storage/encrypted

If everything works then the new zfs native encrypted dataset slots right into where the old one was and all your samba shares should be fine.

Appendix

Sources:

https://rucore.net/en/luks-ecryptfs-or-zfs-encryption-choosing-a-way-to-protect-your-data-in-linux/

13 December 2023

Adventures with ZFS

As part of plan to improve my backup strategy, I was testing out different zfs configurations to see what might be the best option.

The Problem

I was having trouble testing out using native zfs encryption

sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase test-pool/test-encrypt
invalid property 'encryption'

Checking zfs version

So first I checked the zfs version:

sudo modinfo zfs | grep version

version: 0.8.3-1ubuntu12.15

zpool version

returned an error, which I found weird

zpool upgrade -v

returned a maximum version of 23
expected a maximum version of 28

Did some googling and found out that I may have the older zfs-fuse installed
dpkg -s zfs-fuse

confirmed my suspicion and told me that I had 0.7.0 installed

Replace zfs-fuse with zfsutils-linux

* Unmount all zfs datasets*

I didn't do this step, but you definitely should to prevent data errors
sudo zfs unmount <zpool>/<dataset>

sudo apt remove zfs-fuse
sudo apt install zfsutils-linux
Here I rebooted the system
sudo zpool status

no zpools available

sudo zpool import -a

listed my zpool with its last use

sudo zpool import -f <zpool>
sudo zpool status

now correctly showed my zpool

Test setup

I am using 2 disks that have a raw speed of 150 MB/s (benchmarked using dd on a single disk)
Creating the test pool

sudo zpool create test-pool mirror /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number> /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number>
or
sudo zpool create test-pool -o ashift=9 mirror /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number> /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-<serial_number>

Creating test dataset

sudo zfs create test-pool/test
sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase test-pool/test-encrypt

Disable caching

sudo zfs set primarycache=none test-pool/test
sudo zfs set secondarycache=none test-pool/test

10 back-to-back copies of a 5GB file and waiting for the numbers to stabilize

sudo rsync --progress Downloads/Win11_22H2_English_x64v1.iso /test-pool/test/

Deleting the zpool when done testing

sudo zpool destroy test-pool

Results of the upgrade and test

The good news is that I am seeing a 2-3X speed up on my existing zpool setup with ashift=9 from doing the upgrade. It also appears that native encryption has a minimal impact (as long as your processor has AES-NI). Here are the numbers that I was seeing

zfs-fuse 0.7.0:

ashift=9, no encryption: 35-45 MB/s
ashift=12, no encryption: 70-72 MB/s

zfsutils-linux 0.8.3:

ashift=9, no encryption: 108-117 MB/s
ashift=9, native encryption: 105-117 MB/s
ashift=12, no encryption: 134-141 MB/s
ashift=12, native encryption: 133-139 MB/s

Appendix

Article on setting up native zfs encryption:

https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/

checking zfs version:

zfs-fuse being super outdated:

Importing missing zpool:

https://serverfault.com/questions/585228/zfs-import-unable-to-find-any-pools

More about ashift:

rsync stats:

https://unix.stackexchange.com/questions/65077/is-it-possible-to-see-cp-speed-and-percent-copied

09 December 2023

Updating Crucial MX500 Firmware in Linux

On my Crucial MX500, I was noticing a high level of write amplification, which is when you tell it to write 1GB of data, but it actually uses 10GB of writes to the nand flash. To try to fix this, I decided to see if a firmware update would help with this.

Identifying the Problem

Substitute /dev/sdX with you drive

Get the smart attributes

sudo smartctl -A /dev/sdX

write down the values for 247 and 248, I will refer to thus as A
wait a few days and repeat steps 1 and 2, I will refer to this as B
Now lets calculate

247C =247B - 247A
248C = 248B - 248A
(247C + 248C) / 247C

I was seeing values ranging from 10-100, when I believe the typical range should be 1-2.5

Performing the Update

Caution: Before doing any of this be sure that you have up to date backups!

Substitute /dev/sdX with your drive

Use smartctl to check what firmware version you currently have installed so you can download the correct version

sudo smartctl -i /dev/sdX
Example line: Firmware Version: M3CR020

Download the correct firmware for your device:

https://www.crucial.com/support/ssd-support/mx500-support

Mount the iso

sudo mkdir /mnt/iso
sudo mount -o loop,ro MX500_M3CR023_update.iso /mnt/iso

Create a directory to extract the files to

mkdir mx500
cd mx500

Do the extraction

gzip -dc /mnt/iso/boot/corepure64.gz | cpio -idm

List the drives

sudo ./sbin/msecli -L

Perform the Update

sudo ./sbin/msecli -U -v -i ./opt/firmware/ -n /dev/sdX

Conclusion

It seemed to help but has not completely resolved the problem

Appendix

Sources:

https://medium.com/@vdboor/upgrading-crucial-firmware-on-linux-76056254539

Others experiencing a similar write amplification issue:

Guide for calculating Write Amplification:

(PDF) https://www.micron.com/-/media/client/global/documents/products/technical-note/solid-state-storage/tnfd22_client_ssd_smart_attributes.pdf

08 December 2023

Migrating from Unifi USG-3P to UDM Pro

Decision

As my USG-3P was getting on in age and no longer getting updates, I have been shopping around for a replacement. Below were the main competitors for me.

UXG-Lite:

Price: $129 (plus tax and shipping)
WiFi: None
Unifi controller: No
CPU: Two A53 cores at 1 GHz
Memory: 1 GB of DDR3L
Storage: N/A
IPS/IDS max throughput: claimed 1Gbps (subject to third party verification)
Unifi Protect: No support

UDR:

Price: $199 (plus tax and shipping)
WiFi: 6 (however, this was to be deployed in my basement so not much help)
Unifi controller: Yes
CPU: Two A53 cores at 1.35 GHz
Memory: 2 GB of ?DDR3L?
Storage: 128 GB SSD
IPS/IDS max throughput: ~700Mbps
Unifi Protect: would need a SD card

UDM Pro:

Price: $379 --Black Friday $279-- (plus tax and shipping)
WiFi: No
Unifi controller: Yes
CPU: Four A57 cores at 1.7 GHz
Memory: 4 GB of DDR4
Storage: 16 GB eMMC
IPS/IDS max throughput: ~3.5Gbps
Unifi Protect: Hard drive slot
Bonus: LAN + WAN SFP+ ports for 10Gbps networking

To me the extra money was worth it to get the UDM Pro for the Black Friday price of $279, but I would probably not for $379. It just seems like a much more capable product that provides the option to expand later (>1Gbps networking, Unifi Protect).

Initial Impressions

Positives:

Shipping box contained lots of air pillows
High quality foam protecting the device in device packaging
Build quality is superb
Slide out foam screw holder was a nice organizational touch

Negatives:

Single use plastics used to wrap:

UDM Pro itself
Rack mount ears
Instructions, really????

Installation

Background:

This guide was written using Unifi Network 8.0.7
USG-3P network address is 192.168.1.1
Unifi controller is hosted at https://192.168.1.2:8443

Requires:

Internet connection
Laptop with an ethernet port or a PC that can be hardwired
2x ethernet cords (Only 1 is needed if you have an already hardwired PC)

Here are the steps that I used:

Create a backup from your current Unifi controller on a laptop

Navigate to https://192.168.1.2:8443
Settings -> System -> Backups
Click on `Download`
Select number of days (I chose 7)
Click `Download`

Connect a LAN port on your current network to the WAN port on UDM Pro

This is to provide it with internet access

Power on the UDM Pro
Allow it to update (this took several minutes)
Connect a laptop to the LAN port on the UDM Pro

May be helpful to disable WiFi on laptop

Setup Wizard on UDM Pro

Navigate to https://unifi/ (for me https://192.168.0.1 also would have worked)
Login with your Unifi account
Do NOT restore from Backup, skip this step
Finish the setup Wizard

Update the Network Application

You want it to be >= Unifi controller Network version

Restore the backup

Network -> Settings -> System -> Backups
Click on `Restore`
Select the backup you created above
Click on `Restore`
UDM Pro will restart

UDM Pro web UI will become unresponsive

At this point I used the touch screen to reboot the UDM Pro
However, this could be unnecessary and may possibly be resolved by forcing the laptop to get a new DHCP address

Ensure your restore happened correctly

Navigate to https://192.168.1.1
Check that the network settings are correct and that your access points are there (but they won't be connected)

Swap out the USG-3P for the UDM Pro
Migrate the Access Points from Unifi controller to UDM Pro

Navigate to https://192.168.1.2:8443
Settings -> General
Click on `Export Site` on the bottom
You can save the export file, but we won't be using it
Click continue on the `Export Site` dialog
Click continue on the `Migrate Site` dialog (no action needed)
Type in the IP address of the UDM Pro (192.168.1.1)
Select the Access Points to migrate
Click `Migrate Devices`
Check in another tab/window that they migrated to the UDM Pro
Click on `Remove Devices`

Done!

All in all, it took me about 1.5 hours, but this included unboxing, attaching the rack mount ears, mounting in the rack, and some research. The good news is that the network downtime was less than 5 minutes!

Overall Impressions

Positives:

The migration went smoothly, once I pieced together what needed done
Network performance is great
IPS has already started blocking network scans

Negatives:

Will not fit in a 12" rack that is flush mounted

AC plug is very far to one side and almost didn't have enough clearance for the 2x4 supporting my network rack, wish it was more centered
Possibly could be resolved by a 90 degree power connector

There doesn't seem to be a way to manage the screen

It will go into a screensaver mode during the day and turn off at night
However, I cannot find settings to change the times for this behavior
Update 2024-01-04:

To change this you have to set the local account as "Super Admin" on https://unifi.ui.com -> UDM Pro -> OS Settings (At the top looks like a UDM Pro with a gear icon on the bottom right) -> Admins & Users
Then you can update it https://192.168.1.1/console-settings
I set night mode to start at 10:01 PM and end at 10:00 PM thus keeping the screen off for most of the day

Other thoughts:

Lack of detailed official documentation on migration process
Topology is wrong because I have a non-Unifi switch and there isn't a way to manually fix

Research that I found: