--- title: "Lazy Hardened Void Linux on Raspberry Pi" date: 2021-09-05 --- Note: this information is not meant as instructions, and I don't recommend you implement any of these changes without understanding what you're doing. I'm a lazy person and this method works *reasonably well* for my purposes, but these instructions won't be compatible with most people's needs. I'm writing it down for *my own use* and on the off-chance that someone else might find some part of it useful. I use Void Linux on Raspberry Pi for a lot of small projects. In most cases I use Raspberry Pi Zero W for the convenient built-in 2.4GHz 802.11n wifi, and for a few projects that need image processing or other heavy processing tasks I use Raspberry Pi 4. My use cases are very specialized and simple and I have very few services running on these Raspberry Pis. I do a couple of system modifications to hopefully make these devices more resilient against filesystem corruption. ## OS Installation MicroSD cards vary in quality. I take my MicroSD card recommendations from [Jeff Geerling's blog](https://www.jeffgeerling.com/blog/2019/raspberry-pi-microsd-card-performance-comparison-2019) and the only MicroSD failures that I've encountered in the last few years have all been my own fault. I pop the MicroSD card into my Thinkpad and give it a typical Raspberry Pi partition table: ``` Device Boot Start End Sectors Size Id Type /dev/mmcblk0p1 * 2048 206847 204800 100M b W95 FAT32 /dev/mmcblk0p2 206848 1983999 1777152 867.8M 83 Linux ``` Then I uncompress the premade musl-libc Raspberry Pi rootfs image to install Void. ## First boot Swich to bash for some conveniences: ``` chsh -s /bin/bash ``` Put the hostname in the prompt: ``` -bash-5.1# vi ~/.profile PS1="\H \w\\$ " set -o vi -bash-5.1# . ~/.profile void-live ~# ``` Write a hostname to `/etc/hostname` and set the system hostname with `hostname(1)`. Make sure ntpd, wpa_supplicant, dhcpcd and sshd are enabled. Add a network to `/etc/wpa_supplicant/wpa_supplicant.conf`: ``` network = { ssid="network ssid here" psk="plaintext password here" } ``` Make sure the device gets an IP. Install `socat`. ## Set up SSH certificate Copy `/etc/ssh/ssh_host_ed25519_key` to key signing server and sign it with $rpi_host_key. Copy `ssh_host_ed25519_key.cert` and signing server's $user_key_pubkey `rpi_user_key.pub` back to `/etc/ssh/`. Add these lines to the bottom of `/etc/ssh/sshd_config`: ``` TrustedUserCAKeys /etc/ssh/rpi_user_ca.pub HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub ``` Confirm that `rpi_user_key` can authenticate with the new host, and that the SSH client accepts its host key certificate. ## Optional: Allow some unprivileged SSH commands Some of my Raspberry Pis automatically control devices and/or provide data sources. So I have a separate SSH key that's used by each service that needs to control/access each Raspberry Pi. If this new host needs to allow other hosts to automatically SSH into it to run certain commands, do the following: Generate a new SSH keypair. Install the pubkey onto the new host:'s `~/.ssh/authorized_keys`: ``` airquality ~# vi /root/.ssh/authorized_keys command="/root/bin/ssh_command $SSH_ORIGINAL_COMMAND" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHgtXn3mDJ8RJFG0Rh35lVh6WBP7WXF9CGbIc95O8hfF unprivileged key for airquality (not the real public key) airquality ~# mkdir ~/bin airquality ~# vi ~/bin/ssh_command #!/bin/sh case "$1" in on) ~/bin/on ;; off) ~/bin/off ;; get) ~/bin/get ;; *) echo invalid ssh_command ;; esac airquality ~# chmod +x $_ airquality ~# vi ~/bin/on #!/bin/sh # Script to turn device 'on' airquality ~# chmod +x $_ ``` Then copy the private key to the controlling host to be used for unprivileged SSH access to only the services listed in `~/bin/ssh_command`. ## Prepping runit supervise directories for readonly rootfs The services that ship with Void all use symlinks to keep their `supervise` directories in `/run/runit/`: ``` airquality /etc/sv# file */supervise agetty-tty1/supervise: symbolic link to /run/runit/supervise.agetty-tty1 agetty-ttyS0/supervise: broken symbolic link to /run/runit/supervise.agetty-ttyS0 aqi/supervise: directory dhcpcd/supervise: symbolic link to /run/runit/supervise.dhcpcd ntpd/supervise: symbolic link to /run/runit/supervise.chronyd sshd/supervise: symbolic link to /run/runit/supervise.sshd udevd/supervise: symbolic link to /run/runit/supervise.udevd wpa_supplicant/supervise: symbolic link to /run/runit/supervise.wpa_supplicant airquality /etc/sv# ``` Note the broken symlink for `agetty-ttyS0` which means it isn't running. Normally if you create a custom service runit will create a default `./supervise` directory to track its status. But if our rootfs is readonly then that won't be possible and runit won't be able to start our service. So manually create symlinks for all custom services: ``` airquality /etc/sv/aqi# file * run: POSIX shell script, ASCII text executable supervise: directory airquality /etc/sv/aqi# rm /var/service/aqi airquality /etc/sv/aqi# ln -s /run/runit/supervise.aqi supervise airquality /etc/sv/aqi# file * run: POSIX shell script, ASCII text executable supervise: broken symbolic link to /run/runit/supervise.aqi airquality /etc/sv/aqi# ln -s /etc/sv/aqi /var/service airquality /etc/sv/aqi# ``` ## Prepping /var for readonly rootfs ``` ln -s /tmp /var/log ln -s /tmp /var/tmp ``` ## Optional: Device Tree Overlays If this project needs any ARM device tree overlays, now's the time to add them. For example, to enable the UART and disable the UART console: ``` airquality ~# mount /dev/mmcblk0p1 /boot airquality ~# vi /boot/config.txt enable_uart=1 airquality ~# vi /boot/cmdline.txt root=/dev/mmcblk0p2 rw rootwait console=tty1 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 loglevel=4 elevator=noop airquality ~# umount /boot ``` ## Optional: Wireguard ## Optional: Network logging For example, sending syslog to `10.0.0.2:514`: ``` airquality ~# mkdir /etc/sv/logger airquality ~# cd $_ airquality /etc/sv/logger# vi run #!/bin/sh exec socat unix-recv:/dev/log udp:10.0.0.2:514 airquality /etc/sv/logger# chmod +x run airquality /etc/sv/logger# ln -s /run/runit/supervise.logger supervise airquality /etc/sv/logger# ln -s /etc/sv/logger /var/service airquality /etc/sv/logger# ``` Additionally, to log kernel logs the same way: ``` airquality /etc/sv/logger# mkdir /etc/sv/klogger airquality /etc/sv/logger# cd $_ airquality /etc/sv/klogger# vi run #!/bin/sh exec socat /proc/kmsg unix-send:/dev/log airquality /etc/sv/klogger# chmod +x run airquality /etc/sv/klogger# ln -s /run/runit/supervise.klogger supervise airquality /etc/sv/klogger# ln -s /etc/sv/klogger /var/service airquality /etc/sv/klogger# ``` And on `10.0.0.2`, the logging server: ``` logger /etc/sv/# mkdir /var/log/udp logger /etc/sv/# mkdir -p socklog-udp/log logger /etc/sv/# cd socklog-udp logger /etc/sv/socklog-udp# vi run #!/bin/sh exec chpst -Unobody socklog inet 0 1337 2>&1 logger /etc/sv/socklog-udp# chmod +x run logger /etc/sv/socklog-udp# cd log logger /etc/sv/socklog-udp/log# vi run #!/bin/sh exec svlogd -t /var/log/udp logger /etc/sv/socklog-udp/log# chmod +x run logger /etc/sv/socklog-udp/log# ln -s /etc/sv/socklog-udp /var/service logger /etc/sv/socklog-udp/log# ``` ## Optional: Watchdog This will enable a hardware watchdog to potentially recover the system in case it bogs way down. I only started doing this recently so I'm not sure if this will be useful or if it's just going to cause problems. ``` airquality ~# mkdir /etc/sv/watchdog airquality ~# cd $_ airquality /etc/sv/watchdog# vi run #!/bin/sh modprobe watchdog while true; do echo 1 > /dev/watchdog; sleep 14; done airquality /etc/sv/watchdog# chmod +x run airquality /etc/sv/watchdog# ln -s /run/runit/supervise.watchdog supervise airquality /etc/sv/watchdog# ln -s /etc/sv/watchdog /var/service airquality /etc/sv/watchdog# ``` The watchdog will generate some noise in your kernel logs every 14 seconds, which gets noisy if you're doing network logging of kernel logs: ``` kern.crit: [1538830.858527] watchdog: watchdog0: watchdog did not stop! ``` ## Actually making the rootfs readonly Edit `/etc/runit/core-services/03-filesystems.sh` and comment out these two lines near the bottom: ``` #msg "Mounting rootfs read-write..." #mount -o remount,rw / || emergency_shell ``` Then reboot and verify that all of your required services started properly. ## Managing system with read-only rootfs You can still start and stop services as needed. If you need to modify something, you can temporarily remount the root fs read-write with: ``` airquality ~# mount -o remount,rw / ``` Then make the changes and reboot. If it's not a good time to reboot, you can also just remount the root fs again as ro. On a stripped-down Void install there are so few services running that nothing will open any new files and obstruct the remount.