Title: My NixOS workflow after migrating from OpenBSD
Author: Solène
Date: 24 September 2022
Tags: openbsd nixos life
Description: This article tells my journey migrating my computers from
OpenBSD to NixOS
# Introduction
After successfully switching my small computer fleet to NixOS, I'd like
to share about the journey.
I currently have a bunch of computers running NixOS:
* my personal laptop
* the work laptop
* home router
* home file server
* some home lab computer
* e-mail / XMPP / Gemini server hosted at openbsd.amsterdam
That sums up to 6 computers running NixOS, half of them is running the
development version, and the other is running the latest release.
# Migration
## From OpenBSD to NixOS
All the computers above used to run OpenBSD, let me explain why I
migrated. It was a very complicated choice for me, because I still
like OpenBSD despite I uninstalled it.
* NixOS offers more software choice than OpenBSD, this is especially
true for recent software, and porting them to OpenBSD is getting
difficult over time.
* After spending too much time with OpenBSD, I wanted to explore a
whole new world, NixOS being super different, it was a good
opportunity. As a professional IT worker, it's important for me to
stay up to date, Linux ecosystem evolved a lot over that past ten
years. What's funny is OpenBSD and NixOS share similar issues such as
not being able to use binaries found on the Internet (but for various
reasons)
* NixOS maintenance is drastically reduced compared to OpenBSD
* NixOS helps me to squeeze more from my hardware (speed, storage
capacity, reliability)
* systemd: I bet this one will be controversial, but since I learned
how to use it, I really like it (and NixOS make it even greater for
writing units)
Security is hard to measure, but it's the main argument in favor of
OpenBSD, however it is possible to enable mitigations on Linux as well
such as hardened memory allocator or a hardened Kernel. OpenBSD isn't
practical to separate services from running all in the same system,
while on Linux you can easily sandbox services. In the end, the
security mechanisms are different, but I feel the result is pretty
similar for my threat model of protecting against script kiddies.
I give a bonus point for Linux for its ability to account
CPU/Memory/Swap/Disk/network per user, group and process. This allows
spotting unusual activity. Security is about protection, but also
about being aware of intrusion, OpenBSD isn't very good at it at the
moment.
## NixOS modules
One issue I had migrating my mail server and the router was to find
what changes were made in /etc. I was able to figure which services
were enabled, but not really all the steps done a few years ago to
configure them. I had to scrape all the configuration file to see if
it looked like verbatim default configuration or something I changed
manually.
This is where NixOS shines for maintenance and configuration,
everything is declarative, so you never touch anything in /etc. At
anytime, even in a few years, I'll be able to exactly tell what I need
for each service, without having to dig up /etc/ and compare with
default files. This is a saner approach, and also ease migration
toward another system (OpenBSD? ;) ) because I'd just have to apply
these changes to configuration files.
# Workflow
Working with NixOS can be disappointing. Most of the system is
read-only, you need to learn a new language (Nix) to configure
services, you have to "rebuild" your system to make a change as simple
as adding an entry in /etc/hosts, not very "Unix-like".
Your biggest friend is the man page configuration.nix which contains
all the possible configurations settings available in NixOS, from
Kernel choice and grub parameters, to Docker containers started at boot
or your desktop environment.
The workflow is pretty easy, take your configuration.nix file, apply
changes to it, and run "nixos-rebuild test" (or switch if you prefer)
to try the changes. Then, you may want something more elaborated like
tracking your changes in a git or darcs repository, and start sharing
pieces of configuration between machines.
But in the end, you just declare some configuration. I prefer to keep
my configurations very easy to read, I still don't have any modules or
much variable, the common pieces are just .nix files imported for the
systems needing it. It's super easy to track and debug.
# Bento
(HTM) Bento GitHub project page
After a while, I found it very tedious to have to run nixos-rebuild on
each machine to keep them up to date, so I started using the
autoUpgrade module which basically do it for you in a scheduled task.
But then, I needed to centralize each configuration file somewhere, and
have fun with ssh keys because I don't like publishing my configuration
files publicly. Which isn't optimal either as if you make a change
locally, you need to push the changes and connect to the remote host to
pull the changes and rebuild immediately instead of waiting for the
auto upgrade process.
So, I wrote bento, which allows me to manage all the configuration
files in a single place, but better than that, I can build the
configuration locally to ensure they will work once shipped. I quickly
added a way to track the status of each remote system to be sure they
picked up and applied the changes (every 10 minutes). Later, I
improved the network efficiency by central management computer as a
local binary cache, so other systems are now downloading packages from
it locally, instead of downloading them again from the Internet.
The coolest thing ever is that I can manage offline systems such as my
work laptop, I can update its configuration file in the weekend for an
update or to improve the environment (it mostly shares the same
configuration as my main laptop), and it will automatically pick it up
when I boot it.
# Conclusion
Moving to NixOS was a very good and pleasant experience, but I had some
knowledge about it before starting. It might be confusing a lot of
people, and you certainly need to get into NixOS mindset to appreciate
it.