Title: How to account systemd services bandwidth usage on NixOS
       Author: Solène
       Date: 20 July 2022
       Tags: nixos bandwidth monitoring
       Description: This article shows how to easily record the bandwidth
       usage of specific systemd services on NixOS, but this can be applied to
       Linux in general
       
       # Introduction
       
       Did you ever wonder how many bytes a system service is daily receiving
       from the network?  Thanks to systemd, we can easily account this.
       
       This guide targets NixOS, but the idea could be applied on any Linux
       system using systemd.
       
 (HTM) NixOS project website
       
       In this article, we will focus on the nix-daemon service.
       
       # Setup
       
       We will enable the attribute IPAccounting on the systemd service
       nix-daemon, this will make systemd to account bytes and packets that
       received and sent by the service.  However, when the service is
       stopped, the counters are reset to zero and the information logged into
       the systemd journal.
       
       In order to efficiently gather the network information over time into a
       database, we will run a script just before the service stops using the
       preStop service hook.
       
       The script checks the existence of a sqlite database
       /var/lib/service-accounting/nix-daemon.sqlite, creates it if required,
       and then inserts the received bytes information of the nix-daemon
       service about to stop.  The script uses the service attribute
       InvocationID and the current day to ensure that a tuple won't be
       recorded more than once, because if we restart the service multiple
       times a day, we need to distinguish all the nix-daemon instances.
       
       Here is the code snippet to add to your `/etc/nixos/configuration.nix`
       file before running `nixos-rebuild test` to apply the changes.
       
       ```nix
         systemd.services.nix-daemon = {
             serviceConfig.IPAccounting = "true";
             path = with pkgs; [ sqlite busybox systemd ];
             preStop = ''
       #!/bin/sh
       
       SERVICE="nix-daemon"
       DEST="/var/lib/service-accounting"
       DATABASE="$DEST/$SERVICE.sqlite"
       
       mkdir -p "$DEST"
       
       # check if database exists
       if ! dd if="$DATABASE" count=15 bs=1 2>/dev/null | grep -Ea "^SQLite format.[0-9]$" >/dev/null
       then
       cat <<EOF | sqlite3 "$DATABASE"
       CREATE TABLE IF NOT EXISTS accounting (
               id TEXT PRIMARY KEY,
               bytes INTEGER NOT NULL,
               day DATE NOT NULL
       );
       EOF
       fi
       
       BYTES="$(systemctl show "$SERVICE.service" -P IPIngressBytes | grep -oE "^[0-9]+$")"
       INSTANCE="'$(systemctl show "$SERVICE.service" -P InvocationID | grep -oE "^[a-f0-9]{32}$")'"
       
       cat <<EOF | sqlite3 "$DATABASE"
       INSERT OR REPLACE INTO accounting (id, bytes, day) VALUES ($INSTANCE, $BYTES, date('now'));
       EOF
            '';
         };
       
       ```
       
       If you want to apply this to another service, the script has a single
       variable SERVICE that has to be updated.
       
       # Display the information from the database
       
       You can use the following command to display the bandwidth usage of the
       nix-daemon service with a day-by-date report:
       
       ```shell
       $ echo "SELECT day, sum(bytes)/1024/1024 AS Megabytes FROM accounting group by day" | sqlite3 -header -column /var/lib/service-accounting/nix-daemon.sqlite
       day         Megabytes
       ----------  ---------
       2022-07-17  173
       2022-07-19  3018
       2022-07-20  84
       ```
       
       Please note this command requires the sqlite package to be installed in
       your environment.
       
       # Enhancement
       
       I have some ideas to improve the setup:
       
       * The script could be improved to support multiple services within the
       database by using a new field
       * The command to display data could be improved and turned into a
       system package to make it easier to use
       * Provide an SQL query for monthly summary
       
       # Conclusion
       
       Systemd services are very flexible and powerful thanks to the hooks
       provided to run script at the right time.  While I was interested into
       network usage accounting, it's also possible to achieve a similar
       result with CPU usage and I/O accesses.