Title: Mirroring sources used in nixpkgs (software preservation)
       Author: Solène
       Date: 03 November 2022
       Tags: nix life
       Description: In this text, I will show how to download locally all the
       sources used to build packages in nixpkgs
       
       # Introduction
       
       This may appear like a very niche use case, in my quest of software
       conservancy for nixpkgs I didn't encounter many people understanding
       why I was doing this.
       
       I would like to present you a project I made to easily download all the
       sources files required to build packages from nixpkgs, allowing to keep
       offline copies.
       
       Why would you like to keep a local copy?  If upstream disappear, you
       can't get access to the sources anymore, except maybe in Hydra, but you
       rely on a third party to access the sources, so it's still valuable to
       have local copies of software you care about, just to make copies. 
       It's not that absolutely useful for everyone, but it's always important
       to have such tools available.
       
 (HTM) nixpkgs-mirror-tarballs project page
       
       # How to use it
       
       You must run it on a system with `nix` installed.
       
       After cloning and 'cd-ing' into the directory, simply run `./run.sh
       some package list | ./mirror.pl`.  The command `run.sh` will generate a
       JSON structure containing all the dependencies used by the packages
       listed as arguments, and the script `mirror.pl` will iterate over the
       JSON list and use nix's fetcher to gather the sources in the nix store,
       verifying the checksum on the go.  This will create a directory
       `distfiles` containing symlinks to the sources files stored in the
       store.
       
       The symlinks are very important as they will prevent garbage collection
       from the store, and it's also used internally to quickly check if a
       file is already in the store.
       
       To delete a file from the store, remove its symlink and run the garbage
       collector.
       
       # Limitation
       
       I still need to figure how to get a full list of all the packages, I
       currently have a work in progress relying on `nix search --json` but it
       doesn't work on 100% of the packages for some reasons.
       
       It's currently not possible to easily trim distfiles that aren't useful
       anymore, I plan to maybe add it someday.
       
       # Trivia
       
       This task is natively supported in the OpenBSD tool building packages
       (dpb), it can fetch multiples files in parallel and automatic remove
       files that aren't used anymore.  This was really complicated to figure
       how to replicate this with nixpkgs.