Title: Using GitHub Actions to maintain Gentoo packages repository
       Author: Solène
       Date: 04 March 2023
       Tags: gentoo automation
       Description: In this article, I explain how I used GitHub actions to
       build a packages repository for Gentoo and keep it up to date
       
       # Introduction
       
       In this blog post, I'd like to share how I had fun using GitHub actions
       in order to maintain a repository of generic x86-64 Gentoo packages up
       to date.
       
       Built packages are available at https://interbus.perso.pw/ and can be
       used in your `binrepos.conf` for a generic x86-64 packages provider,
       it's not building many packages at the moment, but I'm open to add more
       packages if you want to use the repository.
       
 (HTM) GitHub Project page: Build Gentoo Packages For Me
       
       # Why
       
       I don't really like GitHub, but if we can use their CPU for free for
       something useful, why not?  The whole implementation and setup looked
       fun enough that I should give it a try.
       
       I was using a similar setup locally to build packages for my Gentoo
       netbook using a more powerful computer, so it was actually achievable,
       so I had to try.  I don't have much use of it myself, but maybe a
       reader will enjoy the setup and do something similar (maybe not for
       Gentoo).
       
       My personal infrastructure is quite light, with only an APU router plus
       a small box with an Atom CPU as a NAS, I was looking for a cheap way to
       keep their Gentoo systems running without having to compile locally.
       
       # Challenges
       
       Building a generic Gentoo packages repository isn't straighforward for
       a rew reasons:
       
       * compilation flags must match all the consumers' architecture
       * default USE flags must be useful for many
       * no support for remote builders
       * the whole repository must be generated on a single machine with all
       the files (can't be incremental)
       
       Fortunately, there are Gentoo containers images that can be used to
       start a fresh Gentoo, and from there, build packages from a clean
       system every time.  Packages have to be added into the container before
       each change, otherwise the file `Packages` that will be generated as a
       repository index won't contain all the files.
       
       Using a `-march=x86-64` compiler flag allows targeting all the amd64
       systems, at the cost of less optimized binaries.
       
       For the USE flags, a big part of Gentoo, I chose to select a default
       profile and simply stick with it.  People using the repository could
       still change their USE flags, and only pick the binary packages from
       the repo if they still match expectations.
       
       # Setup
       
       We will use GitHub actions (Free plan) to build packages for a given
       Gentoo profile, and then upload it to a remote server that will share
       the packages over HTTPS.
       
       The plan is to use a docker image of a stage3 Gentoo provided by the
       project gentoo-docker-images, pull previously built packages from my
       server, build new packages or updating existing packages, and push the
       changes to my server.  Meanwhile, my server is serving the packages
       over https.
       
       GitHub's actions are a feature from GitHub allowing to create
       Continuous Integration easy by providing "actions" (reusable components
       made by other) that you organize in steps.
       
       For the job, I used the following steps on an Ubuntu system:
       
       1. Deploy SSH keys (used to pull/push packages to my server) stored as
       secrets in the GitHub project
       2. Checkout the sources of the project
       3. Make a local copy of the packages repository
       4. Create a container image based on the Gentoo stage3 + instructions
       to run
       5. Run the image that will use emerge to build the packages
       6. Copy the new repository on the remote server (using rsync to copy
       the diff)
       
 (HTM) GitHub project page: Gentoo Docker Images
       
       # Problems encountered
       
       While the idea is simple, I faced a lot of build failures, here is a
       list of problems I remember.
       
       ## Go is failing to build (problem is Docker specific)
       
       For some reasons, Go was failing to build with a weird error, this is
       due to some sandboxing done by emerge that wasn't allowed by the Docker
       environment.
       
       The solution is to loose the sandboxing with `FEATURES="-ipc-sandbox
       -pid-sandbox -sandbox -usersandbox"` in `/etc/portage/make.conf`. 
       That's not great.
       
       ## Raw stage3 is missing pieces
       
       The starter image is a stage3 of Gentoo, it's quite bare, one critical
       package missing to build other but never pulled as dependency is kernel
       sources.
       
       You need to install `sys-kernel/gentoo-sources` if you want builds to
       succeed for many packages.
       
       ## No merged-usr profile
       
       The gentoo docker images repository isn't provided merged-usr profiles
       (yet?), I had to install merged-usr and run it, to have a correct
       environment matching the selected profile.
       
       ## Compilation is too long
       
       The job time is limited to 6h00 on the free plan, I added a timeout for
       the emerge doing the building job to stop a bit earlier, to let it some
       time to push the packages to the remote server, this will allow saving
       time for the next run.  Of course, this only works until a single
       package require more than the timeout time to build (but it's quite
       unlikely given the CI is fast enough).
       
       # Security
       
       One has to trust GitHub actions, GitHub employees may have access to
       jobs running there, and could potentially compromise built packages
       using a rogue container image.  While it's unlikely, this is a
       possibility.
       
       Also, please note that the current setup doesn't sign the packages. 
       This is something that could be added later, you can find documentation
       on the Gentoo Wiki for this part.
       
 (HTM) Gentoo Wiki: Binary package guide
       
       Another interesting area for security was the rsync access of the
       GitHub actions to easily synchronize the packages with the builder. 
       It's possible to restrict an SSH key to a single command to run, like a
       single rsync with no room to change a single parameter.  Unfortunately,
       the setup requires using rsync in two different cases: downloading and
       pushing files, so I had to write a wrapper looking at the variable
       `SSH_COMMAND` and allowing either the "pull" rsync, or the "push"
       rsync.
       
 (HTM) Restrict rsync command over SSH
       
       # Conclusion
       
       The GitHub free plan allows you to run a builder 24/7 (with no parallel
       execution), it's really fast enough to keep a non-desktop @world up to
       date.  If you have a pro account, the local cache GitHub cache may not
       be limited, and you may be able to keep the built packages there,
       removing the "pull packages" step.
       
       If you really want to use this, I'd recommend using a schedule in the
       GitHub action to run it every day.  It's as simple as adding this in
       the GitHub workflow.
       
       ```yaml
       on:
         schedule:
          - cron: '0 2 * * *'  # every day at 02h00
       ```
       
       # Credits
       
       I would like to thank Jonathan Tremesaygues who wrote most of the
       GitHub actions pieces after I shared with him about my idea and how I
       would implement it.
       
 (HTM) Jonathan Tremesaygues's website
       
       # Going further
       
       Here is a simple script I'm using to use a local Linux machine as a
       Gentoo builder for the box you run it from.  It's using a gentoo stage3
       docker image, populated with packages from the local system and its
       `/etc/portage/` directory.
       
       Note that you have to use `app-misc/resolve-march-native` to generate
       the compiler command line parameters to replace `-march=native` because
       you want the remote host to build with the correct flags and not its
       own `-march=native`, you should also make sure those flags are working
       on the remote system.  From my experience, any remote builder newer
       than your machine should be compatible.
       
 (HTM) Tildegit: Example of scripts to build packages on a remote machine for the local machine