Title: Nvidia card in eGPU and NixOS
Author: Solène
Date: 05 December 2021
Tags: linux games nixos egpu
Description: How to setup an eGPU with NixOS using offloading prime
nvidia feature
# Updates
* 2022-01-02: add entry about specialization and how to use the eGPU as
a display device
# Introduction
I previously wrote about using an eGPU on Gentoo Linux. It was working
when using the eGPU display but I never got it to work for accelerating
games using the laptop display.
Now, I'm back on NixOS and I got it to work!
# What is it about?
My laptop has a thunderbolt connector and I'm using a Razer Core X
external GPU case that is connected to the laptop using a thunderbolt
cable. This allows to use an external "real" GPU on a laptop but it
has performance trade off and on Linux also compatibility issues.
There are three ways to use the nvidia eGPU:
- run the nvidia driver and use it as a normal card with its own
display connected to the GPU, not always practical with a laptop
- use optirun / primerun to run programs within a virtual X server on
that GPU and then display it on the X server (very clunky, originally
created for Nvidia Optimus laptop)
- use Nvidia offloading module (it seems recent and I learned about it
very recently)
The first case is easy, just install nvidia driver and use the right
card, it should work on any setup. This is the setup giving best
performance.
The most complicated setup is to use the eGPU to render what's
displayed on the laptop, meaning the video signal has to come back from
the thunderbolt cable, reducing the bandwidth.
# Nvidia offloading
Nvidia made work in their proprietary driver to allow a program to have
its OpenGL/Vulkan calls to be done in a GPU that is not the one used
for the display. This allows to throw optirun/primerun for this use
case, which is good because they added performance penalty, complicated
setup and many problems.
(HTM) Official documentation about offloading with nvidia driver
# NixOS
I really love NixOS and for writing articles it's so awesome, because
instead of a set of instructions depending on conditions, I only have
to share the piece of config required.
This is the bits to add to your /etc/nixos/configuration.nix file and
then rebuild system:
```nixos configuration
hardware.nvidia.modesetting.enable = true;
hardware.nvidia.prime.sync.allowExternalGpu = true;
hardware.nvidia.prime.offload.enable = true;
hardware.nvidia.prime.nvidiaBusId = "PCI:10:0:0";
hardware.nvidia.prime.intelBusId = "PCI:0:2:0";
services.xserver.videoDrivers = ["nvidia" ];
```
A few notes about the previous chunk of config:
- only add nvidia to the list of video drivers, at first I was adding
modesetting but this was creating troubles
- the PCI bus ID can be found with lspci, it has to be translated in
decimal, here my nvidia id is 10:0:0 but in lspci it's 0a:00:00 with 0a
being 10 in hexadecimal
(HTM) NixOS wiki about nvidia offload mode
# How to use it
The use of offloading is controlled by environment variables. What's
pretty cool is that if you didn't connect the eGPU, it will still work
(with integrated GPU).
## Running a command
We can use glxinfo to be sure it's working, add the environment as a
prefix:
```shell
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo
```
## In Steam
Modify the command line of each game you want to run with the eGPU
(it's tedious), by:
```command line
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia %command%
```
## In Lutris
Lutris has a per-game or per-runner setting named "Enable Nvidia
offloading", you just have to enable it.
# Advanced usage / boot specialisation
Previously I only explained how to use the laptop screen and the eGPU
as a discrete GPU (not doing display). For some reasons, I've
struggled a LOT to be able to use the eGPU display (which gives more
performance because it's hitting less thunderbolt limitations).
I've discovered NixOS "specialisation" feature, allowing to add an
alternative boot entry to start the system with slight changes, in this
case, this will create a new "external-display" entry for using the
eGPU as the primary display device:
```
hardware.nvidia.modesetting.enable = true;
hardware.nvidia.prime.sync.allowExternalGpu = true;
hardware.nvidia.prime.offload.enable = true;
hardware.nvidia.prime.nvidiaBusId = "PCI:10:0:0";
hardware.nvidia.prime.intelBusId = "PCI:0:2:0";
services.xserver.videoDrivers = ["nvidia" ];
# external display on the eGPU card
# otherwise it's discrete mode using laptop screen
specialisation = {
external-display.configuration = {
system.nixos.tags = [ "external-display" ];
hardware.nvidia.modesetting.enable = pkgs.lib.mkForce false;
hardware.nvidia.prime.offload.enable = pkgs.lib.mkForce false;
hardware.nvidia.powerManagement.enable = pkgs.lib.mkForce false;
services.xserver.config = pkgs.lib.mkOverride 0
''
Section "Module"
Load "modesetting"
EndSection
Section "Device"
Identifier "Device0"
Driver "nvidia"
BusID "10:0:0"
Option "AllowEmptyInitialConfiguration"
Option "AllowExternalGpus" "True"
EndSection
'';
};
};
```
With this setup, the default boot is the offloading mode but I can
choose "external-display" to use my nvidia card and the screen attached
to it, it's very convenient.
I had to force the xserver configuration file because the one built by
NixOS was not working for me.