Title: External GPU on Linux review
Author: Solène
Date: 01 September 2021
Tags: linux gentoo games egpu
Description:
# Introduction
I like playing video games, and most games I play require a GPU that is
more powerful than the integrated graphic chipset that can be found in
laptop or computers. I recently found that external graphic card were
a thing, and fortunately I had a few spare old graphic card for trying.
The hardware is called an eGPU (for external GPU) and are connected to
the computer using a thunderbolt link. Because I buy most of my
hardware second hand now, I've been able to find a Razer Core X eGPU
(the simple core X and not the core X Chroma which provides USB and
RJ45 connectivity on the case through thunderbolt), exactly what I was
looking for. Basically, it's an external case with a PSU inside and a
rack, pull out the rack and insert the graphic card, and you are done.
Obviously, it works fine on Windows or Mac but it can be tricky on
Linux.
(HTM) Razer core X product
(IMG) Attempt to make a picture of my eGPU with an nvidia 1060 in it
# My setup
I'm using a Lenovo T470 with an i5 CPU. When I want to use the eGPU, I
connect the thunderbolt wire and keyboard / mouse (which I connect
through an USB KVM to switch those from a computer to another). The
thunderbolt port also provide power to the laptop which is good to
know.
# How does it work?
There are two ways to use this device, the display can be connected to
the eGPU itself or the rendering could be done on the laptop (let's say
we only target laptops here) using the eGPU as a discrete card (only
rendering, without display). Both modes have pros and cons.
* External display Pros: best performance, allow many displays to be
used
* External display Cons: require a screen
* Discrete mode Pros: no extra wire, no different setup when using the
laptop without the eGPU
* Discrete mode Cons: performance penalty, support doesn't work well on
Linux
The performance penalty comes from the fact the thunderbolt bandwidth
is limited, and if you want to display on your screen you need to
receive the data back which will reduce the bandwidth allowed for
rendering. A penalty of at least 20% should be expected in normal
mode, and around 40% in discrete mode. This is not really fun but for
a nice boost with an old graphic card this is still nice.
(HTM) eGPU on Linux with a Razer core X Chroma
(HTM) eGPU benchmarks
# Configuration (NVIDIA)
It's quite simple now in 2023 (blog update), the first step is to
install nvidia-drivers.
## Discrete mode
- Add your user to the video group (at least on Gentoo)
- No `/etc/X11/xorg.conf` file is required
- The graphical card should appear in `nvidia-settings` under a "PRIME"
menu
- Use `prime-run` as a prefix to run commands, the discrete mode is
simply enabled by environment variables. If `prime-run` isn't a thing
in your distribution, create a script `nvidia-offload` like explained
in the NixOS wiki
(HTM) NixOS wiki: Nvidia - offload mode
If you want to run Flatpak programs with the discrete GPU, you will
need to set all the environment variables in the flatpak program
environment. You can't just set them in your shell and run flatpak
from there because of the sandboxing.
## External display
- Run `nvidia-xconfig` to create a `/etc/X11/xorg.conf` file that uses
the Nvidia card as the main display
## Both mode
I ended figuring a `xorg.conf` allowing me to keep the same file with
and without the eGPU, and to use the discrete and external display at
the same time. The funniest part is if you run a program on the nvidia
screen and move it back to the laptop screen, the eGPU continues to
render it.
It's by far the most convenient configuration as you have nothing to
tweak, and you can use laptop + eGPU displays.
```xorg.conf
Section "ServerLayout"
Identifier "layout"
Screen 0 "intel"
Inactive "nvidia"
Option "AllowNVIDIAGPUScreens"
EndSection
Section "Device"
Identifier "intel"
Driver "modesetting"
BusID "PCI:0:2:0"
EndSection
Section "Screen"
Identifier "intel"
Device "intel"
EndSection
Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:10:0:0"
Option "AllowExternalGpus" "True"
EndSection
Section "Screen"
Identifier "nvidia"
Device "nvidia"
EndSection
```
## Switching between modes
If you want to switch from one to the other, you need to exit all X
servers first. Booting with a `xorg.conf` for Nvidia while not having
a Nvidia card plugged in will prevent X to start, which is annoying.
The program egpu-switch can help in that regard, but it can't choose
between discrete or external display mode, you will need to decide
which mode you prefer when the card is plugged by providing the
according xorg.conf file.
(HTM) egpu-switcher GitHub project page
# What to expect of it on Linux?
I've been using this on Gentoo only so far, but I had a previous
experience with a pretty similar setup a few years ago with a laptop
with a discrete nvidia card (called Optimus at that time), and the GPU
was only usable as a discrete GPU and it was a mess at that time.
As for the eGPU, in external mode it works fine using the nvidia
driver, I needed an xorg.conf file to tell to use the nvidia driver,
then the display would be fine and 3D would work perfectly as if I was
using a "real" card on a computer. I can play high demanding games
such as Control, Death Stranding or other games using my Thinkpad
Laptop when docked, this is really nice!
The setup is a bit weird though, if I want to undock, I need to prepare
the new xorg.conf file and stop X, disconnect the eGPU and restart the
display manager to login. Not very easy. I've been able to script it
using a simple script at boot that will detect the Nvidia GPU and
choose the correct xorg.conf file just before starting the display
manager, it works quite fine and makes life easier.
# Video games?
I've been playing Steam video games, it works absolutely perfectly due
to their work on Proton to make Windows games running. GOG games works
fine too, I use Lutris games library manager to handle them and it
works so far.
Now, there is the tricky discrete mode. On linux, the bumblebee
project allows rendering a program in a virtual display to benefit from
the 3D acceleration and then show it on another device, this work was
done for Optimus hardware hence the bumblebee name (related to
Transfomers lore). Steam doesn't like bumblebee at all and won't start
game, this is a known bug, Steam is bad at managing multiple GPUs.
I've not been able to display anything using bumblebee.
On the other hand, native Linux GOG games were working fine using
bumblebee, however I don't own much high demanding Linux games so I've
not been able to see if the performance hit was hard. Windows GOG
games wouldn't run, partially because the DXVK (directX to vulkan) Wine
rendering can't be used because bumblebee doesn't allow using Vulkan
graphical API and error messages were unhelpful. I have literally lost
two days of my life trying to achieve something useful with the
discrete GPU mode but nothing came out of it, except native Linux
games.
(IMG) Playing Control on Gentoo (windowed for the screen)
# Why using an eGPU?
Laptops are very limited in their upgrade capabilities, adding a GPU
could avoid someone to own a "gaming" tower PC and a good laptop. The
GPU is 100% replaceable because the case offers a pci express port and
a standard PSU (which can be replaced too!). The EGPU could be shared
among a few users in a home too. This is a nice way to recycling old
GPUs for a nice graphic boost to play everything that is more than 5
years old (and that's a bunch of good games!). I think using a top
notch GPU in this would be a waste though.
# Conclusion
I'm pretty happy with the experience so far, now I can play my
favorites games on Linux using the same computer I like to use all the
day. While the experience is not as plug and play than it is on
Windows, it is solid and stable.
# Troubleshoot
Some reminders when something is wrong.
## OpenSUSE setup
This distribution ships with a tool "prime-select" which is very
convenient, you can pick which driver you want to enable first, or if
you want to do discrete rendering.
## No sound
An udev rule is certainly blocking the audio device for some reasons...
On a system, I found the file
"/lib/udev/rules.d/90-nvidia-udev-pm-G05.rules" with a comment about
disabling audio devices, commenting it solves the problem.