[HN Gopher] Rolling your own minimal embedded Linux for the Rasp...
___________________________________________________________________
Rolling your own minimal embedded Linux for the Raspberry Pi
Author : snwy
Score : 129 points
Date : 2021-08-31 22:10 UTC (11 hours ago)
(HTM) web link (kevinboone.me)
(TXT) w3m dump (kevinboone.me)
| rvz wrote:
| Rolling your own Embedded Linux on a device like the Pi is all
| fun and games for hobbyists until you run into the most typical
| security vulnerabilities and maintenance hell which comes from
| being unable to update it to the latest firmware.
|
| Maybe we will see improvements in this space once other secure
| alternative operating systems and kernels designed for running on
| embedded systems are available.
| traceroute66 wrote:
| Rolling your own minimal embedded Linux is all cute fun. But what
| about _maintaining_ it ?
|
| Assuming you're not going to be rebuilding the whole lot manually
| each time, you're also going to have to go through the rather
| cute experience of building ( _and maintaining_ ) your own
| automation tooling around maintaining your minimal embedded
| Linux.
|
| If I were developing an embedded project, I don't know about you,
| but I would suggest focusing on the project itself might be a
| nice idea, not wasting time on things like a custom OS.
|
| Custom OS is fine if you've got a large team working along side
| you. But if its a one-man-band (or small team) then all the
| cuteness will soon evaporate.
|
| If you think all Linux distros are too bloated, why not go BSD
| instead ?
|
| For example, OpenBSD is famous for a minimal default install, and
| guess what, they maintain ready-to-go builds for Pine 64/64+,
| Raspberry Pi 3 and Raspberry Pi 4.[1]
|
| [1] https://www.openbsd.org/arm64.html
| johnklos wrote:
| I've asked people how to easily download the sources for any
| given Linux distro to compile locally, and I've gotten links to
| sites with pages of instructions.
|
| To build NetBSD/aarch64 on most Unix / Linux OSes (replace -j 8
| with a suitable number for however many processor cores /
| threads you have): curl
| "https://cdn.netbsd.org/pub/NetBSD/NetBSD-
| current/tar_files/src.tar.gz" -o src.tar.gz curl
| "https://cdn.netbsd.org/pub/NetBSD/NetBSD-
| current/tar_files/xsrc.tar.gz" -o xsrc.tar.gz tar xzf
| src.tar.gz ; tar xzf xsrc.tar.gz cd src ./build.sh
| -m evbarm -a aarch64 -x -U -j 8 -D ../dest -O ../obj -T
| ../tools -R ../sets tools release
|
| Is there really nothing this simple for Linux distros?
| kbaker wrote:
| Uh, with Yocto at least, it's also pretty simple for a basic
| bootable disk image/packages... downloading all the source
| and compiling the cross toolchain along the way...
| git clone git://git.yoctoproject.org/poky cd poky
| git checkout -b hardknott -t origin/hardknott source
| oe-init-build-env MACHINE=qemuarm64 bitbake core-
| image-sato
|
| But after the first build you may want more - bringing in
| other functional layers for different features (maybe you
| want docker?), support for different boards (same OS distro
| for x86_64 and ARM, or your own custom board... plus it can
| build a whole SDK for you for local development/debugging?),
| customization of target files, pulling from updated
| upstreams, support in deployment and throughout the device
| lifecycle, custom package feeds, etc.
|
| There's a lot going for Yocto, it has a deep learning curve
| but does a pretty good job of taming the complexity of
| managing an entire OS distro.
| forty wrote:
| I would use buildroot for that. It's very convenient even to
| just build random kernels for random targets
| yjftsjthsd-h wrote:
| Maybe nix/guix, but most distros are going to have difficulty
| because they aren't a single thing _to_ build, they 're a
| collection of separate packages that you'd have to build
| individually.
| 1vuio0pswjnm7 wrote:
| Cannot offer any advice to the parent^1 however the build.sh
| instructions he provides bring to mind a general observation:
| By comparison, running buildroot's script will initiate
| downloads of files. This is a key distinction I see with
| Linux/GNU _distributions_ (cf. the Linux kernel) in general
| (there are exceptions). Linux distribution maintainers try to
| automate _everything_. The idea of the user downloading the
| source tree for the entire distribution manually is not
| contemplated. Another example: When NetBSD is first
| installed, generally no networking programs are enabled by
| default. Its expected the user will turn things on, if she
| wants them on. By contrast, when a popular Linux distribution
| is installed, there are usually networking programs enabled
| by default to listen on the network. I use both Linux and
| NetBSD. I am not invoking a Linux v BSD debate nor arguing
| for one 's approach over the other's, I am just pointing out
| how they are philosophically different.
|
| 1 The "NetBSD way" would be to study the NetBSD source tree
| and then modify certain files to suit one's needs. For
| example, I like to create custom crunched binaries; this
| requires editing a list. By comparsion, with Linux, I have
| never read any advice to study the Debian source tree and
| then modify files to suit one's needs. Rather, the advice one
| finds directs readers to Yocto, buildroot, etc.
| johnklos wrote:
| I understand and appreciate that. But the distinction
| between manually downloading or automatically downloading
| isn't important to me.
|
| Perhaps buildroot does the closest thing. I'll have to give
| it a go.
| 1vuio0pswjnm7 wrote:
| Maybe Gentoo is the closest thing.
| kbaker wrote:
| You should really try out Yocto. There is a lot to learn
| but you can configure it from super-minimal tiny distros
| to mega kitchen-sink-included distros.
|
| Everything is customizable, every file, every compiler
| flag and option of every package, with easy overrides in
| your own layer which only 'appends' to the original
| instructions, or adding and removing packages as you see
| fit in the base install.
| spyremeown wrote:
| >If I were developing an embedded project, I don't know about
| you, but I would suggest focusing on the project itself might
| be a nice idea, not wasting time on things like a custom OS.
|
| And that's why you're probably not working on embedded. Minimal
| environments with almost zero footprint is a _must_ in almost
| all cases. If you 're rolling a custom board, you simply have
| to go custom, which is where OpenBSD will fall short on you.
| Check Variscite's or Toradex websites and see if you find any
| BSD in there.
| foxfluff wrote:
| The other thing is that existing distros are rarely good for
| anything truly autonomous. Too many moving parts. There's
| always the expectation that someone with a display and a
| keyboard, or a sysadmin at the console, is ready to pick up
| the pieces when it falls apart.
| PragmaticPulp wrote:
| The Raspberry Pi is a learning platform, first and foremost.
| This is actually a great learning project for someone to
| understand what goes into a Linux system.
|
| > If I were developing an embedded project, I don't know about
| you, but I would suggest focusing on the project itself might
| be a nice idea, not wasting time on things like a custom OS
|
| Using an off the shelf distribution is ideal your goal is rapid
| prototyping, but most real-world embedded projects end up
| requiring custom distributions sooner or later. Distributions
| tailored to interactive use tend not to be so great for
| distributing and upgrading predictable firmware. You really
| want all of your devices in a known state, not a random mix of
| package versions depending on when they last ran 'apt-get
| upgrade', for example.
|
| In production-grade products, the OS is often built with tools
| like Buildroot or OpenEmbedded (Yocto), not from scratch.
| However, doing a tutorial like this once is a good learning
| exercise.
| synergy20 wrote:
| RPI also sells modules for commercial products, so anything
| you play with RPI can be used on products too.
| moondev wrote:
| esxi for arm edition runs on the rpi 4 and 8 gb models. This
| way you can use the vsphere tooling and api to make it pretty
| painless, including the use of snapshots and cloning
| iJohnDoe wrote:
| Wow! Had no idea that VMware would run on a rpi. Thanks.
|
| It's interesting to think about the different markets for the
| rpi. I would never consider the rpi for learning. I only look
| at it as an inexpensive system to run my favorite distro. I
| want it to run CentOS or NetBSD, etc. rpi OS is meaningless
| to me. I want it to run the stuff I use every day.
| stonecharioteer wrote:
| Does anyone who's read this want to weigh in on how it holds up
| against Linux From Scratch?
| LeoPanthera wrote:
| It claims there's no supported way to boot from a read-only
| filesystem with Raspberry Pi OS, but there is. Setting it up is
| built into raspi-config, at:
|
| Performance Options > Overlay File System
| MrBuddyCasino wrote:
| If I wanted to build a RPi based system that has good support for
| bluetooth, audio (ALSA) and wifi, with fast boot times, what
| would be the most sensible route? Alpine or Buildroot? What are
| the advantages and drawbacks of them?
| mro_name wrote:
| this is a remarkably well designed website, thanks for sharing.
| teleforce wrote:
| Not on Linux distribution but this Arm sponsored textbook
| 'Operating Systems Foundations with Linux on the Raspberry Pi' is
| a useful learning guide:
|
| https://www.arm.com/resources/education/books/operating-syst...
| ggm wrote:
| typo in early commands of the form mkdir -p staging boot (space
| should be /)
|
| also, busybox doesn't demand symlinks: you can use hardlinks.
| willjp wrote:
| This is really interesting! Thank you so much!
| bitwize wrote:
| Systemd and dbus are standard parts of the Linux userland now,
| nearly as essential as the kernel itself as they provide vital
| services.
| detaro wrote:
| Plenty embedded projects developed today don't use them (and
| functional desktop and server distros without them are around),
| "nearly as essential as the kernel" is really overstating it.
| igetspam wrote:
| We'll be headed down this path soon for some custom
| appliances. I can't find a compelling reason for systemd or
| dbus.
| [deleted]
| fragmede wrote:
| The fact that desktop Linux systems mostly run systemd _is_
| (unfortunately) a reason to run a Linux distro that also
| runs systemd on new embedded projects (for large values of
| embedded. Storage is cheap these days, as is a CPU with an
| MMU).
| kbaker wrote:
| If you can fit it in your image, systemd is amazing for
| embedded. Dependency tracking, auto service restarts on
| crash, unified start/stop and enable/disable, even journald
| is nice with per-unit unified logging.
|
| It is miles better than SysV or monit. I haven't needed to
| try the 'suggested' replacements like runit or openrc.
| wryun wrote:
| Alternatively, you can use Void Linux and most of the work is
| done... admittedly, no read only fs by default.
| PragmaticPulp wrote:
| This is a great series of tutorials to follow if you want to gain
| a better understanding of what goes into a basic Linux system. I
| think some of the commenters here are missing the point that it's
| a learning exercise.
|
| For production-grade custom Linux firmware, two of the most
| popular options are Buildroot and OpenEmbedded (Yocto). I'd
| recommend that anyone interested in this space get started with
| Buildroot. It's really easy to make a basic image and customize
| it to your needs, plus you'll learn a lot along the way. Yocto is
| much more powerful and customizable, but the learning curve is
| far more difficult. It's great for more advanced situations where
| you need to support multiple boards or bring in 3rd-party layers.
| codetrotter wrote:
| I second that, Buildroot is great
| unmole wrote:
| PTXdist is also pretty good and very similar to Buildroot. My
| experience is a bit dated but PTXdist had better support for
| some boards than Buildroot.
| TickleSteve wrote:
| I would narrow that recommendation to just BuildRoot TBH. It
| fits much more with the kernel-config style whereas Yocto
| config seems like its from another planet.
|
| BuildRoot is much easier to work with IMO and gives equivalent
| or better results.
| Sponge5 wrote:
| Don't know when's the last time you've played with Yocto, but
| at this point all you need is to learn how to use devtool and
| that automates everything for you.
| montecarl wrote:
| I have built several embedded linux distributions for raspberry
| pi using buildroot. I have had great luck with it. There is a
| bit of a learning curve, but that seems to be the case with
| most embedded devices. I would recommend anyone getting started
| to try it out.
| elcritch wrote:
| My favorite is using Nerves system that uses buildroot and
| boots directly to Elixir. Buildroot can be a bit of a pain
| but it's fun to get a 20 mb Linux image with a few commands.
| sigjuice wrote:
| A couple of suggestions for minimal systems for the Raspberry Pi:
|
| https://buildroot.org build the whole Linux system from source
|
| https://gokrazy.org pure-Go userland for your Raspberry Pi 3/4
| appliances
| 01100011 wrote:
| Don't forget Yocto(guide here: https://medium.com/nerd-for-
| tech/build-your-own-linux-image-...).
| synergy20 wrote:
| what's the usage of memory/storage comparing to buildroot for
| similar functionalities? I would expect something like 10x
| larger but I never measured it.
| sigjuice wrote:
| Sorry, I did not understand your question. What are the two
| things being compared?
| forty wrote:
| Maybe buildroot and gocrazy?
| Cyberdog wrote:
| Alternatively, if you're not stuck on Linux, just install OpenBSD
| or NetBSD. Both will try to install X by default, but you can
| easily disable this in the TUI installers. After a reboot you'll
| go straight to a shell with very few daemons running (and
| certainly no systemd). Install the packages you need and start
| the daemons you need from there.
|
| Imagine having the entire output of `top` fit in a terminal
| window with empty space at the bottom.
| [deleted]
| seemaze wrote:
| I do just this with Alpine Linux. I've been running it on
| Raspberry Pi Zeros with great success, and less than 18MB read
| only ramdisk.
| boring_twenties wrote:
| I really wanted to like Alpine, but I can't for the life of
| me understand why it wants to do a ramdisk instead of just
| plain old read-only filesystem?
| riobard wrote:
| RAM disk is way faster and more reliable than SD cards. You
| can even remove the SD cards after boot.
| boring_twenties wrote:
| Everything that's used should be cached in RAM, anyway.
| Everything that isn't used, well, I prefer that not to be
| in RAM. These things don't exactly have gobs of it.
|
| And why would I want to remove the SD card after boot?
| Now I can't reboot the system, and it won't reboot if it
| crashes?
| riobard wrote:
| > Everything that's used should be cached in RAM
|
| This isn't guaranteed, depending on many factors.
|
| > why would I want to remove the SD card after boot
|
| It's unnecessary to remove it. It's more about
| reliability. SD cards are notoriously unreliable, and it
| could potentially fail after boot. Coupled with the
| previous point, it has happened multiple times that I
| wanted to login to troubleshoot the system, but cannot
| because the basic system binaries like `ls` are not
| cached in RAM (as you said these are not used for the
| normal operation of these unattended systems).
|
| This is when RAM disk is very helpful: as long as the
| system is powered on, I have reasonable confidence that I
| can login and see what's going wrong. It's mostly because
| RAM is way more reliable, faster, and cheap enough to
| "waste" a few dozens megabytes for the entire base OS.
|
| Also there're diskless PXE systems where you just load
| the OS from network only once upon boot.
| sydthrowaway wrote:
| This is interesting. I have always worked with baremetal systems.
| Are there any good courses for understanding modern Embedded
| Linux standards who for shops who end up using a COTS dev board?
___________________________________________________________________
(page generated 2021-09-01 10:00 UTC)