[HN Gopher] Oasis: a small statically-linked Linux system
___________________________________________________________________
Oasis: a small statically-linked Linux system
Author : varbhat
Score : 179 points
Date : 2020-12-01 15:05 UTC (7 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| dry_soup wrote:
| I'll never use this but it's cool that it exists.
| anthk wrote:
| >Velox
|
| Arcan would be interesting on this, too.
| young_unixer wrote:
| > No package manager.
|
| > Instead, you configure a set of specifications of what files
| from which packages to include on your system
|
| Isn't that a just a declarative package manager?
| mforney wrote:
| I suppose you could argue that, but it is not a package manager
| in the traditional sense.
|
| My main point here is that once you build the system, there is
| no longer any notion of "package", just files that make up your
| root filesystem. There is no package database tracking which
| files came from which packages. Instead, if you want to
| add/remove/update a package, you rebuild the system with a
| different specification, and then sync the resulting tree it to
| /.
| haolez wrote:
| Is "velox" (mentioned in the article) a display server or a
| window manager?
| fbn79 wrote:
| Window manager I think. Is based on
| https://github.com/michaelforney/swc that is a Wayland
| compositor
| [deleted]
| herodoturtle wrote:
| A wayland window manager.
| chousuke wrote:
| As far as I understand it, that distinction doesn't really make
| sense with Wayland. The display server / compositor is also the
| "window manager".
|
| Client applications have their own buffer they draw on, and
| they negotiate the details of its handling with the compositor
| which manages the _composition_ of all client buffers (and
| other data, such as input events) to form the final result.
|
| I suppose it's technically possible to write a protocol that
| involves a third client process (a "window manager") to draw
| something around another client's buffer and tell the
| compositor how to position things, but that sounds like a
| nightmare to synchronize.
| Subsentient wrote:
| Honestly, I'm team dynamic linking. I prefer to have things
| clearly separated in functionality and easily upgradeable.
|
| Statically linking all the OS utilities to their dependency
| libraries, over and over again? Dear god that sounds awful.
| abainbridge wrote:
| > I prefer to have things clearly separated in functionality
| and easily upgradeable.
|
| What is your thinking here? Are you talking about a use case
| where you have an application that uses a library and a bug fix
| is issued for that library, so you want to just get the new .so
| for the library and have the application use it? That feels
| like a niche use case to me. Maybe there is another benefit, or
| this use case is more common than I realize. I'd be interested
| in further opinions/info.
| cestith wrote:
| That is not at al a niche use case, at least on a general-
| purpose system. Shared libraries on Linux, like most modern
| OSes, are also shared in resident memory among programs using
| them. For very common libraries like TLS, readline, or GTK
| that can be a big savings.
| bityard wrote:
| > That feels like a niche use case to me.
|
| I wouldn't call it a niche use case at all, it's exactly how
| the software for all major Linux and BSD operating systems
| are packaged and built.
|
| For example, when (not if) a vulnerability is found in
| OpenSSL, package maintainers just commit an update to the
| openssl package and users only have to download and install
| one package to patch their system against the vulnerability.
|
| Contrast with a fully-static OS and applications: the
| maintainers would have to update the openssl package, rebuild
| it, and then also rebuild _every single other program_ that
| relies on it as well. On my system, that's 123 packages. I
| don't have all of those installed, but as a user, it means I
| would have to download new full copies of around a dozen
| packages to patch one flaw in one package.
|
| And of course, the packages themselves are much _much_ larger
| too. Most software that I can install as a flatpack or
| appimage are at least an order of magnitude larger than the
| deb version, take longer to start, and also take up more
| valuable RAM when running.
|
| Proponents of static linking have generally experienced the
| pain of dynamically-linked software either through container
| deployment or by accidentally messing up their system with
| third-party packages and repos and the like. I get it, I have
| been there too. But even through managing a dynamically
| linked system is certainly more complex, it brings a lot of
| benefits to the table.
| abainbridge wrote:
| OK, that makes sense. I error in my head was imagining a
| situation where the user was responsible for knowing which
| openssl .so file to download and where to put it.
| jcelerier wrote:
| > I wouldn't call it a niche use case at all, it's exactly
| how the software for all major Linux and BSD operating
| systems are packaged and built.
|
| even though I'm typing this on a linux desktop, this is a
| niche use case. In Windows, macOS, iOS and Android, so
| 99.9% of all user-facing computing, all the libraries
| except the OS API are duplicated for each app.
| tonyedgecombe wrote:
| I think the license for the Microsoft C runtime dll's
| precludes bundling although it does allow static linking.
| It used to cause me no end of problems because the silent
| install would often fail.
| mforney wrote:
| > Statically linking all the OS utilities to their dependency
| libraries, over and over again? Dear god that sounds awful.
|
| Just curious, why does that sound awful to you? Are you worried
| that it will take a long time to relink the binaries? The oasis
| build system is incremental, so updates to a library only
| involve relinking dependent binaries, which is quite fast. Even
| a full rebuild from scratch only takes a few minutes (assuming
| you have the sources downloaded already).
|
| If it's just the idea of relinking all the dependencies over
| and over again, note that with dynamic linking you do this at
| runtime every time you execute a binary.
| cestith wrote:
| For a system I'm going to keep and use in multiple ways,
| dynamic linking is wonderful. Save time building and upgrading,
| and save memory using the various apps.
|
| For a base image for VMs or containers I'm going to build
| automatically and throw away on demand anyway, a slightly
| longer build time to statically link things won't hurt. The
| memory utilization penalty should also be small, because those
| should only be running one application or a couple at most.
| AnIdiotOnTheNet wrote:
| There's no reason that there needs to be such extremes as
| either all statically linked or all dynamically linked. History
| has proven that dynamic linking causes a large number of
| headaches, and static linking has drawbacks as well. There is a
| reasonable middle ground: base system functionality that is
| utilized by most software (stdlib, cryptography, networking,
| gui, etc) should be dynamic and everything else should be
| static.
|
| The problem is that Linux userspace has historically payed
| little attention to backwards compatibility and has no
| definition of "base system". Consequently, Linux distros tend
| to be their own little mutually-incompatible worlds where you
| either play with the package manager/repo combo you have or
| jump through hoops and dodge conflicts to try to get anything
| to work.
|
| Oasis is interesting in this way: there are no conflicts, there
| is no package manager, and the binaries should work anywhere
| provided the arch is the same.
| Wowfunhappy wrote:
| > There is a reasonable middle ground: base system
| functionality that is utilized by most software (stdlib,
| cryptography, networking, gui, etc) should be dynamic and
| everything else should be static.
|
| But isn't this the exact opposite of what Oasis provides?
| AnIdiotOnTheNet wrote:
| Oasis is just the opposite extreme of how most Linux
| distros work. That's interesting because no one else does
| it. I did not say it was the ideal.
| nycticorax wrote:
| This 'middle ground' you describe is essentially how macOS
| and Windows work, isn't it?
| Subsentient wrote:
| >>There is a reasonable middle ground: base system
| functionality that is utilized by most software (stdlib,
| cryptography, networking, gui, etc) should be dynamic and
| everything else should be static
|
| I disagree, I tend to want the opposite. I want big, bloated
| libraries like Qt or Gtk or Boost to be dynamically linked,
| and OS facilities like libc to be statically linked.
| AnIdiotOnTheNet wrote:
| > I want big, bloated libraries like Qt or Gtk or Boost to
| be dynamically linked
|
| These fall under "GUI" in the "base system" category in my
| opinion.
|
| > and OS facilities like libc to be statically linked.
|
| If you're not being sarcastic right now, I'm really curious
| as to your reasoning.
| [deleted]
| AndyKelley wrote:
| How does graphics drivers work on this distro?
| mforney wrote:
| There is rudimentary hardware acceleration for certain older
| GPUs that I own in wld (intel at
| https://github.com/michaelforney/wld/blob/master/intel.c and
| nouveau
| https://github.com/michaelforney/wld/blob/master/nouveau.c).
|
| But, for everything else there is only software rendering via
| pixman. I started working on a new library called libblit that
| will support amdgpu, and I managed to draw some rectangles and
| textures, but there is still a ways to go on that:
| https://git.sr.ht/~mcf/libblit/tree/master/amdgpu
| Hello71 wrote:
| it's advertised as "small". how small is that? an Alpine
| minirootfs is ~2.5 MB tar.gz and ~6.0 MB on ext4 (4k overhead).
| yuribro wrote:
| This looks like a desktop system, with a browser and window
| manager. So comparing it to the alpine minirootfs is apples to
| oranges
| [deleted]
| pelasaco wrote:
| build manifests generated by Lua scripts look so elegant and
| clean.
| enriquto wrote:
| This makes me so happy!
|
| If static linking was ubiquitous, we could have avoided the
| complex craziness of docker and the like.
| remexre wrote:
| Well, you'd also need to fix Python (, Go, Ruby, JS, ...)
| packaging and delivery too...
| enriquto wrote:
| so it's a step in the right direction, then; bravo Oasis
| Linux!
| waynecochran wrote:
| All README's should begin with some sort of mission statement.
| What is the goal or reason for the existence of this? I can
| imagine some, but I am not sure where this is heading.
| mforney wrote:
| It exists because I wanted a complete system where I could
| easily experiment with various minimal/alternative software. I
| wanted this so that I could easily understand and hack on any
| component without spending weeks/months familiarizing with
| massive code-bases and waiting for things to compile.
|
| I made it for myself, and honestly that's all there is to it. I
| tried to describe the things that make oasis unique in the
| README, and I think some of those are pretty cool, but I'm not
| trying to market this to anyone. If it doesn't sound
| interesting to you, then feel free to disregard the project.
| waynecochran wrote:
| I definitely do not want to disregard. It think its cool. I
| just wanted to know what the target was and I think its
| helpful for README's to include their purpose.
| warpech wrote:
| Except that the README of this project answers all your
| questions. A list of distinctive features and a list of
| project's principles make up half of the README.
| waynecochran wrote:
| I read it. It does not say "why" anywhere.
| grafelic wrote:
| Why care about "why" when there is "why not".
| DoofusOfDeath wrote:
| > Why care about "why" when there is "why not".
|
| Life is short. As I've gotten older, I've seen the
| importance on focusing my time and attention on what
| matters.
|
| So, personally, I'd want to know "why" so I could decide
| where the project ranks in my priorities.
| yjftsjthsd-h wrote:
| Okay, how about this: _Why_ are you on Hacker News? It
| clearly doesn 't align well with your self-professed
| interests.
| [deleted]
| 0xbadcafebee wrote:
| This would be 10x faster and smaller if they just built these
| components into Busybox.
|
| Package management, library management, etc are solved problems
| when it comes to Linux distros. The only practical improvement
| you can make is containers (or similar).
|
| Static binaries cannot deal with external dependencies, and many
| applications _require_ external dependencies that _cannot be
| compiled in_. But even by trying to compile-in all the
| dependencies, you 've only shifted the complexity from the
| filesystem to the build system, and you still have applications
| with incompatible features and interfaces across versions.
| AnIdiotOnTheNet wrote:
| > Package management, library management, etc are solved
| problems when it comes to Linux distros. The only practical
| improvement you can make is containers (or similar).
|
| To me this basically reads as "we added a bunch of complexity
| on top of the problem without actually solving it". Package
| managers are supposed to prevent conflicts and manage giant
| dependency graphs, but they're so shit at doing that while
| actually allowing people to use whatever software they want
| that we've invented containers to keep everything separate
| again.
| marttt wrote:
| I might be mistaken, but is Oasis also somehow related to the
| suckless [1] community? (Edit: yes, the maintainer seems to be
| part of it.)
|
| Another static distro by the suckless people is stali (static
| linux) [2].
|
| 1: http://suckless.org
|
| 2: https://sta.li
| fmakunbound wrote:
| Ok maybe this is dumb question and is addressed somewhere: If a
| security problem is found, e.g. in muscl (the C lib), then is the
| user supposed to rebuild everything that statically linked it??
| mforney wrote:
| Yes, but with the oasis build system this is just a single
| command to do an incremental build that relinks your binaries
| and takes a matter of seconds.
|
| The user experience is essentially the same as updating your
| system on a dynamically-linked system. There is no "partial
| build" where some binaries get relinked but not others.
| rini17 wrote:
| Not a dumb question just there is always a catch: dynamical
| link saves you from rebuild only if the fix is carefully
| backported (by a distributor), which can introduce issues, too.
| The upstream has usually moved on in the meantime and the
| library isn't binary compatible.
| qz2 wrote:
| Yep. And also because it's statically linked and optimised,
| which version of musl that ships in that binary may be
| obfuscated somewhat. And no, I don't get it either.
| HALtheWise wrote:
| This seems to "statically link" binaries in the sense that each
| program it ships with is a standalone binary. Does anyone know
| whether it's possible to truly statically link an entire Linux
| install, in the sense that the whole system is a single
| statically linked file including the kernel, display manager, web
| browser, etc so that link-time optimization can deduplicate code
| across the entire system?
| Subsentient wrote:
| Yes, I know that it's NOT possible. Anything that uses the same
| libraries, e.g. GTK+, will end up calling gtk_init() multiple
| times. Callback functions with the same names will collide at
| link-time.
|
| Despite the madness of the code that would be generated if the
| compiler allowed it, since it doesn't, you can expect a list of
| linker errors so long, that you can go to bed and wake up in
| the morning and it's still spewing to stderr.
| nybble41 wrote:
| It could be done. Obviously you would only run one of the
| main functions (via a wrapper) per process invocation and you
| would need to rename conflicting global symbols at some point
| before the link step, but this is basically how tools like
| busybox work. If the application uses global constructors or
| init functions then those would require special handling
| since they bypass main--C tends to work better than C++ in
| this regard. Applying LTO to the resulting code for inter-
| application optimization would be an interesting twist.
| Subsentient wrote:
| It could be done if you _heavily_ patched everything you
| linked in, or developed an entirely new compiler, or both,
| but as-is, no.
| dleslie wrote:
| You're asking for something similar to a Lisp machine, or
| perhaps even an Amiga. Everything runs in the same address
| space and has access to all available library functions;
| "processes", such as they are, are in no way isolated.
| m463 wrote:
| > Does anyone know whether it's possible to truly statically
| link an entire Linux install
|
| I think if you give busybox enough time, it might happen.
|
| That's if systemd doesn't get there first, of course...
| ldng wrote:
| The answer is probably downvoted because of the systemd
| comment but I think that busybox is statically linkable, is
| it not ?
| yjftsjthsd-h wrote:
| It absolutely is; busybox is quite happy to be built into a
| single binary with no outside dependencies. Worked example:
| https://hub.docker.com/_/busybox
| birktj wrote:
| I don't know if this is what you are thinking of, but there is
| this concept in embedded systems sometimes refereed to as
| library operating systems [1]. My understanding is that in this
| model the operating system is implemented as a library and you
| your system would consider of one application that uses the os
| library to do tasks usually handled by the operating system.
|
| [1]: https://en.wikipedia.org/wiki/Operating_system#Library
| cyphar wrote:
| This is kind of the idea behind unikernels. There are
| experimental Linux-based unikernels, but the more mature
| unikernels are custom.
| asveikau wrote:
| I am not sure what you mean.
|
| The Linux kernel has dynamically loaded modules but you can
| build modules as statically part of the kernel. 'Y' instead of
| 'M' in the config.
|
| It doesn't make sense to "statically link" a kernel to a
| program, or multiple programs together, which I read your
| comment to suggest. That would be a different thing than the
| term "statically link" refers to.
| AnIdiotOnTheNet wrote:
| > It doesn't make sense to "statically link" a kernel to a
| program
|
| That's kinda what a Unikernel is.
| [deleted]
| singron wrote:
| I think they might be referring to a unikernel.
|
| https://en.m.wikipedia.org/wiki/Unikernel
|
| In userspace, you can do what busybox does and link a bunch
| of programs together with a single main function that decides
| which program to actually run.
| mittermayr wrote:
| Would be cool to get a bit of a "why this matters" intro on repos
| like this. I clicked through the details, but left wondering if I
| have any potential use for this. Can I compile this onto a USB
| stick and use it as a throw-away boot Linux for maintenance
| tasks? Can I cross-compile this for embedded devices? What is the
| statically-linked advantage here? Not trying to minimize the
| effort, I would actually love to see more such things, but felt a
| bit left out as I couldn't figure out why exactly it exists.
| gameswithgo wrote:
| As someone who uses linux casually, I often experience pain
| when I download a program, try to run it, and it fails because
| some dependency is not installed or is not the right version.
| So you try to figure out how to install it and it in turn needs
| a couple of things.
|
| All of this works fine if whatever package manager your distro
| uses has the program you want, but the package managers don't
| have evertying!
|
| So I'd love an ecosystem where things tend to be statically
| linked unless there is a good reason not to.
| Wowfunhappy wrote:
| > So I'd love an ecosystem where things tend to be statically
| linked unless there is a good reason not to.
|
| Me too, but I'm not sure this project brings us closer to
| that goal.
|
| I don't need the base OS to be statically linked--in fact,
| that's where static vs dynamic linking matters _least_ ,
| because it all comes preinstalled. What I want is for
| everything _else_ which I may want to install on top to be
| available as statically-linked binaries.
| an_opabinia wrote:
| Since most common applications have moved to the web, and
| the biggest Linux end user deployment by far, Android, was
| created from fresh beginnings, the idea of the user being
| concerned with minutia of binaries is already far along a
| long road to the grave.
| krageon wrote:
| This right here is what's wrong with the application
| space. The idea that this could be true leads to terrible
| software design and slow, bloated applications. In the
| current landscape is it perfectly possible (and I'd say
| even usual) to do most of your work and processing on
| local system binaries (that aren't a browser or a browser
| with a different name).
| an_opabinia wrote:
| It's no one's fault really. As long as people pirate
| software, it is economically Darwinian / inevitable that
| all commercial software will migrate to centrally
| controlled subscription gated servers. This includes open
| source software, if you correctly count how much of it is
| repackaged and sold to people this way.
| Wowfunhappy wrote:
| Eh, but if that was completely true I'd be using
| ChromeOS, or something similar. Perhaps some day I will--
| but as of 2020, I still find myself popping into a lot of
| native apps day-to-day.
| jcelerier wrote:
| 99% of the apps I use aren't on the web at all
| 1996 wrote:
| Check out kiss linux, another interesting project:
| k1ss.org/install
|
| I'm considering trying it: having the simplest distribution
| baremetal, and keeping the complexity "outside" and
| "contained" (docker for servers, flatpack for desktops and
| laptops, etc.) makes sense to me
| gameswithgo wrote:
| This is true, though perhaps there is some performance
| benefit.
| arendtio wrote:
| > So I'd love an ecosystem where things tend to be statically
| linked unless there is a good reason not to.
|
| Well, every time there is a bug fix in one of your
| dependencies you/your distribution would have to recompile
| everything that depends on it. Which will use quite some
| resources.
|
| So to actually use this you would have to reinstall (nearly)
| all programs once a week or so (no matter if source or binary
| distribution). While that is quite possible with modern
| internet connections, I am not sure if it solves all the
| problems you might think it solves. After all, even nowadays
| application authors could provide static packages, but many
| don't do it.
|
| After using quite a few distributions over the years, I like
| Arch Linux best at the moment, as the combination of binary
| distribution, with frequent updates, and the source based
| AUR, with the very large amount of packages, suits my needs
| very good.
| CyberDildonics wrote:
| Letting software have dependencies then making the user
| manage them in the off chance there is some bug in the
| foundational mostly frozen libraries should be an obviously
| bad decision at this point.
|
| No other environment forces this kind of complexity on
| their users and almost no non-enthusiast user wants to deal
| with this structure.
| zarkov99 wrote:
| > Well, every time there is a bug fix in one of your
| dependencies you/your distribution would have to recompile
| everything that depends on it. Which will use quite some
| resources.
|
| This is far for more clear cut. Often some dependency bugs,
| might not impact my program. Even worse, sometimes by
| fixing a bug my program does not care about, a dependency
| might break my program in some subtle way. The author of
| the program is the best qualified person to make the
| upgrade/not upgrade decision.
| throwaway894345 wrote:
| You won't have to recompile everything if you cache your
| build artifacts (e.g., object files); you'll only have to
| _relink_ which is quite a lot better.
| nmz wrote:
| So long as I'm not the one building it, its fine. My
| internet can redownload everything. It also depends on what
| libraries are being used. musl does generate smaller
| binaries than glibc. and if you look at the massive project
| that is sqlite, building software _should_ only take a
| couple of seconds. Of course, whether anyone actually goes
| to such lengths for easy compilation is another matter
| entirely.
|
| https://sta.li/
| gameswithgo wrote:
| >Well, every time there is a bug fix in one of your
| dependencies you/your distribution would have to recompile
| everything that depends on it
|
| Well no, I could opt not to apply the bug fix if it isn't
| affecting me. This again is sort of serve mentality, where
| of course you want to get a bug fix in as quick as possible
| in case it is a security issue. A desktop user, maybe you
| don't.
|
| >After all, even nowadays application authors could provide
| static packages, but many don't do it.
|
| Right, and really what I would like is for application
| authors to just static link more, more often, on linux. Or
| at the least ship their programs with installers that apply
| the needed deps if they are not already there! That the OS
| itself is statically linked doesn't really matter, but
| perhaps it would inspire an ecosystem that does this.
|
| > I like Arch Linux best at the moment
|
| Probably suggest arch linux to someone complaining about
| having to track down dependencies does not make sense,
| since such a person is more interested in the OS _just
| working_ rather than making the OS a bit of a hobby that
| that they want to customize.
|
| Which, is why I mostly use windows.
| pixl97 wrote:
| >Which, is why I mostly use windows.
|
| Oh, Windows can have dependency hell. And even worse is
| when you have a WinSXS blowout and you're trying to
| figure out why that subdirectory is using up 80GB of
| space for an unknown reason.
| jbverschoor wrote:
| DLLHell
| arendtio wrote:
| > Probably suggest arch linux to someone complaining
| about having to track down dependencies does not make
| sense, since such a person is more interested in the OS
| just working rather than making the OS a bit of a hobby
| that that they want to customize.
|
| Sure, in the beginning you have to spend some time to set
| it up, but due to the rolling releases it has a great
| longevity. And talking about application integration, I
| prefer the package management style (even through AUR)
| greatly to the chaotic download some binary and execute
| it way windows does it. Over time your windows always
| gets so messy because every device manufacturer seems to
| think that it is cool, to bundle his device driver with
| hundreds of megabytes of non-sense-ware (last weekend I
| installed some Logitech driver on Windows, because the
| default Windows driver didn't work (on Arch I didn't have
| to install anything extra, just saying... yes, anecdotal
| evidence)).
| jcelerier wrote:
| > Probably suggest arch linux to someone complaining
| about having to track down dependencies does not make
| sense, since such a person is more interested in the OS
| just working rather than making the OS a bit of a hobby
| that that they want to customize.
|
| having used Red Hat, Mandrake, Ubuntu, Debian... Arch
| Linuxis by far the most "just works" distro I've used
| boogies wrote:
| > Probably suggest arch linux to someone complaining
| about having to track down dependencies does not make
| sense, since such a person is more interested in the OS
| just working rather than making the OS a bit of a hobby
| that that they want to customize.
|
| Then recommend Manjaro, not an OS that "just" loads them
| down with ads, tracking, forced updates, and more unless
| -- and to some extent still even after -- they customize
| it.
| Shared404 wrote:
| At the risk of starting a distro war in the comments,
| I'll have to recommend Pop!_OS over Manjaro.
|
| Pop has broken itself less often then Windows for me,
| while Manjaro has broken itself slightly more.
| retrac wrote:
| > every time there is a bug fix in one of your dependencies
| you/your distribution would have to recompile everything
| that depends on it
|
| Back in the dark ages, some mainframe operating systems
| like MVS normally stored executables as object files, and
| would link the program every time you executed it.
|
| Has the on-disk space savings and updateable libraries of
| dynamic linking, without most of the complexity. I believe
| the original motivation was to allow an executable to run
| at an arbitrary address on systems without virtual memory
| though.
| 1996 wrote:
| I would be surprised if there was not a way to take a
| dynamic executable, "add in" all the required libraries,
| and return a fat static executable
| AnIdiotOnTheNet wrote:
| If you know of such a utility I'd like to see it, because
| I've looked and never found it. The closest I've seen are
| things like AppImage.
| 1996 wrote:
| Shell scripting and ld tools should allow you to make
| that yourself.
|
| EDIT: people are apparently entitled to solutions for
| their problems...
|
| > Care to elaborate?
|
| > Show us a script then! Your internet fame awaits.
|
| I'm not interested in fame, or in reinventing the wheel.
|
| Apparently, you are unfamiliar with this process, so
| TLDR: there are basically 2 approaches, in order of
| complexity: 1. working with ldtools or 2. running the
| executable to find the various parts then dumping a
| binary, (3 if you consider putting the libraries in a
| directory then run with LD_LIBRARY_PATH=/thisdir
| myexecutable)
|
| Let me google some existing tools for you:
|
| https://github.com/systems-nuts/dynamic_to_static
|
| http://bitwagon.com/jumpstart/jumpstart.html
|
| http://statifier.sourceforge.net/
|
| https://www.ucc.asn.au/~dagobah/things/make-static.html
|
| https://web.archive.org/web/20130114222319if_/http://dago
| bah... with patch and explanation on
| https://stackoverflow.com/questions/17390722/how-can-i-
| conve...
|
| A better approach from 15 years ago is slinky, so that
| you can "update" the libraries if needed: https://www.use
| nix.org/legacy/publications/library/proceedin...
|
| There's nothing fancy in there. It just requires some
| effort.
| AnIdiotOnTheNet wrote:
| Care to elaborate? Because I'm not aware of any tool that
| could be used to combine a binary and all its .so
| dependencies (and all of theirs recursively) into a
| single statically-linked binary.
| dTal wrote:
| Show us a script then! Your internet fame awaits.
| efrecon wrote:
| I have used exodus to do this in the past.
| https://github.com/intoli/exodus
| AnIdiotOnTheNet wrote:
| > Exodus handles bundling all of the binary's
| dependencies, compiling a statically linked wrapper for
| the executable that invokes the relocated linker
| directly, and installing the bundle in ~/.exodus/ on the
| remote machine. You can see it in action here.
|
| That isn't really what is being described, or at least
| not what I'm referring to. It's more like a self-
| extractor.
| throwaway894345 wrote:
| It exists. I came across at least one such project last
| year, but I don't recall the name. :(
| robocat wrote:
| It was for personal operating systems (and ancient
| mainframes) that had a multitasking OS but without any
| MMU support in the hardware. Relocatable executables were
| required so that you could load multiple executables at
| different addresses within the single address space of
| the memory. This is now called Position Independent
| Code[1].
|
| Modern MMUs meant that the OS would define specific
| addresses for use by a program, so for example all ELF
| programs in a distribution would have the same starting
| address in memory hard coded into the executable (one
| virtual address within a process is mapped to a different
| physical memory address for each process by the MMU).
|
| [1] https://en.wikipedia.org/wiki/Position-
| independent_code
| Someone wrote:
| There's a difference. Position independent means you can
| move the executable anywhere and run it.
|
| Relocatable means the binary has enough information to
| fix any absolute addresses in the code after that move.
| That's extra work, and means you cannot have the same
| physical memory mapped to different virtual addresses in
| different processes, and that you have to either swap out
| the relocated memory, or repeat the relocation when you
| swap in the code again (I'm not aware of any OS doing the
| latter, and don't see a nice way to do it, but it is a
| possibility).
|
| https://en.wikipedia.org/wiki/Position-independent_code:
|
| _"In computing, position-independent code[1] (PIC[1]) or
| position-independent executable (PIE)[2] is a body of
| machine code that, being placed somewhere in the primary
| memory, executes properly regardless of its absolute
| address."_
|
| https://en.wikipedia.org/wiki/Relocation_(computing):
|
| _"Relocation is the process of assigning load addresses
| for position-dependent code and data of a program and
| adjusting the code and data to reflect the assigned
| addresses."_
| ChiefOBrien wrote:
| Hot take: A great tragedy of the GNU/Linux ecosystem is the
| fact that ABI+API is still not figured out, and the most
| common interface between programs is the C ABI, which is
| embarassingly primitive even by 80's/90's standards. Some
| people in the FOSS community just want to leave things as-is
| to hinder propiertary software, and it's the same story with
| device drivers. You can debate the merits rightfully so, but
| then there's still companies pushing out binary blobs which
| break every few kernel updates. As a FOSS developer it's an
| eternal fight between good and evil with no winner in sight,
| as a propiertary developer it's pain in the ass to maintain
| old and new software between all the permutation of Linux
| distros, and as a user I get to cry because of the lack of
| popular software and backwards compatibility. Snaps and
| flatpaks are an ugly hack, literal ductape around this very
| fundamental problem, and clearly not the solution. GNU/Linux
| should have adopted COM and/or an adequete proglang with
| stable ABI a long time ago, and should have tried to control
| wasted effort put into duplicate software ecosystems (KDE and
| Gnome).
| Spivak wrote:
| As a primarily Linux developer I'm super jealous of COM. It
| reminds me a lot of dbus with more legacy cruft but also a
| lot better designed.
| qppo wrote:
| That's a searing hot take, to be sure.
|
| There is no alternative to the C ABI. If you look at stable
| ABIs in other programming languages, they're either
| identical to or thin shims around the C ABI! Even COM, with
| the caveat of calling convention on MSVC.
|
| Regardless of ABI stability you also need software to have
| API stability for stable distribution. The two solutions to
| this problem are to statically link everything or package
| your shared objects with your software. Then to launch the
| software you need a shell script that overrides the loader
| search path and move on with your life.
|
| The ultimate solution is the .app/.bundle paradigm of
| MacOS. Everything should copy that.
|
| The other issues of packaging software like
| entitlements/capability based security and code signing are
| similarly orthogonal. If you're shipping big software
| today, it needs to have everything it needs and to ignore
| whatever is on the user's system unless they explicitly
| request an override.
|
| The real tragedy of GNU is that the interchangeability of
| software was always possible, package managers just managed
| to make it accidental and explicit. Too many foot guns
| there.
| tonyedgecombe wrote:
| Actually that's one problem I've never had with Linux. Even
| using stuff that isn't in the default package manager seems
| to just work.
| kevin_thibedeau wrote:
| If you need exotic libraries, you install them into
| /usr/local/lib with GNU Stow for secondary package
| management. Then point your loader there first when starting
| programs that depend on them. Your distro stays pristine and
| the program runs so everything is good.
| zamadatix wrote:
| It seems like your definition of "why this matters" is "how can
| this pick me up as a user" but for some projects "why this
| matters" is "to see if this works as well as we think it could"
| or similar more abstract goals. If the goal of the project is
| to try to be as small and simple of a Linux distro as possible
| by trying a different take and seeing how it pans out (which it
| seems like this is) then it's accomplishing it's goals whether
| or not it tells you how you could use such a distro were it to
| pan out.
| breakfastduck wrote:
| I don't think you're disagreeing with each other - a bit of
| intro text especially helps if the project is abstract.
| mrtweetyhack wrote:
| So this doesn't matter
| colonwqbang wrote:
| I find the build system more interesting than the "statically
| linked" part. Incremental compilation across your whole system is
| something we should have nailed by now, I think. The best I've
| seen is something like openembedded, where you get per-package
| (but not per-file) dependency tracking across the whole system.
|
| I was ever so slightly disappointed to see how manual the
| packaging is, with every .c file listed in each lua script. It
| looks quite maintenance-intensive. I was almost hoping for some
| kind of meta-build system which could parse automake etc. files
| and hoist the dependency graph into the main system build.
| mforney wrote:
| > I was ever so slightly disappointed to see how manual the
| packaging is, with every .c file listed in each lua script. It
| looks quite maintenance-intensive.
|
| I was worried about this, too, but it turns out most of the
| cost in just the initial packaging. Packages do not tend to
| change their build system very much in between releases, so
| updating is usually just `git diff` between the version tags,
| sometimes adding/removing a couple source files to `gen.lua`,
| and regenerating `config.h`.
|
| I'm just one person, but I've been able to keep these 100 or so
| packages up-to-date fairly easily, while still spending most of
| my time developing my own projects.
|
| > I was almost hoping for some kind of meta-build system which
| could parse automake etc. files and hoist the dependency graph
| into the main system build.
|
| This would be amazing, but I think it is too difficult a
| problem to solve, especially with the huge variety of build
| systems (which are sometimes used pretty creatively). For some
| packages I get part of the way there with scripts or command
| snippets to extract source lists from the upstream build
| system. Here's an example:
| https://github.com/oasislinux/oasis/blob/master/pkg/mpv/gens...
|
| One of the main motivations for the oasis build system is that
| it is often very difficult to get the upstream build system to
| do what you want. It is very common for projects to ignore your
| CFLAGS or LDFLAGS, ignore special include/lib directories for
| dependencies, or have automagic dependencies (to borrow a term
| from gentoo) that can't be enabled/disabled explicitly with a
| configure switch. Also, most build systems don't work very well
| for building things statically. libtool will intercept your
| -static flag, hiding it from the compiler, and libraries which
| are dependencies of other libraries get left out from the
| linking command.
| siraben wrote:
| Does statically linking everything not lead to much higher disk
| usage, espcially when you have thousands of binaries? Drew
| Devault has an analysis[0] that appears to claim otherwise.
|
| [0] https://drewdevault.com/dynlib.html
| mforney wrote:
| The idea with oasis is to choose smaller software with fewer
| dependencies that offsets the slightly higher disk usage, as
| well as to deliberately not have thousands of binaries.
| tetraca wrote:
| It might have been a concern long ago when space was a premium,
| but these days in the world of >=1TB drives you could probably
| build your entire software library in the most horrifying,
| space inefficient way, and it would still be completely and
| utterly dwarfed by the user's media library. Excepting games, I
| would bet most anyone's software library would likely
| comfortably fit under 128 GB. The photos, games, music, and
| video would be what necessitates more. And I suspect even then,
| a lot of people don't bother to store these things locally
| besides the games and photos.
| pjmlp wrote:
| Having grown up on a static linked world, where dynamic linking
| was a thing of big iron computers that we dreamt about being able
| to do in home computers this trend of static linked compiled
| stuff feels somehow a tragedy of some sort.
|
| How bad have we gone, that is now trendy to return to the days of
| static compiled binaries and process IPC to achieve any sort of
| dynamism.
| whizzter wrote:
| I think dynamic linking one of those ideas that looks neater on
| paper and therefore feels better but just fails in many ways in
| the real world (at least for statically compiled languages like
| C and C++).
|
| Personally i think at least half the times i really tried to
| use Linux i always wanted a new shiny version of some program
| at some point, this usually devolved into a more or less broken
| system due to dependencies.
|
| I do understand some people who really loves their package
| managers and knowing that their updated system will be secure,
| but many people are even more into shiny new things than me and
| if anything has held back Linux adoption on the desktop I'd
| probably point to this second to drivers and configuration
| troubles historically.
|
| Am I alone in this? I doubt it if we consider the existence of
| Docker, Snap packages and similar things.
| pjmlp wrote:
| Containers fulfil other purpose, and have existence in some
| form on mainframes and big UNIXes, because just using UNIX
| permissions is not enough to actually secure a process.
|
| They just happen to have been misused to workaround
| deficiencies on Linux software distributions.
|
| The only areas where I see a failure of dynamic linking is
| security, given the possible exploits of the host process.
|
| Even IPC comes to IPC hell if not everyone is speaking the
| same version.
___________________________________________________________________
(page generated 2020-12-01 23:00 UTC)