[HN Gopher] Haiku package management
___________________________________________________________________
Haiku package management
Author : julicen
Score : 182 points
Date : 2023-02-21 10:29 UTC (12 hours ago)
(HTM) web link (www.markround.com)
(TXT) w3m dump (www.markround.com)
| haunter wrote:
| Honestly that's incredible. Is there anything close to it in the
| Linux world? Or I guess, I'm not an expert, that's what NixOS is
| trying to solve?
|
| >Just because they never got wide adoption doesn't mean they were
| wrong
|
| This is actually something that I was thinking about recently,
| applies to a lot of topics not just OSes
| outworlder wrote:
| Yeah, although Haiku seems to solve this problem more elegantly
| than Nix(the package manager) does(aka without a billion
| links). It is still great and I use it across many machines,
| including on OSX.
|
| Another thing BeOS did well (and it's still ahead) was the
| filesystem. The mail program was implemented directly in the
| filesystem: https://arstechnica.com/information-
| technology/2018/07/the-b...
|
| At one point in history, there was a chance that BeOS would be
| adopted as the Mac OS. I wonder how that fork in the road would
| look like.
| pxc wrote:
| > Haiku seems to solve this problem more elegantly than
| Nix(the package manager) does(aka without a billion links).
| It is still great and I use it across many machines,
| including on OSX.
|
| If Nix's solution were more elegant in this sense, you
| wouldn't be able to use it on macOS because macOS lacks the
| required sandboxing features.
| alxmng wrote:
| You're right. That's what NixOS is, a Linux distro where all
| software is managed by Nix, so the users environment is fully
| compartmentalized, declarative, and reproducible.
| MuffinFlavored wrote:
| > so the users environment is fully compartmentalized,
| declarative, and reproducible.
|
| I have a question about this, please forgive my ignorance. I
| see this statement repeated a lot about NixOS and I've kind
| of had some "success" in the comments section being
| "critical" / "asking hard questions" about NixOS.
|
| If I have a build script that pulls + compiles + validates
| with hashes/checksums say, Linux kernel, some userspace
| tools, init system, etc. for a boring ol' distribution like
| Alpine or Debian, why isn't that classified as "reproducible"
| the same way NixOS is?
| edgyquant wrote:
| It would be, the problem is that you can not write such a
| script at the moment. As far as I know those checksums are
| almost always for the source code/file itself and not the
| compiled binary.
| yjftsjthsd-h wrote:
| The situation has improved, actually - you'd still have
| to do custom scripting to check the output checksums
| AFAIK but ex. https://tests.reproducible-
| builds.org/debian/reproducible.ht... tracks Debian's
| attempts to make it _possible_ to reproduce packages
| perfectly. It 's just that nixos made it pervasive and
| normal through the whole system.
| mikepurvis wrote:
| I think the main thing to understand is that Debian is
| reproducible as a consequence of significant unenforced
| contracts around what the pristine build environment will
| be, what the state of the package repo will be when
| dependencies are fetched and installed, and so on (see:
| https://www.debian.org/doc/debian-policy/ch-
| source.html#repr...). Your local dpkg-buildpackage
| invocation has pretty much a 0% chance of being a bit-
| for-bit reproduction of what you pulled from the archive
| unless you cooperate with all this ceremony.
|
| Nix, on the other hand, is reproducible _by default_. If
| you `nix build <nixpkgs/gitsha#pkgname>` you are
| automatically getting a local build that takes place in
| the same hermetic sandbox as the build you pulled from
| cache.nixos.org, and will be identical or very close to
| it. (Contrary to popular belief, not _all_ of nixpkgs is
| reproducible-- many packages require patching for various
| sources of nondeterminism, and this is an ongoing effort:
| https://github.com/NixOS/nixpkgs/labels/6.topic%3A%20repr
| odu...)
| pxc wrote:
| > If I have a build script that pulls + compiles +
| validates with hashes/checksums say, Linux kernel, some
| userspace tools, init system, etc. for a boring ol'
| distribution like Alpine or Debian, why isn't that
| classified as "reproducible" the same way NixOS is?
|
| It might be! Nix uses that very same strategy for enforcing
| reproducible outcomes when dealing with non-reproducible
| operations, like fetching things from the great mutating
| mass of the internet.
|
| I would say that Nix differs from that approach mainly in
| the way it divides up the problem. So-called binary package
| managers like you use on Alpine and Debian perform runtime
| dependency resolution against a mutating global namespace
| that usually lives on the internet-- the binary repos
| themselves. Generally, those repos include only one copy of
| each package. So to halt the forward march of all those
| versions, you have to mirror each repo of binary artifacts
| you're interested in using.
|
| With Nix, outcomes also depend on the state of the repo,
| but the repo is just a collection of build recipes which
| are the mapped onto their outputs in a single, huge binary
| cache. Every new build gets added to the same binary cache,
| for every Nixpkgs/NixOS release. It just grows and grows.
| And of course, Nix builds work even without enabling the
| binary cache at all.
|
| So to get the same kind of reproducibility out of many
| other package managers, you generally have to set up
| additional infrastructure.
|
| But if you do everything right, you can certainly have a
| fully reproducible OS image based on Alpine or whatever
| else.
| duped wrote:
| It's only reproducible if you can actually reproduce it in
| a different environment (different
| time/locale/machine/etc).
| yepguy wrote:
| Nix's main innovation with regard to reproducibility is
| that it actually fully specifies the dependency graph. I'm
| not going to say anything about bit reproducibility as
| that's another thing entirely, but virtually every other
| package manager and build system has holes in its
| dependency graph, coming in 2 main forms:
|
| 1. Implicit dependencies that are needed to build the
| software but not managed by the package manager or build
| system itself. For example, if your language package
| manager depends on a C library that it can't build itself.
| Or Linux distribution package managers that implicitly
| assume the presence of a "base system".
|
| 2. SAT solvers for version selection. Using one of these
| usually introduces an implicit dependency on the clock
| time, because your dependencies will resolve to different
| versions at different times.
|
| The reason your script wouldn't classify as reproducible is
| simply that you would probably fail to fully specify your
| dependencies. It's harder than you'd expect once you start
| moving up the software stack to complicated high-level
| applications.
| sparkie wrote:
| The problem might not be your script, but that the system
| is stateful. Users can access and mutate the /usr
| directory, and some software will do this outside of the
| system's built-in package manager. You can also compile and
| link against a shared library in /usr, but then later this
| library may be updated, so now your app links against a
| different library than the one you built against.
|
| Older package managers use a "name-version" scheme to try
| and avoid package conflicts, but in practice there can be
| different binaries which have the same "name-version"
| identifier and they're not always compatible.
|
| The way Nix solves the issue is to store everything in a
| content-addressible store `/nix/store`, where multiple
| concurrent versions can be installed without naming
| conflicts (the directories are named with the content
| hash). When an app is compiled, it is linked to specific
| shared libraries in the nix store which will always be
| present on the system. The `/usr` directory then, is just a
| collection of symlinks to the actual content in the nix
| store (most recent versions), for compatibility.
|
| Because derived packages are reproducible, they will always
| have the same content hash on every system on which they
| are deployed. As a developer then, you can target very
| specific shared libraries when building your software, and
| you know that when you deploy them, users are going to have
| the exact same dependencies you had when you built it.
| odo1242 wrote:
| many compilers aren't guaranteed to be fully deterministic
| tho (e.g. __DATE__ macro)
| iveqy wrote:
| See my comment about apkovl
| Gordonjcp wrote:
| It's kind of what snap is trying to be, but it's not there yet.
| pxc wrote:
| > Honestly that's incredible. Is there anything close to it in
| the Linux world?
|
| Yeah, this is similar in spirit to what Flatpak and Snap do.
| The latter even uses SquashFS images it mounts for every
| package.
|
| I have a feeling, though, that this is more similar to Distri1,
| where the package manager is a bit more traditional. That is to
| say, packages are fine-grained (not packaging large runtimes
| into single chunks, like Flatpak and Snap) and there's not as
| much special tooling as Flatpak has for redirecting inputs and
| outputs, sandboxing, etc.
|
| Distri also eliminates all hooks and triggers from packages,
| leading to super fast installs, which is pretty cool.
|
| > Or I guess, I'm not an expert, that's what NixOS is trying to
| solve?
|
| From the article, three key features highlighted are:
| > - Immutable system directories > - Rollback to
| previous states > - User-managed packages separated from
| system packages
|
| And, going by those criteria: yes. NixOS and GuixSD are two
| Linux-based operating systems which meet all three criteria
| today. One key difference is that Nix and Guix don't require
| namespacing at the filesystem, since they install everything to
| those wacky store prefixes with the leading hashes. Put another
| way: /boot/system on Haiku is a mountpoint, but /run/booted-
| system on NixOS is just a symlink, even though both serve the
| same function2.
|
| But spiritually I'd say the basic idea (avoid dependency hell
| through lightweight package isolation) is similar and so are
| the results, at least in terms of capabilities.
|
| PS: If you're interested in another example, RedoxOS' package
| manager also has a very similar design to Haiku's and Distri's,
| IIRC.
|
| --
|
| 1: https://distr1.org/
|
| 2: Because NixOS allows you to switch to a new system without
| booting it (i.e., it lets you upgrade without rebooting), it
| actually has two symlinks here: booted-system and current-
| system. The current-system symlink rotates as soon as you
| switch configurations, but booted-system gets set only on a new
| boot.
| guipsp wrote:
| > Is there anything close to it in the Linux world?
|
| Check out (as you mentioned) NixOS or Fedora Silverblue
| tmtvl wrote:
| And Guix and OpenSUSE MicroOS. There may be others, but those
| are the ones that immediately come to mind.
| seized wrote:
| Also Fedora Kinoite, the KDE flavor of Silverblue.
| ofalkaed wrote:
| Been watching Haiku for around a decade now, install it every
| year or so to give it a go and it is getting very close to being
| my primary OS. Great project with a good community, devs are very
| helpful.
| secure wrote:
| My Linux research distribution "distri" works the same way :)
|
| (I learnt about Haiku's similar approach after building distri.)
|
| https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...
|
| https://distr1.org/
| pxc wrote:
| I'm a big fan of distri! Really enjoyed your blog posts on its
| design and development. Hopefully I didn't misremember too much
| about it in these comments. :)
| [deleted]
| iveqy wrote:
| Looks a lot like apkovl
| https://wiki.alpinelinux.org/wiki/Alpine_local_backup for Alpine
| linux. Not the most easy to grasp documentation but it's
| basically a package (apk, similar do rpm or deb) that each has
| its own overlayfs.
| supriyo-biswas wrote:
| Interestingly, some niche Linux distributions, such as Puppy
| Linux and Porteus mount packages which are SquashFS filesystems
| at runtime, implementing exactly this paradigm long before Nix
| and the likes were around.
| faefox wrote:
| I'm genuinely so grateful that Haiku exists. The fact that a band
| of enthusiasts has managed to keep the spirit of BeOS alive after
| all these years is really inspiring. To me it's an echo of what
| felt like a more hopeful era in computing, when the internet
| still felt like a magical place.
| user3939382 wrote:
| I remember installing BeOS as a bare metal dual boot on my box
| circa 2001-2. My impression was: wow, this OS is fast, fun, and
| somehow endearing. I've not since had that experience with
| another OS.
|
| I'm also super excited to see Haiku mature. Especially once it
| passes that breakaway threshold where it's not primarily being
| run as a VM and the third-party ecosystem starts getting
| traction.
| mark_round wrote:
| Author of the article here. Thanks for posting this! There's also
| a really interesting thread on lobste.rs [1] where a Haiku
| developer talks more about some of their less user-visible
| innovations, compares the Linux development model to Haiku, and
| gives some background on the drama[2] caused by the adoption of
| their package management system.
|
| Well worth a read and fascinating (to me, anyway!) to see new
| ideas and approaches from the Alt-OS scene.
|
| [1]=https://lobste.rs/s/j9decl/haiku_package_management
|
| [2]=https://lobste.rs/s/j9decl/haiku_package_management#c_owqi3t
| sleepycatgirl wrote:
| That was good article :D
|
| Definitely going to look into Haiku more myself :)
| mark_round wrote:
| I highly recommend it! Even if you haven't had the experience
| or attachment to BeOS it's a refreshing and fun take on a
| desktop system. It just "clicks" with me in a way that
| Linux/Windows/Mac never did. It's truly impressive that
| they've been going for over 20 years now (I still remember
| the first screenshot of app_server drawing a window in the
| early 2000s) and have accomplished so much from scratch.
| Can't wait to see where the project will go next!
| HeckFeck wrote:
| Haiku and Serenity
|
| Two founts of hope
|
| For computing's future
| edgyquant wrote:
| Serenity is just a Unix clone and not one written in an
| interesting language either. Id put redox above it simply
| for being Rust based. Haiku is an entirely different
| paradigm.
| dmacvicar wrote:
| Serenity's starting point is very well described:
|
| "SerenityOS is a love letter to '90s user interfaces with
| a custom Unix-like core. It flatters with sincerity by
| stealing beautiful ideas from various other systems.
|
| Roughly speaking, the goal is a marriage between the
| aesthetic of late-1990s productivity software and the
| power-user accessibility of late-2000s *nix."
|
| I think in terms of paradigm, Serenity has a stronger
| ideology than RedoxOS. It brings BeOS and Mac concepts
| like a consistent and fully integrated monorepo system
| and development kits, and a very strong and defined user
| interface paradigm.
|
| From that perspective I find Redox much less of a
| paradigm for the end-user, if the main feature is the
| language the developers used. Orbital does not look
| remarkable.
|
| The level of productivity of Andreas himself, his videos
| and the community he managed to built, including growing
| a few individuals of to major contributors to the point
| they created a whole browser stack is just outstanding as
| an open-source case study. Also to me, much more
| interesting than a programming language used to built it.
|
| Haiku is written in the same language as Serenity, btw.
| pbowyer wrote:
| As a teenager reading "PC Pro" magazine BeOS was the
| operating system I was convinced was going to be the next
| thing and take over. Sad times!
| cmrdporcupine wrote:
| Is the package system there something that Haiku invented, or was
| BeOS like this as well? I had a BeOS install at one point in the
| 90s, but don't remember how that worked.
| mark_round wrote:
| It's a Haiku thing. Added around 2018 if I recall correctly -
| see the linked lobste.rs thread in another of my comments where
| a Haiku developer gives some (really interesting) background on
| the development and effects on the community.
| hbbio wrote:
| Another great feature of Haiku!
|
| I love playing from time to time with it. If there would be ARM64
| support (I wish I could find time to contribute...), combined
| with the progress of Asahi Linux, it would be fun (and a pun to
| history) to run Haiku on Apple M1s!
| threemux wrote:
| This reminds me of TinyCore Linux, which distributes packages as
| squashfs archives and dynamically mounts them at runtime with
| symlinks to the system directories. Pretty neat.
| dmacvicar wrote:
| Something few people know about: Haiku adopted openSUSE libsolv,
| built for zypper, to solve dependencies even before Fedora.
|
| libsolv, created by Michael Schoeder at SUSE, has two
| innovations:
|
| - a memory optimized format to represent packages and its
| dependencies, using hashed string pools - a SAT solver able to
| operate directly in this representation, battle tested in this
| particular scenario of complex upgrades, including hundred of
| testcases.
|
| Haiku uses mainly the solver only, as far as I saw in the source.
| darksaints wrote:
| Do you know anything more about the solver? The GitHub page
| doesn't really talk algorithms or anything. Does it use DPLL
| under the covers?
| dmacvicar wrote:
| You can start here:
|
| https://github.com/openSUSE/libsolv/blob/347200107f0f17a3697.
| ..
| [deleted]
| bravetraveler wrote:
| Keep in mind that 'even before Fedora' was ~8 years ago
|
| Not sure what my point is, I was just confused - I know
| SUSE/Fedora people are close
| mouse_ wrote:
| I swear, pet project operating systems like Haiku and SerenityOS
| are the only things that still fill me with hope and keep me
| going these days. I could go on for hours writing love letters to
| Haiku.
| 09284421980 wrote:
| the hacker me
| akshayjaggi wrote:
| Haiku was my first experience writing code for a real world
| software organisation, as part of Google Summer of Code.
|
| Glad to see it still thriving and making the news every once in a
| while.
___________________________________________________________________
(page generated 2023-02-21 23:01 UTC)