[HN Gopher] Systemd: The Good Parts
       ___________________________________________________________________
        
       Systemd: The Good Parts
        
       Author : pcr910303
       Score  : 242 points
       Date   : 2021-05-16 18:42 UTC (4 hours ago)
        
 (HTM) web link (christine.website)
 (TXT) w3m dump (christine.website)
        
       | theteapot wrote:
       | Cool         .---------.       .---------.         | SystemD
       | |------>| Service |         '---------'       '---------'
       | Not Cool         .---------.       .---------.         | SystemD
       | |<------| Service |         '---------'       '---------'
        
         | dale_glass wrote:
         | I don't get it?
        
       | mwvr wrote:
       | A furry named Xe promoting systemd, just another day in the Linux
       | twilight zone.
        
       | ohazi wrote:
       | I actually like using systemd. I took the time to learn how to
       | use it, and as a desktop Linux user, it's honestly quite nice.
       | 
       | But the one thing that still really pisses me off about systemd-
       | the-project was the fact that they ate udev-the-project. In my
       | view, that decision was unnecessary and was done for purely anti-
       | competitive reasons.
       | 
       | If you're not familiar, udev is basically the user-facing device
       | manager for Linux. It's what allows you to easily configure rules
       | and permissions for all the USB devices that you plug into your
       | computer. These devices still need kernel drivers, but udev is
       | how you tell your system "Please let user ohazi use this device,
       | and also give it a convenient name like /dev/someDevice0"
       | 
       | By devouring the udev project, the systemd maintainers have
       | guaranteed that dealing with USB devices on non-systemd systems
       | was going to be a giant pain.
       | 
       | Then came the forks -- Gentoo maintains eudev, which is a
       | systemd-free fork of udev. But really, this shouldn't be a fork.
       | Udev should be independent and available on all systems. If
       | systemd wants to do something special with devices, they should
       | use udev APIs like everybody else. In my view, this is what
       | finally allowed systemd to win the init war, despite all the
       | protest. Ideological arguments about how best to start daemons is
       | one thing, but you simply can't use a modern system without a
       | sane approach to USB.
       | 
       | Edit: It seems that some of my concerns were overblown, e.g. you
       | apparently can run udev without systemd as pid 1. Udev has a
       | build-time dependency on systemd, but not a run-time dependency.
       | The eudev fork removes that build-time dependency, and (maybe?)
       | also papers over some other inconsistencies. I'm pleased that
       | this is the case. I'm annoyed that the discourse around
       | systemd/udev has been muddy enough to lead me to incorrect
       | conclusions (and I'm aware that the original version of this
       | comment likely added fuel to this particular fire). Oh well...
       | live and learn.
        
         | egberts1 wrote:
         | systemd still hasn't properly replaced ISC dhclient so I dumped
         | the entire systemd and went both Gentoo and Denuvan (systemd-
         | free Debian).
        
         | edoceo wrote:
         | I love Gentoo and really like they chose to make eudev and a
         | elogind so I can continue to use OpenRC, which has been working
         | great for me since forever.
        
           | ohazi wrote:
           | Don't get me wrong -- I too am extremely grateful to the
           | Gentoo devs for maintaining eudev. It's also the obvious
           | choice for highly memory constrained embedded Linux systems
           | that can't fit systemd. I just think it's absolutely bonkers
           | that it has to be a fork... eudev should really just be udev.
        
         | unilynx wrote:
         | Wasn't it because Kay Sievers was a major developer on both
         | udev and systemd?
         | 
         | I can image someone thinking then "I'm implementing everything
         | twice, both for udev and systemd. I'll just merge them"
        
           | zxzax wrote:
           | According to Lennart that's correct, the reason it was done
           | was to reduce the amount of duplicate code:
           | https://lwn.net/Articles/490441/
        
             | ohazi wrote:
             | I realize that I don't really have any right to complain
             | about this. I don't contribute to either project, and I
             | think that maintainers should feel free to do whatever they
             | think is best for their projects, including things that
             | help reduce burdensome things that they find annoying.
             | 
             | However, as a heavy user of both projects, I see what those
             | decisions have led to in practice, and have opinions about
             | what this might mean for the community in the medium/long
             | term. The reality on the ground today is that you either
             | run systemd+udev, or you run OpenRC/something else + eudev.
             | And having observed other projects that went down the "we
             | forked, but merge regularly to try and keep the forks in
             | sync" rabbit hole (e.g. ffmpeg and libav), this almost
             | always ends badly.
             | 
             | The goal may be to keep the two separate projects
             | compatible, but inevitably environment differences, bugs,
             | refactors, etc. cause the two projects to diverge. This
             | sucks for everybody, but it sucks _a lot more_ for people
             | using the less popular fork, and this can eventually kill
             | the fork. I worry about eudev long-term.
        
               | zxzax wrote:
               | So that really seems to be the core of the problem to me:
               | nobody really wants to seriously maintain udev as an
               | individual project. It's not a fun or glamorous project
               | to work on, it's boring, nobody will notice it unless it
               | breaks, and nobody really seems to care that it depends
               | on libsystemd. For most people this is just an
               | implementation detail. For an embedded distro I can't see
               | why you would use udevd or eudev, you probably want a
               | much simpler device manager without the giant hwdb.
        
             | cyberdelica wrote:
             | That's what code libraries were intended for.
        
               | zxzax wrote:
               | Not sure why you're saying that -- moving it to a library
               | is basically what happened. The libudev code got moved to
               | libsystemd, and now libudev is a wrapper around that.
        
               | cyberdelica wrote:
               | > Not sure why you're saying that -- moving it to a
               | library is basically what happened. The libudev code got
               | moved to libsystemd, and now libudev is a wrapper around
               | that.
               | 
               | Oh, so now one has to include "libsystemd" if one wants
               | to use "udev" - is that what you're writing? Hence the
               | original comment:
               | 
               | > By devouring the udev project, the systemd maintainers
               | have guaranteed that dealing with USB devices on non-
               | systemd systems was going to be a giant pain.
        
               | zxzax wrote:
               | I don't understand, is this not what you were asking for
               | initially? libsystemd.so is just a shared library, it's
               | not any more of a giant pain than any other library.
        
               | cyberdelica wrote:
               | Why not just include libudev, in systemd instead?
        
               | zxzax wrote:
               | Because that would not reduce code duplication. I don't
               | know if you have ever worked on any of the low-level
               | Linux libraries written in C, but the amount of
               | unnecessary code duplication across them is awful. The
               | standard library features provided by glibc are extremely
               | inadequate for modern applications, every non-trivial C
               | project I've seen starts to include their own private
               | implementations of various C++ things like hashmaps,
               | binary trees, dynamically sized strings, unicode support,
               | async event loops, etc.
               | 
               | For utilities that have to interact with low level kernel
               | APIs it's even worse, every library seems to have to
               | reimplement their own parsing of various other random
               | things like netlink, or in the various pseudo filesystems
               | like sysfs, procfs, cgroupfs, etc etc, the situation is
               | really way out of hand. I don't know how to solve this in
               | a reasonable manner beyond what systemd is already doing.
               | Yes people will complain that they have a systemd
               | dependency now but what else can you do? This is the
               | exact reason the BSDs update the kernel, libc, init and
               | core utils in tandem and consider a lot of the kernel API
               | to be private, Linux was just slow to catch on in that
               | regard.
        
               | cyberdelica wrote:
               | > Because that would not reduce code duplication.
               | 
               | I don't want to call you out for making a disingenuous
               | argument, but it was either subsumed by systemd for code
               | duplication reasons, or it wasn't?!
               | 
               | Duplicate code, could be refactored out to a shared
               | library, that could then be incorporated in both udev,
               | and systemd. That would mean, anyone looking to
               | incorporate udev into a system, could do so without
               | depending on libsystemd.
               | 
               | Instead, it would seem udev code, has been subsumed by
               | libsystemd (in your own words) - which would appear to
               | the sceptical eye, as a power play on the part of red hat
               | - to force other distributions into using libsystemd,
               | which would logically end with them also using systemd
               | itself.
               | 
               | > I don't know if you have ever worked on any of the low-
               | level Linux libraries written in C [...]
               | 
               | Yes, I have.
        
               | jhasse wrote:
               | > Duplicate code, could be refactored out to a shared
               | library, that could then be incorporated in both udev,
               | and systemd.
               | 
               | That's basically what has been done. The library was
               | named libsystemd.
               | 
               | You seem to have a problem with the name "libsystemd".
        
               | cyberdelica wrote:
               | How do you compile udev, without "libsystemd", or without
               | glibc?
               | 
               | > You seem to have a problem with the name "libsystemd".
               | 
               | Ah, the ad hominem. Do you have a problem with the Unix
               | Philosophy?
        
               | cwyers wrote:
               | No, you just categorically don't understand. libsystemd
               | is not systemd the init system, it doesn't require
               | systemd to be running, it is simply the name of "the
               | shared library that everything developed under the
               | systemd project uses for common code." Now, you can argue
               | until the cows come home that you'd be happier if udev,
               | systemd, and libredhathatesyou (or some other name for
               | libsystemd that doesn't include systemd in it) were all
               | in different Git repos, but since you can run udev
               | without a dependency on systemd running on the system,
               | that seems like a pretty low-stakes dispute.
        
               | cyberdelica wrote:
               | > No, you just categorically don't understand.
               | 
               | Of course I understand, as do the rest of the people not
               | employed by red hat upvoting me.
               | 
               | Time and time again, I've seen systemd advocates making
               | slippery, disingenuous, and outright false arguments.
               | When they're called out on it, the goalposts magically
               | move, a rotation of usernames appear to downvote and
               | brigade which can be ascertained through downvote timing
               | correlation. When they can't win an argument through
               | facts, then they make bogus arguments that one doesn't
               | get it, or some such nonsense - or claim they're a red
               | hat conspiracy monger.
               | 
               | Seeing it over and and over again is lame, and played
               | out.
               | 
               | Now, why don't you answer the question - if udev was
               | subsumed by "libsystemd" as is claimed due to "code
               | duplication" - then why did they not just include
               | "libudev" as a dependency for "libsystemd"?
               | 
               | Of course, the question will never be answered, as it'll
               | reveal the truth.
        
               | zxzax wrote:
               | The code savings from including udev in systemd turned
               | out to be greater, as there was more re-usable code
               | already written in systemd. Please be the better person
               | and don't revive this flamewar, it's not helpful, let's
               | stick to the technical facts and work together to find
               | the answers we seek -- for example you can look at the
               | git logs to see all the shared functionality and the code
               | that was changed around:
               | https://github.com/systemd/systemd/tree/main/src/udev
        
               | zxzax wrote:
               | The issue is not with the actual udev device event logic
               | itself, but with all the other bits I talked about. Those
               | are the things that would need to be duplicated.
               | 
               | >to force other distributions into using libsystemd,
               | which would logically end with them also using systemd
               | itself.
               | 
               | As I said elsewhere, libsystemd is just a library with
               | some generic functions provided for convenience. This is
               | like saying that installing python libraries on your
               | system logically means that the PSF is trying to take
               | over your system and forcibly rewrite everything in
               | python, it doesn't make any sense.
        
               | mixmastamyk wrote:
               | I'd expect the opposite, if someone said they moved
               | something to a library.
        
         | TacticalCoder wrote:
         | > By devouring the udev project...
         | 
         | That's the main issue (not udev specifically but the overall
         | philosophy of trying to make systemd mandatory for as many
         | things as possible).
         | 
         | Any distro adopting systemd is basically forever giving up any
         | hope of moving away from systemd in the future.
         | 
         | I'm still on Debian (Debian user since "Buzz") but I already
         | tried Devuan (Debian fork without systemd) once and may switch
         | to it at some point.
         | 
         | I simply don't like the overall mindset of those pushing
         | systemd.
        
           | jamal-kumar wrote:
           | I had a hard time trying Devuan because all the package
           | mirrors are really far away from me unfortunately, but I
           | don't have any issues at all just removing it from whenever I
           | am working with debian servers. It all used to work through
           | some shell script called systemd-shim, I'm not entirely sure
           | now though, maybe someone else has an idea off hand
           | 
           | https://sysdfree.wordpress.com/2020/07/02/319/
        
         | nine_k wrote:
         | I cannot shake off the feeling that the core developers
         | standing behind systemd, pulseaudio, Gnome desktop spend a lot
         | of time using macOS, and like it better than Linux.
         | 
         | This would explain why they imitate or copy many of its
         | features (and misfeatures), and seemingly tend to replace the
         | Unix ways with entirely different approaches. Effectively
         | forcing things on users, as it was with pulseaudio and systemd,
         | is also a very macOS thing to do.
         | 
         | I'm not opposed to replacing Unix with different approaches, as
         | long as they keep playing on its key strengths, and do not
         | remove modularity and composability. That is, Plan 9 is fine
         | with me. Much of what Red Hat does, sadly, no.
        
           | zxzax wrote:
           | If plan 9 is what you want, I would encourage you to use
           | 9front. I can't see why you would use Linux if you didn't
           | like Red Hat, they have been a key contributor in Linux land
           | for a long time, employing many long-time kernel developers.
        
         | zxzax wrote:
         | >that decision [...] was done for purely anti-competitive
         | reasons.
         | 
         | This is not correct, I don't know where you got this idea. The
         | systemd-udevd daemon still runs without systemd. The code has
         | moved into the systemd repository, and it has a build
         | dependency on libsystemd, but otherwise it has no runtime
         | dependency; your distro should be able to package it separately
         | if it wants.
        
           | ohazi wrote:
           | Can you point to a distro that does this vs. using eudev? I
           | mean, you might be right, but I haven't found any, and I
           | think the reason for this is that there are subtler inter-
           | dependencies that require additional workarounds:
           | 
           | > > Also, AFAIR the connection between systemd and udev
           | doesn't really go deeper than the fact that they're sharing
           | the same upstream tarball. It is still possible to build and
           | use udev without systemd
           | 
           | > But that's not really the case operationally, and it's why
           | eudev was forked away from it for Gentoo.
           | 
           | https://lwn.net/Articles/778547/
        
             | zxzax wrote:
             | The udevd package in gentoo does exactly this, which is
             | supported as an alternative to eudev:
             | https://packages.gentoo.org/packages/sys-fs/udev
             | 
             | I believe the reason for eudev is to avoid some of those
             | other unwanted changes from upstream udev that broke udev
             | scripts. Which to me is a fine technical justification, but
             | not really related to another udev implementation having a
             | build dependency on libsystemd. The udevd package seems
             | maintained so I really don't understand what operational
             | issues that comment is getting at.
        
               | ohazi wrote:
               | Interesting. I didn't realize this was a supported
               | option, but the Gentoo wiki also seems to agree with you
               | [1], so I may need to walk back some of my frustration
               | around this.
               | 
               | I still have concerns that having a shared codebase makes
               | it easier for interdependencies to materialize later, but
               | if they've managed to keep it as decoupled as is claimed
               | here, then I'm pleasantly surprised, and also annoyed
               | that the communication around this issue has been
               | confusing enough to lead me to an apparently incorrect
               | conclusion.
               | 
               | [1] https://wiki.gentoo.org/wiki/Gentoo_Without_systemd#T
               | he_udev...
        
               | Conan_Kudo wrote:
               | Most of the systemd components actually _work_ without
               | systemd as the init. I 've used udev and nspawn without
               | systemd-init.
               | 
               | As long as you don't have a problem with libsystemd
               | existing, most things work fine without systemd-init.
               | Mainly systemd-journal and systemd-oomd both require
               | systemd-init.
        
           | dralley wrote:
           | >and it has a build dependency on libsystemd
           | 
           | That does seem kind of unfortunate for anyone that wants to
           | bootstrap a systemd-less distibution.
        
             | zxzax wrote:
             | Libsystemd is just a shared library with some common code
             | used by various systemd daemons (an epoll-based event loop,
             | the dbus protocol implementation, uuid generation, parsers
             | for the binary logs, etc). I can understand why people
             | don't want to run systemd as pid1, but having some other
             | external daemon depend on this library doesn't make it any
             | harder to bootstrap that type of distribution.
        
               | salawat wrote:
               | Why take the risk of a maintainer saying fuckit and
               | pulling in the rest of systemd since you're already using
               | their utils?
               | 
               | It's just one of those things where it's better to stay
               | away.
        
               | zxzax wrote:
               | That doesn't really make any sense, libsystemd is just a
               | library with some fairly generic functionality.
        
           | devenblake wrote:
           | I have a feeling if this was as simple as you say, distro
           | maintainers would be doing it.
        
             | JoshTriplett wrote:
             | They do. The few distributions that still support non-
             | systemd init systems _do_ split out udev:
             | https://packages.debian.org/sid/udev
        
           | cbmuser wrote:
           | Which is not that uncommon, FWIW.
           | 
           | binutils and gdb were also moved into a common repository and
           | gcc merged some repositories as well.
        
           | makomk wrote:
           | In practice I think systemd-udevd probably still runs without
           | systemd, but as I understand it that's not an officially
           | supported solution and the systemd developers have said they
           | reserve the right to make changes that break it at any time.
           | (The main reason why they haven't yet is most likely that
           | their attempts to integrate dbus into the kernel and systemd
           | more tightly were rejected by the kernel devs; if I remember
           | rightly that was expected to be the point at which you could
           | not use it without systemd.)
           | 
           | Using the kernel interfaces that udev relies on without using
           | udev itself is also unsupported and the developers consider
           | it within their rights to break those as well.
        
             | zxzax wrote:
             | It has been supported for at least the last 9 years now, I
             | don't see why they would have any reason to break it.
             | 
             | >The main reason why they haven't yet is most likely that
             | their attempts to integrate dbus into the kernel and
             | systemd more tightly were rejected by the kernel devs; if I
             | remember rightly that was expected to be the point at which
             | you could not use it without systemd.
             | 
             | That seems quite dubious, udev has never depended on dbus,
             | and there really would be no reason for it to ever do that.
             | 
             | >Using the kernel interfaces that udev relies on without
             | using udev itself is also unsupported and the developers
             | consider it within their rights to break those as well.
             | 
             | This I know is true, the netlink stuff varies really wildly
             | from driver to driver. The alternative would be to put
             | udevd and the other userspace bits into the kernel, which I
             | really doubt anyone wants to do.
        
         | mjw1007 wrote:
         | What particularly grated was that they managed to introduce
         | bugs into udev and then say they didn't have the "expertise or
         | understanding" to fix them:
         | https://github.com/systemd/systemd/issues/7157#issuecomment-...
         | 
         | (That bug was causing udev to give multiple disks the same name
         | in /dev/disk/by-path, which I think is pretty much core udev
         | functionality, given that udev was intended to supersede
         | devfs.)
        
           | 867-5309 wrote:
           | the systemd stars-to-issues ratio is worrying to say the
           | least
        
             | TheDong wrote:
             | What do you imagine that ratio shows? Different projects
             | use issues differently, so that number isn't super
             | comparable...
             | 
             | And I've also noticed that low-level systems projects
             | generally attract relatively few stars (i.e. a javascript
             | library or go library will have more stars than a well used
             | c library or systemd). My theory on that is that the people
             | involved in projects like systemd, gnu, etc, see github as
             | mostly just a git repository host, and don't bother with
             | stars etc.
             | 
             | I guess from my perspective, I put zero weight into the
             | stars on a project, and issue count only matters in the
             | context of how the project deals with issues. What do you
             | see as "worrying" in there? Do you see more value in those
             | numbers than me?
        
             | Iolaum wrote:
             | lolz - i 'll be keeping an eye on that from now on :) (on
             | some of my fav projects)
        
           | zxzax wrote:
           | I'm just curious, how would you suggest they debug and
           | maintain support for that hardware, without even owning that
           | particular device? It seems like anything they do would just
           | be a wild guess as to whether it actually works or not. In my
           | experience with this specialized datacenter hardware, if
           | people want any of this stuff fixed (in userspace or in the
           | kernel) it usually falls on the hardware vendor to hire some
           | developers to fix it and upstream it, or at the very least
           | donate some hardware to the upstream project so they can test
           | properly.
        
             | mjw1007 wrote:
             | If they felt they needed information from, say, the Linux
             | scsi maintainers then they could have gone out and asked
             | for it. If they thought that in order for udev to do a
             | proper job with these devices they needed the sg3-utils
             | maintainers to write something new, they could have gone
             | and asked for that. As it was they just stuck a "needs-new-
             | home" label on the bug and ignored it.
             | 
             | I think useful action would have been more likely if udev
             | had still been maintained by a team who believed they'd
             | taken on the responsibility to make /dev/disk/by-path work,
             | rather than maintained as a small part of a project whose
             | maintainers are mostly interested in other things.
             | 
             | (Also, IIRC from what I had to do to work around it, they
             | could have fixed that bug by reverting the change that
             | introduced it. It wasn't a case of existing udev code not
             | working with new hardware; it was caused by someone making
             | changes without understanding the consequences.)
        
               | zxzax wrote:
               | That's what I mean though, all of that seems to be just
               | wild guessing as to what will actually work and sending
               | random emails out to random people who may or may not be
               | able to fix it (Disclaimer: I don't know anything about
               | this specific bug or the specific hardware). Wouldn't
               | someone who actually owns the device and knows who to
               | contact on the kernel side be a better person to lead the
               | effort on that?
        
               | mjw1007 wrote:
               | This was a bug in udev.
               | 
               | The right people to lead the effort on fixing bugs in
               | udev are the udev maintainers.
               | 
               | If people did not wish to have the responsibility for
               | leading the effort to fix bugs in udev, they had the
               | option of not taking over the maintenance of udev.
               | 
               | I agree with ohazi, above, that the world would be a
               | better place if the systemd people had availed themselves
               | of that option.
               | 
               | (To be fair, it looks like this issue was fixed a few
               | months later by someone who knew what they were doing. So
               | the main lesson here may be something more like
               | "@poettering should not be triaging udev bugs".)
        
               | zxzax wrote:
               | I see people saying this kind of thing often in open
               | source and I think you are missing the point here: who is
               | going to fix this bug? You can say systemd developers
               | shouldn't triage it, and then you're left with nobody to
               | triage the bug at all. So what will you do? The point is,
               | they don't own the hardware, they can't fix the bug. You
               | have to find somebody who actually does own the hardware
               | who knows what is going on, which seems to be exactly
               | what happened. But if you get unlucky and if zero of
               | those people are contributing to udev or any of its
               | forks, then the bug probably won't ever get fixed.
        
               | 3np wrote:
               | I think their point is: with smaller project scope, this
               | bug wouldn't have existed in the first place.
        
               | zxzax wrote:
               | That is throwing the baby out with the bathwater though,
               | people who want large scale projects will continue to
               | make projects with large scopes.
        
               | unanswered wrote:
               | This wasn't a hardware bug and it didn't materialize only
               | when some particular hardware was plugged in. Were they
               | just supposed to open a phone book, pick a random hard
               | drive manufacturer, and say "hey this is your problem
               | now"?
        
               | zxzax wrote:
               | No, it seems the specific hardware causing the issue was
               | mentioned in the bug report.
        
               | mjw1007 wrote:
               | For what it's worth, I didn't mean that the systemd
               | people shouldn't triage the bug.
               | 
               | I meant that I suspect @poettering was mistaken when he
               | said the current maintainers didn't collectively have
               | "the expertise and understanding to maintain this
               | properly."
               | 
               | As far as I can make out, there was nothing particularly
               | hardware-specific about the bug. The bug was that udev
               | was assuming that there could be no more than one disk
               | per SATA host node, which turns out not to be the case.
               | 
               | `udevadm info` output would have been enough to see what
               | was going on, which I'm sure the reporter would have
               | happily supplied.
        
               | zxzax wrote:
               | From what you were saying earlier, from the perspective
               | of the bug triager, it sounds like there were multiple
               | areas that maybe this could have been fixed in, not
               | necessarily udev. So still that comes back to: the
               | correct people to triage that bug really would have been
               | the distro maintainers, who have total visibility over
               | the whole system and who are better equipped to pinpoint
               | where the bug should be fixed. And then from there they
               | can pass it off to a systemd person or an OEM person, or
               | both, or whatever (Again disclaimer, I don't know
               | anything about this particular SATA hardware or whether
               | this is normal behavior for a host node, this is just my
               | experience from trying to triage this type of bug).
        
         | Valmar wrote:
         | > But the one thing that still really pisses me off about
         | systemd-the-project was the fact that they ate udev-the-
         | project. In my view, that decision was unnecessary and was done
         | for purely anti-competitive reasons.
         | 
         | The developers of udev at the time agreed to merge to the
         | project into systemd, as it made a lot of sense to them.
         | 
         | There was no "devouring", no "anti-competitive" bullshit.
        
       | dijit wrote:
       | Copying my reply from Lobste.rs because there's more people here
       | who can pick me apart:
       | 
       | _FD: I'm not a large supporter of SystemD after using it when it
       | was first introduced in Fedora and watching it take over the
       | ecosystem, I do use it and as a sysadmin/SRE type I'm heavily
       | versed in it and how it functions, so, everything with a pinch of
       | salt._
       | 
       | I gave the video a good shot, half the time is spent going over
       | some systemd-ecosystem basics;
       | 
       | The author then makes the case that Alpine is a less attractive
       | development/production-deployment platform due to lack of this
       | ecosystem (though they make effort to avoid directly saying that
       | systemd would be a good solution itself.)
       | 
       | I couldn't disagree more honestly. I love the concept of Alpine:
       | as small as possible and no smaller.
       | 
       | The attention to minimise dependencies makes it the distro of
       | choice in containers; I've even considered (strongly) running it
       | on some machines where systemd would have been a lot of needless
       | complexity.
       | 
       | I think deviating from this path for alpine is a grave error,
       | there will always be a place for a niche system that is small;
       | especially when it's commonly paired with some other log
       | managment/service orchestrator like Alpine is.
       | 
       | Additionally, it gives people like me an option to run without
       | init systems that could be considered heavy and intrusive with
       | heavy dependency chains.
       | 
       | So, kindly, Alpine: please don't.
        
       | AshamedCaptain wrote:
       | I have a system where systemd decides, about half the times I
       | boot it, to immediately unmount all of the filesystems listed in
       | fstab literally right after having finished mounted them as part
       | of the regular boot process. All of this with other daemons
       | starting in parallel.
       | 
       | It will fail to unmount most of the filesystems, since they are
       | of course busy, but often it will succeed in unmounting /var,
       | /tmp, /home and others. Then it will continue starting on further
       | services as if nothing happened and even proceed to gdm. But of
       | course without /home I can't even login.
       | 
       | No idea how to even start debugging it.
        
         | sigg3 wrote:
         | I'd start by looking at the systemd unit files in
         | /etc/systemd/system. Beware of service wants.
         | 
         | I had straight up errors in a network unit file that caused
         | weird behavior (IP address disappearing). This was a long time
         | ago though. Sometimes moving a unit from one target (graphical)
         | to an earlier (multiuser) solves the issue.
         | 
         | You might also have "defaults" in /usr/lib/systemd/ to compare
         | with.
         | 
         | systemctl and friends are really good tools for troubleshooting
         | but takes a little while to get used to. Duck duck go is your
         | friend.
         | 
         | Finally, some volume technologies like VDO requires a
         | x-systemd.requires statement in fstab.
        
         | jraph wrote:
         | I would run systemd-analyze critical-chain and then systemd-
         | analyze blame to see if there is a service that could do this.
         | And then I would look at the detailed boot logs. If nothing can
         | be found, I'd look for some way to increase the log level.
         | 
         | If you are not curious, reinstalling everything could be quite
         | fast too, depending on what you have installed.
        
           | AshamedCaptain wrote:
           | The logs are mostly useless and random. There is nothing that
           | consistently appears between the point where it finishes
           | mounting (lots of "Mouting...", then "Reached target Local
           | File Systems") and where it starts unmounting ("Stopped
           | target Local File Systems", then lots of "Unmouting ...").
           | Sometimes these two lines are less than one second apart. But
           | I haven't found any message which is consistently between
           | these two points that would point to a suspect.
           | 
           | And this is not a problem that happens on every boot, so it
           | doesn't show up on analysis... not to mention that most of
           | the analysis is designed on trying to reduce boottime, which
           | obviously is no help (thought it is rather fast already).
           | 
           | It is rather easy to guess why a service is starting, but not
           | why it decides to stop 'orderly', much less why a .target
           | decides to stop...
        
             | jraph wrote:
             | Most certainly you can only analyze things on boots where
             | the problem appears, unless logs are persisted across
             | boots.
             | 
             | Indeed systemd-analyze focuses on boot times, though I find
             | it is useful to get a list of services that are run.
             | 
             | If you are adventurous you could probably replace the
             | umount (resp. the systemd-umount) binary by a script that
             | runs umount (resp. systemd-umount), and also prints the
             | process tree (and its PID) in some file so you can get
             | insight on what ran it.
        
         | teddyfrozevelt wrote:
         | Systemd generates mount units for the filesystems listed in
         | fstab. Maybe check those with systemctl status? (Like
         | `systemctl status -- -.mount` for my root partition)
        
           | AshamedCaptain wrote:
           | Sadly no. The problem is that the entire local-fs.target is
           | stopped 'cleanly' but with no clear reason. i.e.
           | $ systemctl status local-fs.target         Active: inactive
           | (dead) since [...]         21:12:56 ... systemd[1]: Reached
           | target Local File Systems.         21:12:57 ... systemd[1]:
           | Stopped target Local File Systems.
           | 
           | Yes, the log just shows it stopping one second after being
           | started, nothing happening inbetween, and no reason given for
           | it being stopped.
           | 
           | Incredibly enough, local-fs.target is a dependency for
           | graphical.target, so the system should not have continued
           | booting. But not only it has continued booting, systemd even
           | thinks that it finished booting all-OK. State is "running"
           | (not "degraded" as it would be if any service/mount failed),
           | with 0 failed services. Even though both graphical.target and
           | local-fs.target are 'dead'.
        
           | dijit wrote:
           | Candidly: that command looks like the least discoverable
           | command I've seen in the last year.
        
             | heavyset_go wrote:
             | The command                   systemctl status *.mount
             | 
             | Just shows the status of all mount units.
             | 
             | It's the same command you'd use to check any other systemd
             | unit, for example, if you want to check Docker's status:
             | systemctl status docker.service
             | 
             | Seems consistent and discoverable as long as you have a
             | surface understanding of systemd units.
        
             | teddyfrozevelt wrote:
             | Fair. I think it's because systemd lets you use slashes in
             | unit names, but you can't represent those as files so you
             | have to use a dash instead. This is the worst for the root
             | filesystem mounted at / since that's the entire name. In
             | systemd's defense, it warns you when you try systemctl
             | status the normal way.                 Hint: to specify
             | units starting with a dash, use "--":             systemctl
             | [OPTIONS...] COMMAND -- -.mount ...
             | 
             | I found my mount names by running systemctl list-units, so
             | it's not something I had to look up.
        
             | azornathogron wrote:
             | It is at least built in a logical way and once you
             | understand that logic then _related_ commands will be
             | discoverable.
             | 
             | `systemctl` is the systemd command line tool for inspecting
             | and manipulating services/units. `systemctl status` is the
             | command for showing the status of a service or unit.
             | 
             | The `--` part is not specific to systemd at all, it's used
             | in _many_ commands to separate flags from positional
             | arguments. This lets you use positional arguments that
             | might happen to start with a dash without them being
             | interpreted as flags. [1].
             | 
             | systemd manages mounts, they're named according to the
             | mount point, with a transformation to turn it into a
             | filename (mostly converting slashes to dashes, plus a
             | .mount suffix). Hence, the root mount ('/') is named
             | '-.mount'.
             | 
             | [1] See, e.g.,
             | https://unix.stackexchange.com/questions/11376/what-does-
             | dou...
        
               | unilynx wrote:
               | The annoying part is that now that '-' stands for '/',
               | you need to use '\x2d' for '-'. Which is pretty annoying
               | when manipulating mount names during the shell.
               | 
               | I wish they had picked ':' for '/' instead of '-'. That
               | one would be pretty uncommon to find in a mount path, and
               | has a bit of precedent on OSX.
               | 
               | (Who would ever mount anything on a path with a colon
               | anyway? That would be as silly as using slashes instead
               | of dashes for options)
        
               | salawat wrote:
               | C:\ much?
        
           | heavyset_go wrote:
           | If you use a wildcard operator, you can list all the mount
           | statuses: systemctl status *.mount
        
         | viraptor wrote:
         | You can check in journal which service startup does the
         | unmounting align with. Maybe generate a bootchart
         | http://manpages.ubuntu.com/manpages/bionic/man1/systemd-boot...
         | 
         | You could also replace umount with a wrapper that logs the
         | parent process tree on execution.
         | 
         | And of course you can open an issue on GitHub.
        
           | pengaru wrote:
           | > And of course you can open an issue on GitHub.
           | 
           | Only do this if you've reproduced what seems to be a bug in
           | one of the latest two releases, otherwise contact your
           | distro.
           | 
           | The systemd github issue tracker is _not_ a support forum,
           | you 'll find this stated in the issue submission form.
        
             | kristjansson wrote:
             | Fair advice, and they're probably right to redirect most
             | users that would file a bug with them.
             | 
             | But on a normative level, perhaps a project that doesn't
             | want to support users shouldn't have subsumed a huge swath
             | of user land things?
        
         | stevenhuang wrote:
         | If that happens again I'd try opening a virtual terminal, login
         | as root and check logs.
        
           | Twirrim wrote:
           | > If that happens again I'd try opening a virtual terminal,
           | login as root and check logs.
           | 
           | If the system has unmounted all the drives, there won't _be_
           | any logs. This happens often enough with systemd  / journald
           | to be a big source of annoyance.
           | 
           | The only option you get is to tell systemd to boot in debug
           | mode _but_ that fundamentally alters the timing and behaviour
           | of so much that it often stops the weird behaviour from ever
           | happening (similar fun can be had with `dracut` where
           | enabling debug tends to stop any race condition from ever
           | happening)
        
             | AshamedCaptain wrote:
             | Note systemd does dump its own log to the kernel ring
             | buffer until journald starts (the last message I see on the
             | kernel ring buffer is Started Journal Service). It's no
             | help though when (as in my case) journald starts but then
             | has /var unmounted below its feet. I can write a script
             | that keeps /var busy though, preventing it from being
             | unmounted.
        
         | isodude wrote:
         | We had that kind of problem because we relied on /dev/disks/by-
         | label
        
           | heavyset_go wrote:
           | Can you tell us more about this? How did you determine that
           | that was the problem? What was your solution?
           | 
           | I ask because I use /dev/disks/by-label and have never had
           | this problem, and if I ever had to experience it, it would
           | drive me insane.
        
             | sigg3 wrote:
             | I've solved a similarly sounding issue by using the UUID.
             | Get it with blkid, then use UUID="<uuid goes here>" in
             | fstab instead of LABEL="<label>".
             | 
             | But this was a long time ago, on a much older systemd (like
             | Fedora 17 or 18).
        
             | dharmab wrote:
             | This is a common error even back in the Sysvinit days.
             | /dev/disks/by-label is not guaranteed to map the name
             | labels to disks on every boot, and it can change with boot
             | timing based on the storage controller behavior. Use
             | /dev/disks/by-uuid instead.
             | 
             | /dev/disks/by-label is a legacy from the old days of
             | systems with static disks wired in a specific, unchanging
             | configuration.
        
               | heftig wrote:
               | You've confused by-label with the kernel's device nodes
               | (e.g. /dev/sda1). Kernel names are assigned in order.
               | Labels are just like UUIDs and part of the filesystem.
               | 
               | However, unlike UUIDs they don't necessarily exist, are
               | not as likely to be unique (you should only get duplicate
               | UUIDs by cloning filesystems) and are easier to change.
               | This makes them less suitable for mount configuration.
        
         | corty wrote:
         | Deactivate graphical boot screens (often just by pressing Esc),
         | increase the log level[0] and try to read the logs somehow.
         | Maybe you can still log in as root. If you cannot log in as
         | root even, but networking works, configure rsyslog to send your
         | logs to another host and read them there.
         | 
         | [0] https://freedesktop.org/wiki/Software/systemd/Debugging/
        
           | jraph wrote:
           | if you can't login as root, you can try to put bash as init
           | (with init=/bin/bash in the kernel boot command line) to
           | inspect the system. Might be difficult on some systems (with
           | SELinux?)
        
             | megous wrote:
             | if you have access to kernel command line to add init= you
             | can also disable selinux there.
        
         | pengaru wrote:
         | That's super annoying. I've experienced similarly frustrating
         | boot problems when I had some FS corruption causing /var fsck
         | to fail @ boot.
         | 
         | Despite dropping me to a rescue shell, systemd entered a
         | continuous loop of retrying fscking /var while I was attempting
         | to identify and fsck the problematic filesystem.
         | 
         | Few things are more infuriating than having your system drop
         | you to a rescue shell, while it continues to endlessly change
         | the state of the system - and not just any state, but the very
         | area you're in the rescue shell to investigate and resolve. It
         | just kept making /var busy while I kept attempting to manually
         | fsck it...
        
       | [deleted]
        
       | bfrog wrote:
       | Yeah it's all pretty darn good. People forget the cluster of bash
       | scripts, various cronz, inetd, log, and other supervisor type
       | daemons all to do what systemd does better with a simpler config.
       | 
       | Now ask me about pulseaudio and there's a project with nothing
       | but issues, looking forward to the new pipewire overlords.
        
         | tediousdemise wrote:
         | > Now ask me about pulseaudio and there's a project with
         | nothing but issues
         | 
         | Don't forget Bluetooth, WiFi, and graphics ( _cough_ Nvidia).
         | Or any peripheral, for that matter.
         | 
         | Until vendors stop forcing us to play games with their broken
         | blobs, Linux will forever be just a grand experiment for
         | frustrated hobbyists and unfortunate professionals around the
         | globe. A digital tragedy of the commons.
        
           | techrat wrote:
           | >Linux will forever be just a grand experiment for frustrated
           | hobbyists and unfortunate professionals around the globe.
           | 
           | Linux is the most widely used kernel in the entire world. If
           | you consider 'Linux' to be the OS, it is #1. If you consider
           | all distributions that use the Linux kernel, it is also #1,
           | collectively and individually (Android.)
           | 
           | If you look at the ranked list of Supercomputers, Linux has
           | the entire top 500.
           | 
           | What the hell are you talking about?
        
             | OhSoHumble wrote:
             | Probably just referring to desktop usage.
        
             | kllrnohj wrote:
             | > and individually (Android.)
             | 
             | Which, worth noting, does its own thing for init & services
             | ( https://cs.android.com/android/platform/superproject/+/ma
             | ste... ).
             | 
             | I think it's fair to assume they were talking about Linux
             | as a desktop OS, which for some reason absolutely refuses
             | to learn lessons from the other 99.9% of the market. And is
             | also infatuated with internal fragmentation despite its
             | tiny usage. That makes it really fun as a hobby for sure,
             | and powerfully flexible for datacenter usages (from the
             | mundane to the supercomputers). But also pretty terrible as
             | a "I just want my laptop to _work reliably & well_" OS.
        
       | GekkePrutser wrote:
       | I don't agree with her sentiment that alpine should adopt
       | something similar. The whole reason I use alpine is because it
       | doesn't use systemd.
       | 
       | I like alpine because it uses a simpler, lighter approach. It's
       | not for everyone, and I'm pretty sure alpine maintainers don't
       | intend it to be for everyone. It serves its niche really well as
       | it is. For those who like systemd there's many distro choices
       | already.
       | 
       | Edit: As detaro mentioned in another comment (
       | https://news.ycombinator.com/item?id=27176391 ) they are actually
       | working on something, but they aim to solve the various issues
       | around alpine that don't fit well with alpine's philosophy. It
       | sounds like a good story and I'm open to this idea. The main
       | reasons I don't like systemd are its complexity and its dbus
       | reliance, basically bringing too many desktop paradigms into all
       | Linux systems, even servers and containers.
       | 
       | So about my above point: I'm open to it adopting a service
       | manager but not too similar to systemd :)
        
         | [deleted]
        
       | yuuta wrote:
       | I'm using split horizon DNS with WireGuard on my systems (I mean
       | using special DNS servers for some domain names and the public
       | ones for the rest. Correct me if I'm wrong). I used to try
       | systemd-networkd and systemd-resolved because they seem to fit
       | together. I found systemd-resolved is extremely buggy with
       | systemd-networkd managed WireGuard. My WireGuard endpoints use
       | domain names and systemd-networkd fails to resolve them after a
       | manual restart. I couldn't figure out why.
       | 
       | I also could not find a way to NOT combine split horizon DNS with
       | network interfaces (like most documents online only include
       | scenarios like resolving corporate domain names using a VPN
       | interface but that wasn't my case). I have multiple VPN
       | connections to the same network (runs BGP basically) and I did
       | not want the split DNS to bind to some specified interfaces.
       | 
       | Moreover, systemd-resolved seems not able to do split DNS without
       | systemd-networkd. It needs systemd-networkd managed interfaces to
       | do that. What? I just want separate DNS servers for separate
       | domains. My WireGuard interfaces are managed by wg-quick. Correct
       | me if it is possible to use systemd-resolved standalone in this
       | case.
       | 
       | The hard systemd-resolved dependency of systemd-networkd also had
       | issues. Sometimes, I just want systemd-networkd to configure
       | resolv.conf using DHCP DNS but it looks like I HAD TO START
       | systemd-resolved AND LINK /run/systemd/stub-resolv.conf to
       | /etc/resolv.conf in order to achieve that (correct me if there's
       | a way to not use resolved completely). Excuse me?
       | 
       | systemd-networkd itself was buggy, too. I once encountered
       | https://github.com/systemd/systemd/issues/18108 and my friends
       | reported they faced this issue as well. I don't mean that
       | software should not have bugs but encounter them is really
       | inconvenient.
       | 
       | In conclusion, I mean, systemd components seem to fit together
       | and help you do a lot of things, but they did't do the things
       | well, compared to separate programs for their specified tasks.
       | Moreover, you have to deal with various components of systemd and
       | figure out their relationships in order to achieve some specified
       | tasks. For split DNS, I'm using dnsmasq now and it's working
       | pretty well.
        
       | jph wrote:
       | > By not having something competitive Alpine is less and less
       | attractive for newer production deployments.
       | 
       | Alpine uses OpenRC (https://wiki.gentoo.org/wiki/OpenRC). IMHO
       | Alpine and OpenRC tend to make production deployments more
       | predictable and reliable, because of deliberately simple init
       | ordering, as well as a smaller surface area as compared to
       | systemd.
        
         | coolreader18 wrote:
         | I think given these lines in the article (and the fact that she
         | mentions OpenRC):
         | 
         | > I have been an Alpine user for almost a decade and it's one
         | of my favorite linux distributions.
         | 
         | > This talk is intended to show how green the grass is on the
         | other side and how Alpine can benefit from these basic ideas.
         | 
         | She's probably familiar with it.
        
         | detaro wrote:
         | There is some work on a replacement that does more while
         | fitting the requirements and philosophy of alpine:
         | https://skarnet.com/projects/service-manager.html
        
           | GekkePrutser wrote:
           | Interesting, I have a lot more faith in the alpine team
           | implementing something like this than in RedHat. Even when
           | RedHat made consumer OSes (before spinning them off to
           | Fedora) I didn't like them.
           | 
           | What they're doing looks like a great alternative. I also
           | hope they will not have any reliance on dbus as systemd does.
           | Reliance on dbus is one of the main issues I have with
           | systemd (besides its size and complexity).
        
           | mendelmaleh wrote:
           | Good read, thank you!
        
       | xena wrote:
       | Author of the talk here. If you have questions for me please feel
       | free to reply to this comment with them.
        
         | Hackbraten wrote:
         | No question, just thank you for an excellent article. I've
         | found it difficult to come across articles that focus on the
         | good parts of systemd.
        
         | Traster wrote:
         | Hey, thanks for the great talk, I often type something like
         | "journalctl -xfe" and just get completely lost, you mentioned
         | that's out of scope for this talk but do you have a
         | recommendation for some hints about how to use journalctl
         | effectively?
        
           | dale_glass wrote:
           | A quick guide:
           | 
           | * Want logs since boot? journalctl -b. The first boot is -b
           | 0, the previous boot is -b -1.
           | 
           | * Want logs for smartd? journalctl -u smartd
           | 
           | * Want warnings and above? journalctl -p warning
           | 
           | * Want logs since yesterday? journalctl -S yesterday
           | 
           | * Want logs on April 22 between 10 and 11? journalctl -S
           | "2021-04-22 10:00" -U "2021-04-22 11:00"
           | 
           | * Want logs with microsecond timestamps? journalctl -o short-
           | precise
           | 
           | * Or in JSON for parsing? journalctl -o json
           | 
           | * Want tail -f? journalctl -f
        
             | themulticaster wrote:
             | To add some of my own favorite commands:
             | 
             | * Want only kernel logs? journalctl -k or journalctl
             | --dmesg
             | 
             | * Want to filter the output right away? journalctl
             | --grep=some_regex
             | 
             | * You booted a live environment, but want to look at the
             | installed OS's journal: journalctl
             | --root=/path/to/mounted_root or journalctl
             | --directory=/path/to/journal
             | 
             | * Want to look at the logs from a container you booted
             | using systemd-nspawn (and friends)? Just use journalctl
             | --machine=container_id
             | 
             | * What time intervals do the boot indices (for -b)
             | correspond to? Look no further than journalctl --list-boots
             | 
             | I'd say journalctl really has a lot of generally useful
             | flags.
        
         | jen20 wrote:
         | I did a talk at HashiConf EU a few years ago with the same
         | title - interesting to see where our ideas of "the good parts"
         | overlap!
        
           | xena wrote:
           | Wow, I think that was completely by coincidence. I've never
           | seen your talk before. Where did we overlap?
        
       | fghorow wrote:
       | "This page intentionally left blank."
       | 
       | (Yeah, I'm a dinosaur who remembers dead tree IBM manuals. Sue
       | me.)
        
       | ape4 wrote:
       | 1. You don't have to clone then modify a 100 line bash script for
       | a normal daemon's /etc/rc.d/init.d script.
        
         | jude- wrote:
         | The OpenBSD init system and OpenRC init system haven't had this
         | problem for years -- even before systemd became a thing.
        
           | CameronNemo wrote:
           | Similarly runit, s6, and Upstart have never had that issue.
           | It is the oddest complaint to come up with when considering
           | life without systemd.
        
             | unanswered wrote:
             | I for one have great sympathy for people who are placed in
             | the position of having to give good reasons for using
             | systemd. "All your distro are belong to us" was the
             | original plan but it didn't quite work out. The shills had
             | to scramble but they managed to make the best of a bad
             | situation.
        
             | jraph wrote:
             | Upstart was good and probably paved the way to systemd.
        
               | cbmuser wrote:
               | No, Upstart's design was so bad that even Scott James
               | Remnant (the creator of Upstart) admitted it's design was
               | inherently flawed.
               | 
               | The biggest problem with Upstart was that it put the
               | dependency chain upside down and just started a service
               | whenever it's dependencies were fulfilled, no matter
               | whether it actually made sense to start the service.
        
               | ahartmetz wrote:
               | I remember reading about it and thought it seemed wrong,
               | but presumably the author did his research and knew what
               | he was doing. Oh well. His attitude and the readability
               | of Upstart code were very good, though.
               | 
               | The Upstart design was probably easier to implement. With
               | the systemd design, the dependency tree needs to be
               | traversed bottom up (figure out what needs to be started
               | to start the targeted set of services), then top down
               | (start dependees when dependencies are ready). With
               | Upstart, top down was enough. Or so it seems.
        
               | agumonkey wrote:
               | I wonder if systemd is influencing a new init right now
        
           | moshmosh wrote:
           | OpenRC is the only init system I've ever used on Linux that I
           | actually _like_ , rather than just tolerating.
        
             | dbcurtis wrote:
             | This
        
         | [deleted]
        
         | fiddlerwoaroof wrote:
         | Instead you have to write a couple-hundred-line bash script and
         | then modify an under-documented ini-file to reference it
        
           | merb wrote:
           | you never used systemd?
        
             | tgv wrote:
             | There's a bit of truth in that. The file format is
             | underdocumented. You've got to look everywhere to find
             | snippets of information, and then there are these weird
             | incompatibilities between versions (Service.Type bit me
             | once, and either After or Wants simply didn't do what the
             | scant information suggested). And it often starts a script
             | or a dedicated tool, e.g. for backing up, or watching
             | directories.
             | 
             | However, it's easier to deploy and keep track of your own
             | services, sudo and chdir are built in, there's a watch dog,
             | easy journaling. That's nothing to sneeze at.
        
               | fiddlerwoaroof wrote:
               | I've written a handful of systemd units now and, while
               | the simple cases are nice, anything more complicated is
               | really hard to figure out by reading manpages or other
               | official sources of documentation. And, additionally,
               | I've found it's often easier to write a shell script to
               | be invoked by systemd than to handle certain issues in
               | the unit file itself.
               | 
               | As far as the watchdog but things go there's plenty of
               | good enough solutions that don't really on systemd:
               | daemonize, DJB's daemontools, etc.
        
         | iso1210 wrote:
         | I have 90 scripts in my /etc/init.d directory. 51 are less than
         | 100 lines, the most recent one I created is actually a perl
         | script which is 29 lines long.
        
           | fiddlerwoaroof wrote:
           | Yeah, with the #!-lines, sysv init scripts can be written in
           | just about anything and, as long as you implement the
           | expected command line interface, the init system will handle
           | it correctly.
        
         | Arch-TK wrote:
         | I've not seen even sysvinit based systems with this problem.
         | Every service manager I know of solves this problem. This is as
         | much an argument in favour of systemd as "don't have to clean
         | up horse excrement" is an argument in favour of owning a fiat
         | 500.
        
           | viraptor wrote:
           | Most init scripts are in that ballpark. See openssh for
           | example (88 lines) https://github.com/openssh/openssh-
           | portable/blob/master/open...
           | 
           | And yet, it still tries to use pidfiles (which systemd has a
           | better alternative for), and notes "We really should check if
           | the service is already going" which they don't do because
           | it's an actual non-trivial issue if you're writing a
           | standalone init script.
        
             | Arch-TK wrote:
             | Does anyone use these init scripts? Every popular sysvinit
             | based init system uses short declarative service scripts
             | for most services. Every non-sysvinit init system seems to
             | mostly manage with 2 line scripts.
             | 
             | And yes pidfiles are crap and I don't like them either (a
             | genuine criticism of sysvinit and advantage of systemd) but
             | systemd isn't the first init system to do away with
             | pidfiles so it's hardly a unique strength.
        
               | viraptor wrote:
               | Could you show an example of a declarative sysv-init
               | script? To be honest, I'm not aware of it being possible
               | and ddg doesn't bring up anything.
               | 
               | > Does anyone use these init scripts?
               | 
               | By default every popular distro before systemd/upstart
               | did. Many services still use them anyways (systemd just
               | generates a shim to run the start/stop/status operations
               | on them)
        
       | ChuckMcM wrote:
       | Okay, I'll admit I was hoping for a blank web page :-). Not a
       | fan.
       | 
       | I always ask ""What problem is this trying to solve?" and then
       | "Is this a good solution for that?"
       | 
       | I appreciate the "startup dependency problem" and when I was in
       | the Systems Group at Sun we worked with AT&T on their solution
       | which was the whole "run level" thing.
       | 
       | And I appreciate the whole "shell scripts can do _anything_ "
       | problem that just using a naming scheme on shell scripts to run
       | can lead to unexpected behaviors.
       | 
       | From an architectural taste perspective, I prefer systems which
       | are more comprehensible in parts and thus don't try to abstract
       | out semantics to the point of incomprehension.
       | 
       | So at the end of the day, something with a better model, better
       | visibility into its operation and configuration, with a way to
       | validate and debug dependencies would be good. But I haven't been
       | impressed with systemd.
        
         | Arch-TK wrote:
         | >I appreciate the "startup dependency problem" - except systemd
         | doesn't actually solve this problem.
         | 
         | Good to see people down-voting a completely factually correct
         | statement that they don't believe and don't want to personally
         | investigate. Go read the systemd sources or read any article
         | analyzing how systemd does dependency based startup and see for
         | yourself.
        
         | addicted wrote:
         | SystemD solved enough problems that such a large number of
         | distro owners switched to it that one of the biggest complains
         | against it is how prevalent it is.
        
         | dale_glass wrote:
         | It solves a whole lot of problems to me.
         | 
         | * It gets rid of a system cobbled together from init, inittab
         | (which people rarely remember exists), monit, cron, and
         | inetd/xinetd.
         | 
         | * It removes a bunch of horrible hacks in init scripts such as
         | calls to sleep.
         | 
         | * It removes the need for pid files.
         | 
         | * It removes the need for a lot of stupid boilerplate, like the
         | whole start-stop-daemon stuff in scripts.
         | 
         | * It allows precisely tracking what belongs to each service. I
         | can easily find out what a random process belongs to.
         | 
         | * It captures all the log output of each process
         | 
         | * It drastically improves the upgrade process -- I never need
         | to look at a 3-way diff of /etc/init.d/apache2 again
         | 
         | * As a developer it means I don't need to write a slightly
         | different script for every distro
         | 
         | * It makes logging a whole lot pleasant
         | 
         | * It allows having user services without having to hack around
         | it with cron
         | 
         | * It allows a whole bunch of settings to be applied to any
         | random service, without having to edit a shell script and
         | figure out how to stick it in there.
         | 
         | * It means services run identically when started by hand,
         | without the confusion of a service inheriting my local
         | environment.
         | 
         | * It makes my VM servers boot faster. And if you're going to
         | pipe up with "I reboot my servers once a year", that doesn't
         | cut it for me. I boot my servers on demand, when they're
         | needed. They adapt to the load.
        
       | A4ET8a8uTh0 wrote:
       | "I hope Alpine ends up with something similar!"
       | 
       | Direct quote from the website.
       | 
       | Eh, no. There is already a short list of viable non-systemd
       | candidates. We.. well.. I don't need less. It already feels like
       | I am transgressing some law by typing this on PopOS.
        
         | rzzzt wrote:
         | There is some work going on around replacing OpenRC in Alpine,
         | this might be the thing the author is alluding to:
         | https://skarnet.com/projects/service-manager.html
        
           | gforce_de wrote:
           | whoever maintains the comparison of init-systems at
           | https://skarnet.com/projects/service-manager.html should
           | include OpenWRT's 'procd'
        
           | A4ET8a8uTh0 wrote:
           | Thank you for sharing this. I was not aware of it.
        
         | darthrupert wrote:
         | To each their own, but I won't touch Alpine (outside of docker
         | use) until they get systemd or something fairly similar.
        
           | efiecho wrote:
           | > To each their own, but I won't touch Alpine (outside of
           | docker use) until they get systemd....
           | 
           | Luckily that won't happen as systemd doesn't support musl
           | which Alpine uses for libc.
        
             | 2pEXgD0fZ5cF wrote:
             | I'm very glad for that too.
        
           | lixtra wrote:
           | The same design goals that make them a good docker
           | distribution may have made them to stay away from systemd.
           | 
           | Meanwhile others claim that systemd makes docker redundant
           | anyway[1].
           | 
           | [1] https://news.ycombinator.com/item?id=26473581
        
       | cerved wrote:
       | I can't weigh in on the pros/cons of systemd because that's out
       | of my depth. But I am somewhat fascinated by the hate it seems to
       | endure. Like this
       | 
       | https://suckless.org/sucks/systemd/
       | 
       | I guess it's a bit of a watershed
        
         | Arch-TK wrote:
         | For something which the authors seem intent on forcing down
         | everyone's throat whether they want it or not (genuinely you
         | can find quotes of Lennart claiming he wants to make it more
         | difficult to not use systemd or that he wants to "gently push"
         | people towards it) it's not really surprising. If their only
         | goal was to gain adoption based on the merits of their software
         | while not intentionally making themselves intrusive nobody
         | would really care.
        
       | exabrial wrote:
       | Things that are terrible: logging. Why did we have to screw this
       | up? Just writing stuff to a file was fine.
       | 
       | Things that are good: simple syntax, easy to use utils
       | 
       | Hidden gems: cgroups, file systems containment, etc. Systemd
       | could completely eliminate the need for stuff like docker by
       | totally isolating a process.
        
         | moshmosh wrote:
         | The nice thing about docker is that you can use the same
         | tooling on lots of operating systems, not just that it wraps
         | some Linux functionality. Sure, it may involve using a VM,
         | depending on the system, but I still get to just run "docker"
         | and "docker-compose" and such, as if it were native. A systemd
         | solution would need a VM running Linux w/ systemd _even on
         | Linux_ , in some cases.
         | 
         | Really, though, if I care about isolation and don't want to use
         | docker, I'm gonna reach for FreeBSD + jails anyway, and I don't
         | think systemd's gonna change that for me any time soon. If I
         | want containers with better tooling outside the target OS, I'll
         | use docker. Systemd falls in an awkward middle that I don't
         | have much interest in.
        
           | vially wrote:
           | > A systemd solution would need a VM running Linux w/ systemd
           | even on Linux, in some cases.
           | 
           | But that's exactly what happens with docker anyway (e.g.:
           | Docker is still running in the Linux VM when running it from
           | a different operating system). And considering the widespread
           | systemd adoption throughout the Linux ecosystem, it's far
           | more likely for systemd to be already installed in the VM
           | than docker anyway.
        
         | comboy wrote:
         | It's just status quo. How would you feel about introducing
         | having to log things into file after all you needed was stdout?
         | 
         | Btw, you still can do that nothing is stopping you, but I don't
         | think each and every service needs to decide (and let user
         | configure) where to put its logs, how to separate them (maybe
         | by errors, maybe by users/domains) and how to rotate them.
         | 
         | I'm still not a big fan of systemd but it does make some things
         | easier (but why wouldn't they think for a moment how easy
         | "systemctl restoptart foo" will be to type, tab complete and
         | issue different commands one after another)
        
         | dale_glass wrote:
         | The logging is amazing. Transparent compression, easy search,
         | output in JSON if you want to parse logs, timestamps with
         | microseconds if you need them.
         | 
         | No, writing stuff to a file was not fine. It's only fine if all
         | you do is to 'tail' and 'grep' it, but any time you want to do
         | something more advanced, it's a bloody mess.
         | 
         | Why parse a text date, when it started its life as an unix
         | timestamp? Why write a bunch of regular expressions trying to
         | get basic data such as an HTTP response code, when it could be
         | a field in the log that you could just search for? Why deal
         | with parsing while the log may rotate or truncate under you,
         | when the log system could be making your life easier?
         | 
         | I wrote a whole bunch of log parsing stuff in my day, and so I
         | have no nostalgia whatsoever regarding this "just write stuff
         | to a file" approach.
        
           | Arch-TK wrote:
           | > The logging is amazing.
           | 
           | Amazing for you maybe. If you have a slower spinning HDD
           | journalctl -eu <service> takes 10-30 seconds (not an
           | exaggeration). Meanwhile cat or tail -f of a log file is
           | almost instant somehow.
           | 
           | > Why write a bunch of regular expressions trying to get
           | basic data such as an HTTP response code, when it could be a
           | field in the log
           | 
           | I mean, this is a problem which doesn't require systemd to
           | solve, you just need a better log format. Someone could have
           | standardised a line based logging format with fields and then
           | someone else could have written libraries to write it and
           | libraries to parse it.
           | 
           | > Why deal with parsing while the log may rotate or truncate
           | under you
           | 
           | That's a log daemon problem that I haven't found to exist
           | with any log daemon I have used.
        
             | dale_glass wrote:
             | > If you have a slower spinning HDD journalctl -eu
             | <service> takes 10-30 seconds (not an exaggeration).
             | 
             | That's because you're asking for a log of a service since
             | logging began, which may well have been months worth of
             | logging. You're saying "give me everything that happened
             | ever to this service in a pager, then scroll down to the
             | bottom".
             | 
             | If you want tail -f, use a -f: 'journalctl -f -u
             | <service>'. That's going to be a lot faster. You can also
             | use '-b' to make it return a log of what happened since
             | boot (worth looking into the manual for the details of that
             | one), or specify a start date with -S, like '-S yesterday'.
             | 
             | > Someone could have standardised a line based logging
             | format
             | 
             | And what were they waiting for? In the end, systemd did it,
             | but it's a pretty recent invention. There was plenty time
             | to get there first.
             | 
             | > That's a log daemon problem that I haven't found to exist
             | with any log daemon I have used.
             | 
             | Because log daemons classically don't care about this at
             | all -- it's a problem for whatever reads the files that the
             | log daemon generates afterwards.
        
               | Arch-TK wrote:
               | > That's because you're asking for a log of a service
               | since logging began, which may well have been months
               | worth of logging. You're saying "give me everything that
               | happened ever to this service in a pager, then scroll
               | down to the bottom".
               | 
               | journalctl -fu is just as slow. I don't keep months of
               | logs and the amount of logs I do keep easily should fit
               | within a minimal amount of space. Somehow systemd manages
               | to screw this up completely and I really don't think this
               | has anything to do with user error.
               | 
               | > And what were they waiting for? In the end, systemd did
               | it, but it's a pretty recent invention. There was plenty
               | time to get there first.
               | 
               | I don't know what people were waiting for. But I think
               | what systemd did could have been done in a more
               | transparent and decoupled fashion that didn't involve
               | binary logs. Just because they did something right
               | doesn't mean that the way they went about achieving it
               | isn't inherently flawed. I get that logs suck but fixing
               | that shouldn't require me to change my entire system
               | management stack.
               | 
               | > Because log daemons classically don't care about this
               | at all -- it's a problem for whatever reads the files
               | that the log daemon generates afterwards.
               | 
               | The log daemon I use (svlogd) first closes the current
               | file, renames it and then opens a new empty current file.
               | If it finds that there's now too many old logfiles it
               | deletes the oldest one. I don't see how this could
               | possibly break any tool which is reading the file except
               | for cutting the stream short when tailing during a
               | rotation. Moreover I don't think that's a problem since
               | if you need to do some kind of processing of the logs
               | then you should just put that in front of or after the
               | instance of svlogd in the pipeline (svlogd can be asked
               | to forward logs to stderr which then means you can chain
               | it with something else).
        
         | ATsch wrote:
         | > Just writing stuff to a file was fine.
         | 
         | Haha, I wish. Then that file becomes too large. So you want to
         | truncate the start. However, removing bytes from the start of
         | an open file is a bad idea. So you open a new log file every
         | day instead. You really want to compress the old log files too,
         | to save space. So now you can no longer just grep the logs as
         | before. You're now also duplicating code and configuration for
         | every daemon. So you write a daemon called `logrotate` that
         | does this centrally instead. Into which you then copy the log
         | paths configured for every program on your system and how long
         | the logs should get kept. However programs don't much like it
         | when you move the log files out from under them. So you patch
         | every program respond to a signal by checking whether it's log
         | files have been moved. Which then breaks for multi-process
         | daemons. You come up with a whole buffet of ugly hacks to make
         | that work most of the time.
         | 
         | Then you install a program which doesn't come with a logrotate
         | config. Then you decide you'd really want a more structured way
         | to analyze logs than writing fragile regexes. Then you want to
         | run something as a non-root user. Then you have an ephermeral
         | batch job. Then you want to ship off all of your logs to
         | another location and really don't want to duplicate the log
         | paths yet again. Then you want to see what happened just before
         | the server crashed last boot. Then you run two of a daemon and
         | give them different log paths. Then you want to look at your
         | kernel and application logs together. Then you want to delete
         | just last month's logs to free up some space. Also wouldn't it
         | be handy if there was some kind of index to make things faster?
         | ...
        
           | magicalhippo wrote:
           | > However, removing bytes from the start of an open file is a
           | bad idea.
           | 
           | A digression, but I keep wondering why filesystems never
           | optimized this scenario. Being able to trim the start of the
           | file is really useful in certain cases, and without support
           | options are pretty bad.
           | 
           | It also seems fairly easy to support at the cost of a few
           | bytes in the file entry (a start-of-data index into first
           | sector), and resizing a file by seeking past end is already a
           | thing.
           | 
           | I must be missing something...
        
           | Arch-TK wrote:
           | All of this seems solved by svlogd honestly.
        
             | ATsch wrote:
             | I have never used svlogd but at it looks like it does read
             | logs from it's stdin, which is a solid step forward as it
             | avoids much of the pitfalls caused by applications writing
             | directly into log files.
             | 
             | Collection is just the first part of a log system though of
             | course.
        
         | ex_amazon_sde wrote:
         | JournalD is excellent. Indexing structured data like logs is
         | the right thing to do. Log messages have a lot of useful
         | metadata to do filtering on.
         | 
         | > Systemd could completely eliminate the need for stuff like
         | docker by totally isolating a process.
         | 
         | It does: nspawn is much leaner than docker and it's available
         | on all Linux systems with SystemD.
        
           | stock_toaster wrote:
           | > JournalD is excellent
           | 
           | This literally made me laugh out loud.
           | 
           | Over the years I have historically had several production
           | system outages due to journald corrupting its own data store,
           | being unable to log, and services hanging due to being unable
           | to log (process stdout buffer full, journald not reading it,
           | so process stuck trying to write output).
           | 
           | It certainly has improved over the years, and it has been a
           | long time since this issue has happened (on one of the
           | systems I manage at least), but I have accrued some pretty
           | deep seated feelings about it at this point due to historical
           | pain.
        
         | porsager wrote:
         | That's interesting. Do you have a link to more info about
         | isolating a process like that?
        
           | comboy wrote:
           | Arch wiki is usually a good place to learn something in a
           | succinct way (I've never even used Arch):
           | https://wiki.archlinux.org/title/cgroups
        
       | memco wrote:
       | > If you need to reboot the server right now for some reason,
       | will that service come back up on reboot?
       | 
       | I didn't see this discussed more after it was asked, but I could
       | use some pointers on how to answer this question since I have a
       | service I wrote that I think should start on reboot, but has to
       | be started manually. It would be nice to know when I have fixed
       | it without having to reboot after each change to just to test.
       | Anyone know how I can do that?
        
         | viraptor wrote:
         | The answer is there in the status output:
         | Loaded: loaded ... ; enabled ...
         | 
         | If it's enabled and starts from the service file currently, you
         | can expect that it will start after a reboot. Of course you can
         | never be sure it's successful, but at least it will attempt to.
        
         | corty wrote:
         | Problem with systemd is, startup is not deterministic. systemd
         | starts everything it can start at the same time, leading to
         | subtle race conditions between services and the network. If you
         | have any dependency that isn't explicitly spelled out or any
         | service that is falsely reported as ready, you will get funny
         | system states on some boots, whereas it will work on others.
         | systemd-udev does its part to make things even worse there,
         | because it also introduces more races against kernel devices.
         | So with systemd, even when you tested the system, you can not
         | always be sure that it will start everything properly.
         | 
         | Compared to sysv-init this is a regression, because init
         | scripts usually ran sequentially and also didn't involve
         | unreliable systemd-udev events.
         | 
         | Compared to Solaris SMF this is pitiful. SMF takes into account
         | service dependencies from its declarative service descriptions
         | and computes a deterministic sequential startup order. If it
         | has booted once, it will always boot again, in the same order,
         | reliably. Best of all worlds.
        
           | viraptor wrote:
           | > Compared to sysv-init this is a regression, because init
           | scripts usually ran sequentially and also didn't involve
           | unreliable systemd-udev events.
           | 
           | This goes both ways - compared to sysv-init we finally get a
           | good way to manage services in response to device changes.
        
             | corty wrote:
             | It depends on the application. For laptops, sure,
             | hotplugging stuff is important. For servers, not so much,
             | reliability is far more important than getting your USB-
             | stick automatically recognized. systemd is heavily skewed
             | towards the unreliable but fast and dynamic laptop usecase.
        
               | viraptor wrote:
               | A server won't have many udev messages happening after
               | boot, (Past the drive and network cards) so it shouldn't
               | hold up and services either. Have you got specific
               | situations in mind that are not reliable?
        
               | corty wrote:
               | Network cards aren't simply just cards anymore. Often you
               | get multiple interfaces depending on each other, e.g.
               | bridges depending on multiple ethernet interfaces which
               | wait for STP delay to be over until they are really
               | forwarding packets. VLAN interfaces that are dependent on
               | their parent ethernet interface. VPN and other tunnels.
               | So you get stuff like "start wpa_suplicant for 802.1x,
               | take the ethernet interfaces up. start the bridge
               | interface. check if DNS works for the VPN. then start the
               | VPN daemon and the VPN interface. then start network
               | services listening on the VPN interface". That would be a
               | typical setup for a server in a remote office somewhere.
               | 
               | Another fun situation is a server with LVM, RAID and
               | encrypted disks (also typical for a remote office
               | somewhere). SAS disks will come online (after spinup
               | time), generate an event. LVM and raid detection will
               | run, generating more events, usually dependent on one
               | another. Device naming is in some cases dependent on the
               | event order, so you also have to rely on systemd-udev not
               | fucking up by-id or by-name links. Then you have
               | cryptsetup, which decrypts the disk, generating another
               | event, but only upon an automatic or manual all-clear and
               | password from the (hopefully working) network. Then you
               | arrive at the actual filesystems in fstab, after a chain
               | of udev events, each of which (and especially the LVM and
               | cryptsetup ones) sometimes fails. Same "fun" for SAN
               | devices.
               | 
               | Oh, and another fun problem is systemd exhausting
               | randomness for too many UUIDs, so cryptsetup hangs for
               | swap devices because those get a key from /dev/random.
               | 
               | Of course those bugs get often fixed over time, but only
               | until the next RedHat release arrives with yet another
               | systemd-somethingd, networkmanager, systemd-udev-bug that
               | introduces another new problem in the above. Usability
               | has gone up since RH7, but reliability is definitely
               | down.
        
               | viraptor wrote:
               | This is a long list of complexities, but I'm not sure how
               | they relate to udev/systemd. Yes, setting up aggregate
               | interfaces, VPNs and fancy drive configurations is
               | complex and can fail due to various delays. That's not a
               | problem with udev reliability if an underlying system
               | fails. We had the same issues before systemd/udev was a
               | thing.
               | 
               | > cryptsetup hangs for swap devices because those get a
               | key from /dev/random
               | 
               | You can choose urandom (third column of crypttab).
               | Systemd doesn't enforce anything here.
        
           | klodolph wrote:
           | At best, ordered startup should be optional, since boot
           | otherwise takes too long on workstations.
           | 
           | It would be nice to have hard errors when dependencies are
           | not explicitly declared. For example, if you don't declare
           | dependency on the network, your service gets put in a sandbox
           | where it can't access the network at all. In practice, that
           | kind of sandboxing sounds difficult.
           | 
           | My personal experience is that it is fairly straightforward
           | to debug races in startup. Some service fails, you look in
           | the log and see that it has a dependency that hasn't started,
           | and then add that missing dependency to the service's
           | declaration. Just speaking from personal experience--this
           | doesn't come up especially often and doesn't require a large
           | amount of time to fix, and the benefits of fast startup save
           | you much more time than the extra debugging costs you.
        
             | drewg123 wrote:
             | I reboot my workstation about once a month, and I couldn't
             | care less how long it takes. What I _do_ care about is that
             | my workstation actually reboots predictably. If it needs
             | manual intervention when I 'm traveling, I'm screwed.
        
             | aidenn0 wrote:
             | My POST takes longer than my serialized init scripts on my
             | workstation.
        
             | iso1210 wrote:
             | I rarely boot my workstation, certainly no more than once a
             | month (I don't bother with every new kernel update)
        
           | acdha wrote:
           | > Compared to sysv-init this is a regression, because init
           | scripts usually ran sequentially and also didn't involve
           | unreliable systemd-udev events.
           | 
           | I find this really odd because we spent decades with that
           | exact class of problems on SysV init, in part due to the
           | primitive dependency model and the lack of standard
           | mechanisms to launch daemons or react to events meant so many
           | different reinventions of that particular wheel (need Apache
           | to wait for NFS mounts? Here's another shell kludge to
           | maintain!).
           | 
           | Switching to systemd gave a single, simple answer for all of
           | that: use the init system. I don't know what services you
           | manage but for the mostly Debian/RHEL environments I've
           | worked on your description more fits the before state than
           | after in my experience.
        
             | corty wrote:
             | Apache waiting for NFS mounts is still not solved. Now
             | Apache may depend on a file system, but it still cannot
             | depend on "the LDAP is available so you can resolve the
             | UIDs on that NFS". So we are a tiny step ahead maybe. But
             | the problem isn't solved at all, and back in the day we
             | just solved it by reordering Apache manually all to the
             | back, maybe with a "check that UIDs are resolvable"-script
             | in front. systemd can't do that (except if you allow "evil"
             | shell-scripts in bootup again...).
        
               | acdha wrote:
               | 1. We didn't "solve" that back in the SysV days by
               | reordering - mostly we just lived with the occasional
               | race conditions and restarted it.
               | 
               | 2. You could use systemd's health check or pre-exec to
               | handle prerequisite checks like this if you can't use the
               | dependency mechanism for some reason (there's no reason
               | why you couldn't have a custom type=oneshot service set
               | as a requirement for Apache).
               | 
               | There's nothing "evil" about scripting like this - the
               | thing which makes me tired of SysV is the limited
               | standard feature set (daemonization, logging, restarts,
               | dependencies, event handling, etc.) and challenges
               | coordinating across distributions for things like
               | overrides. The wheels fall off shell scripting when you
               | need to deal with that much complexity and do so in a
               | portable manner with a bunch of shell code you don't
               | control - a health check is far less tedious to deal
               | with, especially since you have a ton of functionality in
               | systemd which avoids the need to handle the hard parts.
        
               | zxzax wrote:
               | I don't see why you couldn't do that in systemd (or any
               | other init), it's not hard to make a small unit that
               | simply runs a shell script.
        
         | rav wrote:
         | In the systemctl status output, it says "Loaded: loaded (...;
         | enabled; ...)" - "enabled" means that the service starts on
         | boot. Use "systemctl enable" to "enable" a service. It creates
         | a symlink in /etc/systemd/system/multi-user.target.wants
         | assuming the service's unit file says to enable it through the
         | "multi-user" target. By listing that directory you can see what
         | services load on boot.
        
           | CameronNemo wrote:
           | Not exactly. Socket units can pull in services that are not
           | otherwise enabled. You also have to search through all of
           | your udev rules to check if there are SYSTEMD_WANTS
           | directives pulling services in.
        
             | Arnavion wrote:
             | rav (correctly) said that enabled services are started
             | automatically on boot. They didn't say that _non_ -enabled
             | services are _not_ started automatically on boot.
        
               | nemetroid wrote:
               | They also said:
               | 
               | > By listing that directory you can see what services
               | load on boot.
        
               | Arnavion wrote:
               | Yes, and I didn't read that sentence as disallowing other
               | ways to autostart services either.
        
               | nemetroid wrote:
               | To me it's equivalent to "To see what services load on
               | boot, you can list that directory.".
        
       | ratsmack wrote:
       | >systemd is a set of building blocks that you can use to make a
       | linux system.
       | 
       | A more accurate description would be "Systemd is a set of
       | building blocks that you can use to make Linux more like
       | Windows".
        
         | dale_glass wrote:
         | systemd is incredibly Linux centric. In fact it intentionally
         | doesn't even try to be remotely portable, because an explicit
         | design aim was to expose all the cool stuff that Linux has, but
         | wasn't getting enough use.
        
           | trasz wrote:
           | Sure, but design-wise it follows Windows principles: it's
           | tightly coupled, and not particularly tweakable.
        
             | [deleted]
        
             | dale_glass wrote:
             | These days I see it as a bonus. I don't _want_ to mess
             | around with the boot process. I don 't _want_ to write some
             | sort of glue layer in between loosely coupled parts. And I
             | certainly don 't want somebody else to do it either,
             | because that too often tends to result in some kind of
             | surprise that one has to deal with at 3 AM.
             | 
             | What I want is a solid system that does what it's supposed
             | to -- start stuff -- in a maximally boring fashion.
        
             | Valmar wrote:
             | This is far from true, lmao. It's only tightly coupled to
             | depending on the Linux kernel.
             | 
             | systemd was in fact heavily inspired by MacOS's launchd. At
             | worst, it can be said that complex boot managers will begin
             | to converge on a design that actually works ~ MacOS did,
             | Windows did, and now Linux has with systemd. The three have
             | plenty of differences, though systemd is still closer to
             | launchd than not.
             | 
             | And in actuality, it's very tweakable. Did you notice that
             | the source code is open, for a start...?
        
       | viseztrance wrote:
       | I ignored systemd because I thought it didn't affect me, as I
       | only run linux (fedora) on my desktop, and hardly had any issues
       | at that.
       | 
       | But recently, when I changed my fstab file to mount a drive at
       | boot, I made a typo. My system wouldn't boot at all and it
       | wouldn't even drop to a shell so I can chroot the file system
       | like you would with init.d.
       | 
       | I thought that there's some arcane command I don't know about.
       | But there wasn't. You're locked off, and expected to boot off
       | something else to fix your system. Horrible.
       | 
       | Having this said, I honestly expected way more on the section
       | about benefits to users.
        
         | zaarn wrote:
         | Atleast on Arch, Systemd should drop you into the Emergency
         | shell if it cannot mount a device that isn't defined as
         | optional. IIRC it might require a root password to be set,
         | however.
         | 
         | Otherwise, edit the kernel commandline to add
         | "systemd.unit=emergency.target" to the end, which triggers
         | systemd to straight boot into this console without trying
         | anything that it doesn't need to bring up the console.
        
           | viseztrance wrote:
           | It tried when the mounting failed, saying it would drop me to
           | a shell and then said "it fail to open up a shell".
        
             | zaarn wrote:
             | That usually indicates something is very wrong and might
             | need an init=/bin/bash instead, as a last resort, to find
             | out why it couldn't open the shell (usually, because there
             | is no root password or it couldn't find any shell binary to
             | run).
        
           | otterlicious wrote:
           | Or if you're blocked by the root password prompt add
           | `init=/bin/bash` and make your filesystem writable if
           | necessary with `mount -oremount,rw /`
        
             | zaarn wrote:
             | This will block systemd however. The nice thing about the
             | emergency shell is that you can use systemd things like
             | trying to mount filesystems. And you can later isolate
             | multi-user to continue the boot when you finished your fix.
             | /bin/bash would be atleast useful to try to get a root
             | password set.
        
         | ajross wrote:
         | > But recently, when I changed my fstab file to mount a drive
         | at boot, I made a typo. My system wouldn't boot at all
         | 
         | I don't know exactly what went wrong in your case, but killing
         | Unix systems by messing with fstab has been a rite of passage
         | for more than four decades. Systemd certainly didn't invent
         | that. Hell, I broke a chromebook not 48 hours ago messing with
         | the boot setup.
         | 
         | But FWIW: managing a recovery image is, amusingly, _not_
         | historically a job systemd has tried to take on. This is what
         | your live image is for.
        
           | viseztrance wrote:
           | I've been using linux on my desktop for almost 20 years. This
           | was the first time I couldn't get a shell to fix my problem.
        
         | ATsch wrote:
         | It sounds like what you ran into are the effects of the general
         | trend of doing more in early boot, as
         | https://freedesktop.org/wiki/Software/systemd/separate-usr-i...
         | elaborates on, which is independent of systemd
        
           | viseztrance wrote:
           | Thanks for sharing this.
        
       | kbenson wrote:
       | I think systemd brought a lot of good stuff to the table, but I
       | also think it's a sprawling mess with poorly named and documented
       | (as in there are behaviors implied by statements in different
       | config directive sections that make it hard to reason about how
       | stuff works).
       | 
       | I'm looking forward to the successor of systemd, or at a minimum,
       | them rolling out config syntax 2.0 which breaks compatibility in
       | an effort to make stuff more sanely named and documented.
       | 
       | I know that would cause huge churn. I know it will never happen.
       | But I can dream, because I don't think anyone can look at how
       | systemd configs define dependencies and say that they both
       | understand the nuances and that those directives are are named
       | sanely in a way that normal people can understand, and that's
       | indicative of a lot of stuff with systemd.
       | 
       | Even though it was probably designed by some of the most
       | knowledgeable people WRT init systems, it grew organically as
       | they discovered new things they needed, and it needed a lot
       | (which I think just goes to show how little was really understood
       | by anyone about how to tightly couple these things).
        
       | Decabytes wrote:
       | I just learned about Vtubers this week. I would have never
       | thought this would have crossed into the Linux world but here we
       | are!
        
       | efiecho wrote:
       | Poetteringware seems to be highly praised by mostly young people
       | who would like Linux to be a Windows clone. People who
       | understands Unix concepts tends to dislike it.
        
         | StreamBright wrote:
         | Well yes and no. I like the good parts of SystemD (not the
         | parts this video is talking about).
         | 
         | For me there are 3 things that SystemD gets right:
         | 
         | - no shell scripts for service configurations
         | 
         | - no forking services (no need for daemon mode anymore)
         | 
         | - logging and service management is tightly coupled (this is
         | arguably good with some bad parts, required CPU for logging
         | with systemd is not great)
         | 
         | And the rest is just pure madness.
         | 
         | - DNS (but whyt?)
         | 
         | - NTP (but why????)
         | 
         | - system limits (seriously, da f???)
         | 
         | - pid 1 should be a very simple service (More:
         | https://ewontfix.com/14/)
         | 
         | I have spent 25 years on Unix, so I guess I understand the
         | concepts.
        
           | larschdk wrote:
           | DNS + NTP are completely optional, but you get really good
           | integration with systemd-networkd's DHCP client if you use
           | them. Switching network is completely seamless, and the
           | resolver updates fully automatically (direct DNS and NTP are
           | blocked in many networks, sadly). Also, you get something
           | that works out of the box, with essentially 0 configuration.
           | You can't even install a system today over the internet
           | without setting the time, and systemd-timesyncd does it with
           | 0 configuration.
        
           | Arch-TK wrote:
           | > - no shell scripts for service configurations
           | 
           | Other than "sysvinit based init had horrific shell scripts
           | with 100 lines of repeated nonsense", given that almost every
           | alternative to systemd which still uses shell scripts needs
           | around 2 lines of shell per service (including the shebang)
           | what exactly is the issue with shell scripts (if all they do
           | is run a wrapper which sets up the environment of a program
           | or the program itself)?
           | 
           | > - no forking services
           | 
           | When I used archlinux these were extremely common and I used
           | archlinux as recently as last year.
           | 
           | > - logging and service management is tightly coupled (this
           | is arguably good with some bad parts, required CPU for
           | logging with systemd is not great)
           | 
           | So runit handles spawning a log daemon before the service is
           | started to ensure all log entries are kept and comes with a
           | really simple yet extremely powerful log daemon (but
           | obviously you can use almost anything else including cat if
           | you want). It manages to integrate logging tightly into
           | service supervision while being maximally flexible and
           | decoupled. What do you think about this?
        
             | 1vuio0pswjnm7 wrote:
             | I use daemontools instead of runit. What am I missing.
             | 
             | multilog is the simplest (smallest) and most versatile log
             | rotating daemon I have found.
        
               | Arch-TK wrote:
               | A few extra features, but you probably don't need them. I
               | use svlogd because I use runit and I use runit because it
               | happens to be supported by my distro.
        
             | StreamBright wrote:
             | OK I bite. Nothing is wrong with shell scripts. However, I
             | would like to know if something _wont_ work. Shell scripts
             | do not have this feature. You know if something does not
             | work when you run it. There might be external dependencies
             | missing or other very typical problems. If we have a way to
             | push all of these to "compile time" it is an improvement. I
             | also spent many years in the land mines of implicit
             | overrides in shell script hells and thank you but no. As a
             | good example you can have how people try to set JVM
             | parameters in a big data project using shell scripts. I it
             | is absolute insanity. Same thing in init scripts. Did you
             | implement reload or restart? Is stop actually doing
             | something or a noop. And so on.
             | 
             | If there is only one thing that I could keep from systemd,
             | it would be the use of unit files instead of shell scripts.
             | 
             | The rest is your observations, nothing to disagree or agree
             | with.
        
         | tristan957 wrote:
         | No it is the old people that don't understand the systemd
         | project is exactly the Unix philosophy
        
           | dijit wrote:
           | "Old people", I guess I meet the definition _now_ but When I
           | first encountered systemd and used it as my daily driver I
           | was 21.
           | 
           | I'm 31 now.
        
       ___________________________________________________________________
       (page generated 2021-05-16 23:00 UTC)