[HN Gopher] Unix's technical history is mostly old now
___________________________________________________________________
Unix's technical history is mostly old now
Author : ingve
Score : 54 points
Date : 2022-11-25 07:54 UTC (2 days ago)
(HTM) web link (utcc.utoronto.ca)
(TXT) w3m dump (utcc.utoronto.ca)
| joeatwork wrote:
| Rob Pike's talk "Systems Software Research is Irrelevant" (given
| 22 years ago!) provides some nice color here - we kinda just
| settled on "good enough" in a lot of ways:
| http://doc.cat-v.org/bell_labs/utah2000/utah2000.html
| throwawaylinux wrote:
| And in that time, the state of the art in systems software has
| advanced so dramatically as to be almost unrecognizable to
| someone that short length of time ago. In programming
| interfaces, bringing the hypervisor into the OS, concurrency
| and scalability and performance techniques, etc.
|
| I think the subject is right: systems software "research" is
| irrelevant. Because academia could no longer keep up.
| physPop wrote:
| "All these things academics invented that are now
| commonplace..."
| throwawaylinux wrote:
| Academics didn't invent them.
|
| IBM and Bell Labs, maybe.
| aidenn0 wrote:
| None of those things is novel in academia though...
| throwawaylinux wrote:
| No, they couldn't be, because academia didn't invent them.
| I think academia has been too busy proving how wonderful
| microkernels are for the past 50 years to do much useful
| real systems work.
| neilk wrote:
| > (there has been no large move to adopt ACLs or file attributes,
| for example, although file capabilities have snuck into common
| use on Linux systems)
|
| This is largely true for any server you're likely to ssh into.
| But the most common Linux distributions are Android which make
| extensive use of SELinux features.
| skissane wrote:
| Once upon a time, it was common to have time-shared multiuser
| Unix systems - a single server and everyone in the department
| could login and use it simultaneously. Filesystem permissions -
| first the classic mode bits, then ACLs - were originally
| designed for that use case.
|
| While such systems still exist, they are a much smaller
| percentage of all installed systems than they used to be. Most
| contemporary Unix(-like) systems fall into one of two
| categories:
|
| (a) single-user machines (such as a laptop)
|
| (b) application servers, database servers, etc, which serve
| many users, but those users are defined at the application
| layer, the OS and filesystem don't know anything about them
|
| Maybe part of the reason why filesystem ACL adoption is weaker
| than people expected, is that (a) and (b) don't have the same
| need for filesystem ACLs as the classic multi-user timesharing
| environments do.
|
| > But the most common Linux distributions are Android which
| make extensive use of SELinux features
|
| SELinux context and filesystem ACLs are two separate things. I
| see many machines with heaps of the former and little of the
| later (such as most Red Hat boxes).
|
| Even with single-user machines, there is still some value in
| inter-process security - application sandboxing, corporate-
| managed devices, etc - but there are many technologies
| available now to meet those requirements, and filesystem ACLs
| are often not the most useful among them.
| eru wrote:
| We still do something like time-sharing these days, but often
| it's on the level of VMs instead of OS processes.
| lkrubner wrote:
| If you have the slightest interest in the history of operating
| systems, I strongly recommend reading Joe Armstrong's thesis.
| While it is mostly about Erlang, he also goes into detail about
| many of the experimental systems of the 1960s, 1970s and 1980s,
| which gave him his ideas:
|
| https://erlang.org/download/armstrong_thesis_2003.pdf
|
| Chapter 5 is especially relevant:
|
| excerpt:
|
| What is a fault-tolerant system and how can we program it? This
| question is central to this thesis and to our understanding of
| how to build fault-tolerant systems. In this chapter we define
| what we mean by "fault-tolerance" and present a specific method
| for programing fault-tolerant systems. We start with a couple of
| quotations:
|
| We say a system is fault-tolerant if its programs can be properly
| executed despite the occurrence of logic faults. -- [16]
|
| ...
|
| To design and build a fault-tolerant system, you must under-
| stand how the system should work, how it might fail, and what
| kinds of errors can occur. Error detection is an essential com-
| ponent of fault tolerance. That is, if you know an error has
| occurred, you might be able to tolerate it by replacing the
| ocending component, using an alternative means of compu- tation,
| or raising an exception. However, you want to avoid adding
| unnecessary complexity to enable fault tolerance be- cause that
| complexity could result in a less reliable system. -- Dugan
| quoted in Voas [67].
|
| The presentation here follows Dugan's advice, I explain exactly
| what happens when an abnormal condition is detected and how we
| can make a sodware structure which detects and corrects errors.
|
| The remainder of this chapter describes:
|
| * A strategy for programming fault-tolerance -- the strategy is
| to fail immediately if you cannot correct an error and then try
| to do some- thing that is simpler to achieve.
|
| * Supervision hierarchies -- these are hierarchical organisations
| of tasks.
|
| * Well-behaved functions -- are functions which are supposed to
| work correctly. The generation of an exception in a well-behaved
| function is interpreted as a failure.
| rramadass wrote:
| Just an FYI in case you don't already know of this, i found the
| following two papers by Armstrong useful too;
|
| * A History of Erlang
|
| * The Development of Erlang
| retrac wrote:
| This is why I'm interested in old, now mostly obsolete/dead
| operating systems. No one has really felt free or able to break
| out of the UNIX mould, since. And we've been using the mould for
| a long time. It wasn't always so uniform. There have been some
| craaaaazy things out there, that we simply never see today
| because it doesn't fit. Off the top of my head:
|
| * ITS (of PDP-10 hacker fame) - processes could debug and
| introspect their child processes. The debugger was always
| available, basically. The operating system provided support for
| breakpoints, single-stepping, examining process memory, etc.
| Whatever part of the system manages processes, is the natural
| place for the debugger, really. Some of this has been brought to
| Unix systems recently, but you still can't trivially freeze a
| process, tweak a few values in its memory, and resume it as an
| integrated part of the operating system. Why not? It seems very
| basic to an operating system, now that I think about it.
|
| * KeyKOS (developed by Tymshare for their commercial computing
| services in the 1970s) - A capability operating system. If
| everything in UNIX was a file, then everything in KeyKOS was a
| memory page, and capabilities (keys) to access those pages. The
| kernel has no state that isn't calculated from values in the
| virtual memory storage. The system snapshots the virtual memory
| state regularly. There are subtle consequences from this.
| Executing processes are effectively memory-mapped files that
| constantly rewrite themselves, with only the snapshots being
| written out. Snapshotting the virtual memory state of the system
| snapshots everything -- including the state of running processes.
| There's no need for a file system, just a means to map names to
| sets of pages, which is done by an ordinary process. After a
| crash, processes and their state are internally consistent, and
| continue running from their last snapshot. For those who are
| intrigued, there's a good introduction, written in 1979, by the
| system's designers available here: http://cap-
| lore.com/CapTheory/upenn/Gnosis/Gnosis.html (It was GNOSIS before
| being renamed KeyKOS.) And a later document written in the 90s
| aimed at UNIX users making the case: http://cap-
| lore.com/CapTheory/upenn/NanoKernel/NanoKernel.ht... Some work on
| capability systems continues, but it seems the lessons learned
| have largely been forgotten.
| eru wrote:
| Ironically, Microsoft's Windows is one of the only big non-Unix
| operating systems out there. I say ironically, because people
| seldom bring up the Microsoft ecosystem as a beacon of adding
| to diversity.
| frankharv wrote:
| Didn't Windows Network stack come from NetBSD? There were
| many BSD contributions to Windows.. BSD386 was a direct Unix
| derivative I would argue.
| eru wrote:
| Of course, Windows (and especially Windows NT) learned and
| borrowed from the Unix world. But it retains a distinct
| personality and lineage.
| smegger001 wrote:
| Correct me if i am wrong but isn't the NT design heavily
| based off of (some would say a rip off of) VMS, Microsoft
| having poached much of DEC's OS dev team including chief
| architect David Cuttler.
| dhosek wrote:
| I think that a lot of interesting things were lost from the IBM
| and DEC operating systems. I spent a lot of time in VM/CMS and
| VAX/VMS (plus a little MVS/TSO) in the 80s and 90s and my Unix
| time was relatively limited for quite a while. I really liked
| Rexx (IBM's scripting language which they brought into OS/2,
| but is now pretty much dead) and the way that command line
| options for VMS programs could be declared independently of the
| executable (plus its automatic allowance for abbreviations
| where you could abbreviate a command to its minimal unique
| prefix).
| AlbertCory wrote:
| Sometime in the 90s, I was at Oracle and got invited to what was
| probably the most boring meeting I've ever attended in my life:
|
| A bunch of systems admins who were puzzling over how to create a
| standard way to administer Unix systems. The premise was "in the
| old IBM mainframe days, every shop ran the exact same way, so it
| was easy to train a new hire. Now, with Unix, there are a million
| different customs, so how do we standardize it?"
|
| You probably never even thought this was a problem. My big
| problem was staying awake. I guess now that it's all Linux, it's
| easier. Back then, there was HP-UX, IBM AIX, Sequent's whatever-
| it-was-called, Pyramid, SCO & all the other PC variants...
|
| The group was called Moses, which makes it almost unsearchable
| since there are so many other uses of that word.
| tjr225 wrote:
| meadhbh-hamrick wrote:
| meadhbh-hamrick wrote:
| It sounds almost like the OP is bemoaning the stability of Unix-
| like systems. I like that ls, cd, mkdir, emacs and cp all work
| more or less the same way. And as much as I sometimes curse at
| autotools, I appreciate how nice it is I can generally just type
| `./configure` and something reasonable happens.
| sanxiyn wrote:
| Many important things happened to Unix in 1990s and 2000s. dpkg
| and apt come to mind for 1990s, and udev and systemd for 2000s.
| asveikau wrote:
| I was about to name some innovations from Linux and *BSD
| worlds, but that is not technically Unix. (*BSD is Unix derived
| but cannot use the trademark). So I think your example is not
| technically Unix technical history.
|
| But if we could include these things, I'd say also:
|
| * Epoll and kqueue. The recognition that select(2) and later
| poll(2) do not scale.
|
| * Interactivity. One thing I remember about 90s Unix is how
| terrible the usability was on those old terminal emulators or
| termcaps. You'd type a backspace and see ^H. Arrow keys that
| didn't do the right thing. You'd log into a Linux box and by
| contrast you'd get good tab completion, colors, etc.
|
| * Secure defaults. First noticed this in OpenBSD. But the
| internet forced everybody to reconsider the set of daemons you
| get on a default install.
|
| * Deprecation of unsafe libc functions
| sanxiyn wrote:
| "Linux is not Unix" is useless pedantry, but sure, Solaris
| SMF did it earlier than systemd, both dependency based boot
| and parallel boot.
| contingencies wrote:
| _Hardware has changed dramatically; software is stagnant._ - Rob
| Pike (2000)
|
| ... added to https://github.com/globalcitizen/taoup
| johannes1234321 wrote:
| > I think that if you took a Unix user from the early 1990s and
| dropped them into a 2022 Unix system via SSH, they wouldn't find
| much that was majorly different in the experience.
|
| That is probably true as many of those old tools kept
| compatibility, but at the same time we got a whole bunch of new
| tools, more powerful than `ps` for gaining more insights. For an
| example look at all the changes in service management, eBPF and
| such. Not to mention that the shell prompt itself (see zsh etc.)
| is more powerful than back then, which could give a very
| different experience.
|
| So yeah, old tools keep mostly working, but improvement goes on.
| chungy wrote:
| Linux's refusal to adopt RichACLs/NFSv4 ACLs forever perplexes
| me, and maybe some day I'll be pleasantly surprised when they do
| show up.
|
| A complete superset of NTFS's ACLs, thus providing good cross-
| platform compatibility, and already implemented in illumos, Mac
| OS X, and FreeBSD, Linux is the only holdout on them.
| wmf wrote:
| How many people use ACLs?
| pinewurst wrote:
| In the real world only a small minority even use NFSv4 and
| that mostly for the figleaf of encryption via (yuck!)
| Kerberos. As a (too) long time storage industry person, I
| have hard numbers on this.
| chungy wrote:
| It's primarily a corporate-level thing, but just as Windows
| has a strong set of ACLs that people at home ignore, it could
| be the same scenario on Linux.
| asveikau wrote:
| This is a Yogi Berra type comment. "ACLs aren't supported.
| Nobody uses them."
| wmf wrote:
| Not really. Linux supports "old" posix ACLs and chungy is
| complaining that it doesn't support "new" rich ACLs. My
| point is why bother upgrading a feature that no one uses.
| nerdponx wrote:
| I have used them to set up a group-readable/writable
| directory to ensure that all subdirectories remain so.
| throw0101c wrote:
| > _Linux 's refusal to adopt RichACLs/NFSv4 ACLs forever
| perplexes me, and maybe some day I'll be pleasantly surprised
| when they do show up._
|
| As someone who has had to deal with NFSv4-styles ACLs (on an
| Isilon server handling Linux HPC clients): they can get really
| messy, really quickly (especially the inheritance aspect of
| them).
| tyingq wrote:
| _" I think that if you took a Unix user from the early 1990s and
| dropped them into a 2022 Unix system via SSH, they wouldn't find
| much that was majorly different in the experience."_
|
| I agree. In fact, since there's fewer actively used unix
| variants, the same job would seem simpler in many ways. I think
| the fragmentation of tech above the OS would be much more
| intimidating. So many more languages, build+deploy systems,
| frameworks, things like api gateways, caching platforms, load
| balancers, and so on. In the 1990's an admin could maintain a
| decent amount of personal knowledge about what a developer might
| be doing on the platform.
| mannyv wrote:
| As someone who used to administer solaris, sunos, hpux, and aix
| boxes, i can almost guarantee that the first thing they'd say
| is "What the fuck is up with this systemd shit?" Because i say
| that all the time.
| semireg wrote:
| In my early 20s I was a Linux sysadmin contractor in the
| Midwest US and got dropped into a job site where the core team
| was being offshored. I had to quickly learn how to maintain
| HPUX, AIX and some Solaris boxes using docs in the form of word
| docs scattered across network drives. These old admins hated
| Linux... and I hated the unfamiliarity of the non-Linux
| systems. How could anyone live with a shell where tab
| completion wasn't a feature? Gag!
| Cupertino95014 wrote:
| You couldn't count on emacs being available on some random
| Unix system. Bad times, those were.
| AnIdiotOnTheNet wrote:
| I feel the same way today when I can't tab complete parameter
| or variable names in typical unix shells.
___________________________________________________________________
(page generated 2022-11-28 05:00 UTC)