[HN Gopher] The wild world of non-C operating systems
___________________________________________________________________
The wild world of non-C operating systems
Author : dharmatech
Score : 249 points
Date : 2022-03-30 06:07 UTC (1 days ago)
(HTM) web link (www.theregister.com)
(TXT) w3m dump (www.theregister.com)
| rbanffy wrote:
| Fun to think Clearpath is the oldest operating system in
| production, since 1962, and that's still fully supported.
|
| Comes close to another one that also predates C, IBM's z/OS,
| which has parts written in their own assembly dialect, as well as
| PL/1 and others.
|
| Next week we'll probably see a new version of it with support for
| the new z16 (based on the Telus chip they've shown at the last
| Hot Chips).
| qubex wrote:
| I'm delighted they mentioned TAOS and (following through) the
| amazing Transputer architecture of the 1980s. Being a teen
| 'interested' in computers in the early 1990s I was fascinated by
| massively parallel architectures (Connection Machine 1/2 and
| Connection Machine 5, IBM's SP/2 "scalable parallel" RS/6000
| based machines, and the Transputer concept) and I'm still
| figuring out whether GPUs are true embodiments of that concept or
| not.
| dfox wrote:
| Massively parallel was just an marketing buzzword that was used
| for a lot of different things. One thing to look for is wether
| all the compute elements run from same clock or not.
|
| CM1/2 is SIMD machine that is mostly nothing more than FPGA
| turned inside out and in fact just an accelerator that cannot
| do anything useful standalone (control flow is mostly done in
| software on the frontend machine). In this regard it is
| somewhat similar to today's GPUs.
|
| SP/2 is more or less equivalent of the "Beowulf clusters" of
| the turn of certury and Slashdot fame, that is bunch of
| computers with full OS instances connected together with some
| reasonably fast network. But done few years before on somewhat
| specialized IBM hardware (And well, Myrinet of the early large
| Linux based clusters was kind of funky specialized hardware,
| with these 18pair Cat5 cables...).
|
| CM5 is weird as it is cluster of more or less off the shelf
| SPARC nodes with several independent communications networks
| with one of the networks being capable of doing bunch of
| "collective communications" (reduce, broadcast...) operations
| in hardware. As with CM1/2 it does not run any kind of OS and
| is mostly managed by some set of frontend computers than do the
| job setup and network configuration for each new job.
|
| And then transputer is not an concept, there was real hardware
| and solar system is full of things that are directly descended
| from that hardware. The issue with transputers as an HPC
| platform (even for the early 90's) is that the nodes are too
| small, slow and have too slow interconnect to be meaningfully
| usable as general purpose HPC platform because you end up using
| significant fraction of the potential performance for
| housekeeping tasks (notice that above mentioned CM1/2/5 has
| some out of band method for the frontend to manage the nodes
| and even directly access their local memory, on transputers
| this had to be done through the main communication network and
| supported by software running on the nodes themselves, which
| makes even booting the thing an interesting excercise in
| parallel programming).
| yvdriess wrote:
| The Tilera [1] processors were a recent embodiment of the
| Transputer. (comedy option: the TIS-100 [2])
|
| Modern GPU architectures are very much their own thing. They
| have more in common with Vector machines in how the ALUs are
| organized, but the macro control flow is more like Dataflow
| architectures.
|
| [1] https://en.wikipedia.org/wiki/TILEPro64 [2]
| https://en.wikipedia.org/wiki/TIS-100
| mst wrote:
| TIS-100 is my current favourite puzzle game and the plethora
| of implementations on github plus the various community
| resources (including a rather good subrediit) make it even
| more fun. If it sounds interesting, I recommend
| https://alandesmet.github.io/TIS-100-Hackers-Guide/ as a
| starting point.
| jliptzin wrote:
| Was this written by a machine?
| weare138 wrote:
| This article misses a ton. In addition to all the ones mentioned
| already what about all the early operating systems written in
| assembly like Apple DOS or Commodore DOS and the Pascal family of
| OSs like Mac OS Classic and Toro Kernel.
| em3rgent0rdr wrote:
| and MS-DOS [1] and BareMetal [2].
|
| [1]
| https://web.archive.org/web/20170506152047/http://www.paters...
|
| [2] https://github.com/ReturnInfinity/BareMetal
| bitwize wrote:
| MCP (as in the Burroughs OS) stood for Master Control Program.
| The villain in _Tron_ was named for the OS. The technical
| consultant for _Tron_ was none other than Alan Kay -- himself a
| huge Burroughs fan.
| skadamat wrote:
| Yeah! Also I think Alan Kay's wife (Bonnie MacBird) was the
| writer for the movie:
| https://en.wikipedia.org/wiki/Bonnie_MacBird
| bitwize wrote:
| Yep. They met and fell in love while working on Tron.
| Doubtless Bonnie thought that MCP Alan was talking about made
| a cool villain name. Also, Tron's programmer Alan Bradley is
| named after Kay.
|
| The major conflict in Tron is also themed after Kay's
| thought: Whom should a computer serve, its users or the
| bureaucracy that administrates it? Kay believed in user-
| centric computing, and was profoundly influential in making
| that happen, but it's clear that this struggle still goes on
| today.
| dboreham wrote:
| MCP written in some version of Algol, iirc.
| bitwize wrote:
| ESPOL, a systems dialect of ALGOL.
|
| Uniquely for a machine at the time, you _didn 't_ program a
| Burroughs in assembly. The tools just weren't available.
| Everything from the kernel to user applications was written
| in some HLL.
| p_l wrote:
| Porting a mention from the other submission of this article:
|
| It misses the big name of Multics, written in PL/I, and direct
| inspiration (both positive and negative) for Unix
| ChuckMcM wrote:
| And Smalltalk. But this was a great article which highlighted
| some of the creativity that got me interested in computers in the
| first place. Building an operating system isn't "hard" but it is
| a lot of work the more you do. That is time consuming (but in a
| fun way, like reading books is time consuming)
|
| Off and on I've experimented with a small event driven real time
| OS for my robots that started as a Forth kernel, switched over to
| a C kernel, and then to a Rust kernel. The experimental part has
| been looking at "infinite asynchronous threading" and it was
| hugely influenced by Rodney Brook's subsumption architecture. My
| thesis being that organisms aren't driven by a single scheduler
| and a bunch of processes, why should an OS be. (there are
| arguments pro and con on that one :-)).
|
| Anyway, not everyone is an 'OS' type. At Sun we used to joke you
| were either an 'OS' type, 'Language' type, or a 'GUI' type. Those
| were the core areas of investigation in the software
| organization. I tend to be an 'OS' type with a bit of 'language'
| thrown in :-)
| MaxLeiter wrote:
| I've been on a Smalltalk binge the last few days and have
| really been enjoying this two hour long interview / demo with
| Dan Ingalls on Smalltalk and the Alto:
| https://youtu.be/uknEhXyZgsg
| OnlyMortal wrote:
| I remember the original MacOS been a mix of 68K and Pascal.
|
| I tended to use MPW C and 68K in those days and recall the stack
| sizes been different between C and Pascal.
| dmtroyer wrote:
| click-bait title for programmers if I ever saw one.
| pyinstallwoes wrote:
| DSP is a language and operating system (forth-like) made in the
| Soviet Union. I believe it was originally crafted upon ternary
| computation hardware but eventually transitioned to whatever was
| being used throughout the Union.
|
| http://www.euroforth.org/ef00/lyakina00.pdf
|
| Apparently they came to the syntax independently of Forth.
| However my research shows that some material about Forth may have
| showed up in a journal around the same time DSP was made. Either
| way the resemblance is uncanny. There is an interesting quote
| about the interface of the design requirements that manifested as
| the language: "DSP wasn't invented, it was discovered" - I
| probably butchered it. But I'm pretty sure Chuck may have said it
| too or at least agrees.
|
| The point is, you reach this interface following a method of
| reaching for something akin to "irreducible computational
| language that makes sense between humans and machines."
| ncmncm wrote:
| The article falsely claims, "Rust is eclipsing C++". "Rust is
| desperately chasing after C++" would be accurate.
|
| It also fails to mention Apollo Aegis, coded in Apollo's extended
| Pascal, and both at introduction and retirement was more advanced
| than Unix. Some key features still are not seen in Linux or most
| BSDs.
|
| And, it fails to mention SerenityOS, in C++.
| ModernMech wrote:
| It depends on what they mean by popularity. If they mean % of
| developers using Rust vs C++ they are wrong. But I teach
| students both C++ and Rust, and by far Rust is preferred. It's
| never even a contest.
|
| C++ is long in the tooth, and if you want to talk desperation,
| C++ is really struggling with its identity now that Rust is on
| the scene. Before there were some pretty good arguments to use
| it, but in light of Rust, the best argument for C++ is just
| that it has a larger ecosystem (which says more about how old
| C++ is than how good it is as a language). That argument
| becomes less convincing with each passing day.
|
| IMO in 2022 there's not a good reason to start a new project in
| C++ unless that's all you know, or you have some constraint
| tied to C++.
| Koshkin wrote:
| > _by far Rust is preferred. It's never even a contest_
|
| But it is (for the most part) C++ that pays the bills.
| tialaramex wrote:
| Judging from the same Stack Overflow polling, only by about
| 3:1 and so we might also say it's really _Javascript_
| paying the bills since that 's what is most used rather
| than C++. Just not to write operating systems.
|
| If we're thinking about University students it's very
| likely the ratio is even smaller by the time they graduate
| and are looking for employment. Rust jobs grow, there are
| lots of new Rust codebases, and they need maintenance
| people just the same as C++. You may have fewer mysterious
| unsafety bugs to fix but your customers still have changing
| requirements, other people's APIs still change, so you
| aren't going to go hungry any time soon programming in Rust
| any more than C++
|
| And if we're talking about in-house training, whatever they
| are teaching pays the bills, the employer isn't sending
| employees to learn how to do stuff they've no use for,
| otherwise they'd just give you indefinite leave on full
| pay.
| pjmlp wrote:
| Those students are up for a surprise when they get to do
| anything graphics or HPC related.
| aeonik wrote:
| Can you explain more? I don't have any experience to infer
| what your comment means.
| pjmlp wrote:
| HPC is all about C, C++, Fortran, Python (as scripting
| language), and now Julia is slowly being the new kid in
| town.
|
| Stuff like OpenACC, OpenMP and MPI are expected to be in
| the box, regardless of the language.
|
| On the graphics space, Khronos, NVidia, Intel and AMD are
| driving the adoption of standard C++ as compute language.
|
| NVidia now has many key contributors to ISO C++ on their
| payroll and as shown at GTC 2022, wants to make CUDA the
| best GPU implementation of such features.
|
| Then on game consoles, C++ is the main language on the
| SDKs, alongside C# and Unity.
|
| One can naturally use other stacks, like Embark Studios
| is doing with Rust, but then it is on them to spend
| development resources on making it work, and deal with
| possible integration issues.
| scythe wrote:
| Most of the interfaces you're discussing are more-or-less
| "purely functional", though. Interop overhead is going to
| be a lot less when you're squaring a matrix than when
| you're trying to write an event loop. In fact, most
| people who use those C++ libraries never touch C++, as
| you've pointed out yourself: they use Python or
| (recently) Julia. The ecosystem as it exists is already a
| hodgepodge of C[++] and Fortran, and while the interop
| between those might be annoying if you were building a
| video game, a NumPy user doesn't care if one subroutine
| is in C++ and the next one is in Fortran. So, I don't see
| why one more backend language would be a problem,
| realistically.
| tux3 wrote:
| I think they mean that C++ has a lot of momentum in
| graphics and HPC.
|
| So in their model of the world, students who learn Rust
| would be surprised that there are people who do not use
| Rust.
| ncmncm wrote:
| They will find that their Rust experience is worth
| little, because they will need to go back and learn C++
| to make any progress in those, and indeed most areas.
| stouset wrote:
| This statement is true for literally any pair of
| [technology] and [thing-theres-a-better-technology-for].
| dan-robertson wrote:
| If you have a very deep understanding of C++ then you are
| potentially yourself a good reason to start a project in C++
| and this statement is true of any language. Knowing the
| language very well can mean more time focusing on the actual
| goals and less time messing around with incidental language
| things. C++ is not sufficiently bad that mastery of it
| doesn't trump other concerns for personal projects. Even in
| companies you are often likely to find more C++ expertise
| than Rust expertise.
| ilovecaching wrote:
| I don't think you understand just how much of your system and
| the libraries out there are C/C++. Popularity isn't a measure
| of entrenchment. C/C++ will likely exist forever because the
| industry isn't going to rewrite every single piece of the
| system into Rust. And it isn't like you can just go to one
| community and ask them to switch. These are huge federations
| that talk to each other using C/C++ bindings and have hashed
| out protocols over years and years of board meetings. I think
| Rust has a place. We can build new things in Rust or replace
| some things with Rust. But acting like everyone should learn
| Rust and C/C++ has no value is dangerously ignorant.
| zozbot234 wrote:
| C library bindings can be used directly in Rust, and there
| are facilities to improve interfacing with C++ as well.
| pjmlp wrote:
| Try to use those facilities with CUDA or Unreal.
| DrBazza wrote:
| > But I teach students both C++ and Rust, and by far Rust is
| preferred. It's never even a contest.
|
| Why is Rust preferred though? Is it "fearless concurrency"
| (or whatever the old tag line was), a more modern stdlib, or
| is it Rust's tooling with `cargo` rather the shitty tool-
| scape of C++'s make/cmake/meson and vcpkg/conan and whatever
| else.
| e3bc54b2 wrote:
| As a newbie who only learned Java and JS in bootcamp, and C
| in EE, Rust's tooling is absolutely phenomenal.
|
| Things just work, and that is a rare experience.
| ModernMech wrote:
| It's a rare experience but once you've had it, anything
| less becomes unacceptable. Rust has ruined me for so many
| languages, my bullshit tolerance used to be much higher.
| jcelerier wrote:
| Can you point to a Rust alternative to
| https://doc.qt.io/qt-5/qtreeview.html ?
| steveklabnik wrote:
| https://slint-ui.com/ is an in-progress one from some ex-
| Trolltech folks.
| jcelerier wrote:
| As much as I have an extreme confidence in the technical
| ability of the involved folk, even TQTC did not manage to
| replicate it in an useful enough way so far in QML, and
| QML has been around for something like 11 years now.
| steveklabnik wrote:
| Yeah, I'm certainly not claiming it's equivalent today, I
| haven't even tried a demo. Just a thing to watch if
| you're interested in this space.
| scoutt wrote:
| Exactly. I think you managed to summarize all my concerns
| in a single sentence.
|
| https://www.areweguiyet.com/
| kllrnohj wrote:
| > C++ is long in the tooth
|
| C++ is mature, there's a difference. It's definitely not
| "long in the tooth" as it still has plenty of modern features
| & niceties and is getting more all the time.
|
| > C++ is really struggling with its identity now that Rust is
| on the scene
|
| C++'s identity struggles have nothing to do with Rust
| whatsoever. It's struggling to balance ABI & historical
| compatibility with moving the language forward, that's about
| it. And that's just something many mature languages face at
| some point (see Python3 for example).
|
| Beyond that struggle of supporting legacy code, there's no
| identity crisis with C++ at all.
|
| > the best argument for C++ is just that it has a larger
| ecosystem (which says more about how old C++ is than how good
| it is as a language). That argument becomes less convincing
| with each passing day.
|
| It's still an incredibly convincing argument. Rust has decent
| C interop, but it's not especially seamless to do safely. But
| Rust's interop with C++ is rather complicated. So if you have
| any C++ dependencies, you're almost certainly _far_ better
| off to just go with C++. Even if you just have C dependencies
| and someone else hasn 't yet made a crate for it, you might
| still be better of with C++.
|
| Particularly since many of the Rust crates that bind to
| popular C & C++ libraries are not owned or maintained by the
| library in question. So you're entering into a bit of a
| sketchy maintenance world. And there's already been churn &
| abandonware here, for example the various projects (including
| one from Mozilla/Servo!) to bind to libpng that were then
| later all abandoned, and you're supposed to migrate to
| rewrites of PNG support in Rust like LodePNG. This is not
| ignorable churn & risk.
|
| It can be fun, and if it's a hobby project absolutely go for
| it. But if you're on the clock to ship & maintain something,
| it's a risk you must consider quite seriously.
| zahllos wrote:
| I mostly agree. C++ is mature, has standards and a whole
| array of production ready libraries. If I wanted to build a
| production cross platform GUI app today and not use
| electron (vomit) I would likely use Qt for it. Rust has
| nothing that is quite ready yet to offer there.
|
| However, my overall view is that Rust is the right
| direction. If I had to improve C++, it would look a lot
| like Rust: move bounds checking and ownership to the
| compiler not the template compiler, remove classes,
| introduce traits and make C++ have concepts that don't
| require me to type every punctuation symbol on my keyboard
| to use, const by default.
|
| Getting the compiler to do some of this checking by default
| is exactly what I want. If we can have an Ada/Spark style
| subset via Kani, also done at compile time to check
| correctness, even better.
|
| > It's still an incredibly convincing argument. Rust has
| decent C interop, but it's not especially seamless to do
| safely.
|
| I would make this statement slightly stronger. If you
| interop with C code and there are bugs in C code, all bets
| are off. You don't get any magical protection because your
| main routine is written in Rust. Cross-language safety,
| e.g. C++/Javascript, for example, is a very difficult
| problem. Here's a discussion of C++ and Rust
| https://dl.acm.org/doi/fullHtml/10.1145/3485832.3485903 If
| you want a tl;dr of that paper, it is that memory
| corruptions in C or C++ code sharing the address space of
| safe Rust code can make safe Rust code unsafe too. There
| have been plenty of attempts to study for example the
| interactions between browser C++ and the various runtimes a
| browser hosts.
|
| The compact you are making with the Rust compiler when you
| type 'unsafe' is that you are saying you have verified the
| correctness of that code: that it is in fact already safe
| and the compiler should not (probably because it cannot)
| check it. The same is true of any FFI code you use.
| However, this is not how many people actually _use_ Rust.
| Here for example is a 2 year old unfixed segfault in a Rust
| crate caused by interfacing with non-Rust code:
| https://rustsec.org/advisories/RUSTSEC-2020-0159. This a)
| demonstrates the issue and b) demonstrates that random
| crates pulled off crates.io can contain safety issues for
| 2+ years with no fix.
| hulitu wrote:
| > I mostly agree. C++ is mature, has standards and a
| whole array of production ready libraries. If I wanted to
| build a production cross platform GUI app today and not
| use electron (vomit) I would likely use Qt for it.
|
| C++ is mature ? Every couple of years a new C++ standard
| emerges which breakes old programs. Qt ? Which version ?
| 3 ? 4 ? 5 ? 6 ?
| andrewaylett wrote:
| An "unsafe" set of calls from Rust to C are still no less
| unsafe than calling C from other C or from C++. The biggest
| difference is in programmer expectation, which is why it's
| so unfortunate when a "safe" wrapper has bugs. Rust can't
| fix bugs in the underlying C library either, of course.
| That's one place where Mozilla's recent foray into FFI via
| compiling to WASM then compiling the WASM to native code is
| very interesting to me.
| kllrnohj wrote:
| But making that unsafe Rust block safe to call from safe
| Rust isn't the most trivial of things to do. Conforming
| to all the safety requirements is still a thing to deal
| with, otherwise you contaminate all your safe Rust code.
|
| It's not an unreasonable burden for what you get in
| return, but it is still a burden.
| stouset wrote:
| That burden exists if you're calling into it from C, it's
| just implicit. At least Rust gives you the tools to write
| safe wrappers. Once you've done that consumers of your
| library don't have to tiptoe around the possibility of
| unsafe behavior.
| notriddle wrote:
| > C++ is mature, there's a difference. It's definitely not
| "long in the tooth" as it still has plenty of modern
| features & niceties and is getting more all the time.
|
| C is mature. C++ drinks and sleeps around, but that hardly
| makes it mature.
| boznz wrote:
| > C++ is long in the tooth
|
| Welcome to the world of any language that the cool guys
| dont use anymore :-)
| bayindirh wrote:
| > IMO in 2022 there's not a good reason to start a new
| project in C++ unless that's all you know, or you have some
| constraint tied to C++.
|
| I personally don't agree. I can start with performance and
| continue with a better compiler ecosystem, and add
| suitability for low resource applications. I'm just not
| scratching the surface here.
|
| Rust is not bad, but it's not a replacement of C++. They
| aspire to be different things.
| [deleted]
| umanwizard wrote:
| When is C++ higher-performance than Rust?
| jandrewrogers wrote:
| I would frame the question more in terms of economy than
| performance. In theory, I could write a database engine
| in Rust that is as performant as C++, but it would not
| make sense to.
|
| Making Rust perform the same as C++ in database engines
| requires marking large portions of the code base as
| "unsafe", while requiring substantially more lines of
| code due to being less expressive. Note that this doesn't
| imply that the code is _actually_ unsafe, just that the
| Rust compiler cannot reason about the safety of correct
| and idiomatic code in database engines.
|
| And this is why state-of-the-art database engines are
| still written in C++. There is not much point in using
| Rust if doing so requires disabling the safety features
| and writing a lot more code to achieve the same result.
| pkolaczk wrote:
| Definitely not my experience with file io and database-
| related software in rust. Actually two of my publically
| available programs - fclones and latte - seem to be the
| most efficient in their respective classes, beating a
| wide variety of C and C++ programs, yet they contain
| almost no unsafe code (< 0.1%).
|
| The state of the art database engines are written in C++
| (and Java) because at the time they were developed Rust
| didn't exist.
| umanwizard wrote:
| I work at Materialize, which is writing a database in
| Rust. We use some `unsafe` code, but it's a very small
| proportion of the overall codebase and quite well
| abstracted away from the rest of the non-unsafe code.
|
| To be fair, we're trying to do a bit of a different thing
| (in-memory incrementally maintained views on streaming
| data) than most existing databases, so you could argue
| that it is not an apples-to-apples comparison.
|
| But there are plenty of other databases written in
| non-C++ languages -- there is even TiDB which is written
| in Rust too.
| nivenkos wrote:
| There's no placement new IIRC so you always build on the
| stack and copy it to the heap.
|
| https://users.rust-lang.org/t/how-to-create-large-
| objects-di...
| andrewaylett wrote:
| I think that's a question of an "insufficiently smart
| compiler" rather than _necessarily_ something the
| programmer should be concerned about?
| Koshkin wrote:
| The programmer should certainly be concerned about the
| compiler being insufficiently smart.
| steveklabnik wrote:
| No, it is a language semantics issue. You _can_ get the
| compiler to optimize it away today, if you 're careful.
| But that's a bad user experience. You should be able to
| specifically request this semantic, and be sure it works.
| umanwizard wrote:
| Fair point.
| tialaramex wrote:
| You can do "placement new" (Rust has no new operator, but
| in this context) unsafely with MaybeUninit --
| https://doc.rust-
| lang.org/stable/std/mem/union.MaybeUninit.h...
|
| Make a MaybeUninit<Doodad> on the heap, initialise it,
| and then unsafely assume_init() remembering to write your
| safety justification ("I initialised this, so it's fine")
| and get a Doodad, on the heap.
|
| The reason it isn't mentioned in that 2019 u.r-l.o post
| is that MaybeUninit wasn't stabilized until mid-2019.
| infamouscow wrote:
| Rust's lack of fast thread-local storage makes it a non-
| starter for me.
|
| It's really disappointing, especially when the language
| has incorporated so many other excellent performance
| improvements like Google's SwissTable and pattern-
| defeating quicksort.
|
| https://github.com/rust-lang/rust/issues/29594
|
| https://matklad.github.io/2020/10/03/fast-thread-locals-
| in-r...
| roblabla wrote:
| FWIW, Rust does have fast thread locals, but only in
| nightly. https://github.com/rust-lang/rust/issues/29594
| bayindirh wrote:
| High performance and scientific computing, or more
| generally when you transfer a lot of data structures via
| pointers to a lot of threads. In that scenarios, a half
| second performance difference in main loop can translate
| to hours in long runs.
|
| The code I'm developing is running with ~1.7M
| iterations/core/second. Every one of these iterations
| contain another set of 1000 iterations or so (the number
| is variable, so I don't remember the exact mean, but the
| total is a lot). Also, this number is on an 8 year old
| system.
|
| More benchmarks are here: https://benchmarksgame-
| team.pages.debian.net/benchmarksgame/...
| umanwizard wrote:
| You haven't actually explained when Rust is slower. You
| just described a situation where you care about
| performance.
|
| What specific operations happen slower in Rust than in
| C++?
| [deleted]
| ModernMech wrote:
| The link you provided shows C++ and Rust both performing
| best in various benchmarks. Seems like a wash to me, and
| to the extent one beats the other it's marginal. Which is
| to be expected because as long as the language is
| compiled and the compiler is good enough, you should be
| able to get as much performance out of it as your CPU can
| handle.
| bayindirh wrote:
| The numbers might look small, and indeed for most cases
| it can be considered a wash, but in some cases, these
| small differences return as hours.
|
| For example, I'm using a lot of vector, matrix and array
| accesses in my code. A lot of these are copied around for
| computation. Moving that amount of data will trigger a
| lot of checks in Rust, and will create a slowdown
| inevitably.
| chlorion wrote:
| Moves don't require any runtime checks in Rust to my
| knowledge. Moves do involve some compile time analysis to
| make sure you aren't accessing a value that has been
| moved, maybe that is what you are thinking of?
|
| In Rust, moves are always a simple memcpy from one place
| to another, that's it! With non `Copy` types this
| destroys the original value which is why the compiler
| makes sure you don't access it after it's moved.
|
| There is also `Clone` which is generally a deep copy of
| everything, but this also doesn't involve any runtime
| checks as long as you aren't using reference counted
| pointers or something.
|
| There _are_ bounds checks when indexing into vectors and
| whatnot though, but non checked methods are available.
| ModernMech wrote:
| Do you have a link to your code that you've benchmarked
| in C++ and Rust? I'm also doing a lot of
| vector/matrix/array access in my code, so I'm curious as
| to what you'd be doing that would cause slow downs to the
| point that C++ can beat Rust by hours. That would be very
| enlightening to see.
| bayindirh wrote:
| I would love to, but the framework I'm developing is
| closed source (it's my Ph.D. and postdoc work at the end
| of the day), but we have a paper for some part of it. Let
| me see if I can share it.
| umanwizard wrote:
| What makes you think Rust forces more "checks" when
| moving data than C++ does? Can you give an example?
| stouset wrote:
| To wit, the Rust compiler _is_ "sufficiently smart" in an
| extremely high number of scenarios such that it can
| achieve optimal performance by eliding things like bounds
| checks when it's able to determine that they would never
| fail.
|
| This is, in my experience, the overwhelming majority of
| idiomatic Rust code. For cases where it can't, and
| performance is absolutely critical, there are easy escape
| hatches that let you explicitly elide the checks
| yourself.
| jcelerier wrote:
| Rust's Vec indexing is bound-checked by default. Just
| something like this is abysmal for performance (but
| pretty good for consultants who get called to fix stuff
| afterwards, so, like, keep them coming!):
|
| - Rust: https://rust.godbolt.org/z/GK8WY599o
|
| - CPP: https://gcc.godbolt.org/z/qvxzKfv8q
| steveklabnik wrote:
| Idiomatic Rust here would use something like this, not
| that in this case it makes them performance identical,
| but it is more representative of real Rust code, and is
| slightly smaller https://rust.godbolt.org/z/brh6heEKE
|
| EDIT: lol don't write code five minuets after waking, the
| reply to this has far better code
|
| (EDIT: incidentally, I was also curious about a non-
| extern function, but the compiler is seeing through all
| of my attempts to call a native Rust function with an
| opaque body here...)
|
| That being said this code will also be a no-op instead
| of... something worse, if the vector is empty, so,
| they're not directly comparable, but they weren't really
| in the first place, as you mention, so...
| umanwizard wrote:
| Why not for i in &v[1..num]
|
| instead of all the iter take skip stuff?
| steveklabnik wrote:
| Because I wrote this comment shortly after waking up and
| made a mistake. That code is far far better. Thank you.
| nivenkos wrote:
| You can just use unsafe if you care in this case though.
|
| It's a saner default option tbh.
| amadvance wrote:
| C is even better. One instruction less than C++ ;)
|
| https://gcc.godbolt.org/z/oc19Gc4eY
| 3836293648 wrote:
| https://rust.godbolt.org/z/zM58dcPse
|
| And look, putting a limit that the compiler and guarantee
| once outside the critical loop eliminates it. You're
| already using unsafe. If you really want to shoot
| yourself in the foot with this nonsense you can use a get
| unchecked or an unchecked assumption
|
| https://rust.godbolt.org/z/naeaYad5T
|
| On top of that, noone would write rust code like that.
| This version has one bounds check at the beginning and is
| a lot more idiomatic
|
| https://rust.godbolt.org/z/1s9zsMx36
| jcelerier wrote:
| The point is to have some code that exhibits what happens
| when a bound check is needed. You can also do
| for(double v : std::span(vec).subspan(1)) { }
|
| in C++ and that will be safe and not need a bound check
| except an user-provided one at the creation of the span
| (-1 for C++ here :-)) - and also not matter at all
| regarding the bound checks problem which will occur as
| soon as you don't just iterate linearly which is
| extremely common in the kind of problems that end up
| showing at the top of the profiler and the whole point of
| this conversation.
| froydnj wrote:
| Just because indexing is bounds-checked by default
| doesn't mean that the bounds-checking is necessarily in
| your final code:
|
| https://rust.godbolt.org/z/er9Phcr3c
|
| Yes, you still have the bounds check when constructing
| the slice, but that's fine.
|
| And since it's probably slightly more common to iterate
| over the length of the vector anyway:
|
| https://rust.godbolt.org/z/e5sdTb7vK
| umanwizard wrote:
| Both languages let you access vector elements with or
| without bounds checking. Stroustrup even recommends you
| use .at by default in c++.
| titzer wrote:
| > Rust's Vec indexing is bound-checked by default. Just
| something like this is abysmal for performance
|
| Unless you can provide numbers to back this claim up,
| I'll continue to rely on my measurements that bounds
| checking (in Virgil) costs anywhere from 0.5% to 3% of
| performance. It's sometimes more or less in other
| languages, but it is by far _not_ the reason any
| programming language has bad performance. I have no
| reason to suspect that it costs more in Rust, other than
| misleading microbenchmarks.
| jcelerier wrote:
| Here's a quick and dirty example:
| https://gcc.godbolt.org/z/3WK7eYM4z
|
| I observe (-Ofast -march=native -std=c++20 ; CPU is intel
| 6900k with performance governor on Arch... blasting at
| 4GHz) :
|
| - clang 13: 8.93 ms per iteration
|
| - gcc 11: 9.46 ms per iteration
|
| so roughly around 9ms.
|
| Replacing return mat[r * matrix_size +
| c];
|
| by return mat.at(r * matrix_size + c);
|
| I observe
|
| - clang 13: 20.95 ms per iteration
|
| - gcc 11: 18.11 ms per iteration
|
| so literally more than twice as slow. I also tried with
| libc++ instead of libstdc++ for clang and the results did
| not meaningfully change (around 9ms without bound-
| checking and 21 ms with).
| nybble41 wrote:
| Part of the problem is that the multiply() function takes
| variable-size matrices as inputs and isn't inlined
| because it's declared as an external function and so
| could, in theory, be called from another compilation
| unit. If it's declared as "static" instead then the
| compiler generates identical code (modulo register
| selection) at -Ofast for both versions--eliminating the
| bounds checking at compile-time:
|
| https://gcc.godbolt.org/z/qqWr1xjT8
| jcelerier wrote:
| > Part of the problem is that the multiply() function
| takes variable-size matrices as inputs
|
| so does real-life code ? I don't know about you but I
| pretty much never know the size of my data at compile
| time.
|
| > If it's declared as "static"
|
| again, real-life code calls math operations defined in
| other libraries, sometimes even proprietary.
| titzer wrote:
| In Java or other languages with bounds-checked arrays,
| you would typically just loop from 0 to the end of the
| array. In Java the JIT will analyze the loop, see that it
| is bounded by the length of the array, conclude all
| accesses are in-bounds and eliminate bounds checks.
| Virgil does this simple type of bounds check elimination
| as well in its static compiler, but its analysis is not
| quite as sophisticated as Java JITs.
| nybble41 wrote:
| > I don't know about you but I pretty much never know the
| size of my data at compile time.
|
| Whole-program optimization (LTO) can deal with that.
| Also, Rust inlines across modules much more aggressively
| than C++ does, even without LTO, so optimization will be
| more effective and its bounds-checking won't have as much
| (if any) runtime overhead. Especially if you write your
| Rust code idiomatically, as others have already
| suggested. Your earlier Rust example was practically
| designed to inhibit any attempt at optimization. Simply
| iterating over a slice of the vector, rather than the
| indices, results in a much tighter inner loop as it only
| needs to check the bounds once.
|
| That being said, in this case I think it would be better
| to have fixed-size matrices (using const generic/template
| arguments) so that the bounds are encoded in the types
| and known to the compiler locally without relying on
| whole-program optimization.
| stouset wrote:
| > Especially if you write your Rust code idiomatically,
| as others have already suggested. Your earlier Rust
| example was practically designed to inhibit any attempt
| at optimization.
|
| This is pretty much word-for-word what I've seen in other
| similar disagreements. Rust Skeptic transcribes their
| solution from its original programming language into Rust
| line by line, producing something nobody using Rust would
| actually write. They find that Rust performs poorly. Rust
| Proponent writes it the way you'd expect someone who
| actually knows Rust to write it, and performance meets or
| exceeds the original. Sometimes they even catch a few
| bugs.
|
| Yes, if you don't know Rust, and you think it's just a
| weird re-skin of C++ that can be gsub'd from one to
| another, you're going to have a bad time. If you're an
| expert in C++ and have never written Rust, you probably
| aren't going to beat your finely-tuned C++ program with a
| ten minute Rust translation. But someone with a year of
| experience in Rust is probably going to be able to
| rewrite it in half an hour and come within spitting
| distance of the thing you've optimized by hand over the
| course of a few years.
|
| I've written Rust for half a decade and I'm not sure I've
| ever actually explicitly indexed into a Vec or slice in
| production code. If I needed to, and it was in a hot
| loop, and it wasn't a fixed-size array whose size is
| known at compile time... there's always `get_unchecked()`
| which is functionally identical to indexing into a C++
| vec without `at`.
| titzer wrote:
| From the godbolt link, it looked like most of the vector
| operations were not getting inlined. You'll need -O2 or
| higher to have representative results.
|
| I could counter by just writing a Virgil program and
| turning off bounds check with a compiler flag. We could
| stare at the machine code together; it's literally an
| additional load, compare, and branch. The Virgil compiler
| loves eliminating bounds checks when it can. I know your
| original comment was in the context of the STL, but it
| really muddies the waters to see huge piles of code get
| inlined or not depending on compiler flags. Machine code
| is what matters.
|
| Regardless, this is still microbenchmarking. Maybe matrix
| multiply is an important kernel, but we need to benchmark
| whole programs. If I turn off bounds checks in a big
| program like the Virgil compiler, I cannot measure more
| than about 1.5% performance difference.
| jcelerier wrote:
| > From the godbolt link,
|
| I just used godbolt to quickly share the code. On my
| computer I tried with -Ofast -march=native (broadwell)
|
| > I could counter by just writing a Virgil program and
| turning off bounds check with a compiler flag. We could
| stare at the machine code together; it's literally an
| additional load, compare, and branch.
|
| This sounds like Virgil is not vectorizing, which makes
| the comparison much less useful.
|
| I see much more than a couple instructions changed there:
| https://gcc.godbolt.org/z/8jPMb734x
| titzer wrote:
| > Virgil is not vectorizing
|
| That's just even more confounding variables. We are,
| after all, not even doing experiments with Rust vectors,
| which is what your original comment was about. You wrote
| examples in C++ and we went down a wormhole already, but
| I let it slip since at least it was empirical. But I
| think we shouldn't get distracted with even more side
| alleys now. Bounds checks are literally just a couple
| machine instructions, and often eliminated by the
| compiler anyway, which enables vectorization.
| steveklabnik wrote:
| I think you're confusing two different senses of
| "vector", there is the contiguous series of items
| "vector" data structure that both C++ and Rust have in
| their standard libraries that's used in the code example,
| and there's "vectorizing" which is an optimization to use
| things like SIMD to operate on multiple things at the
| same time.
| titzer wrote:
| I understood what was meant by these terms.
| steveklabnik wrote:
| Okay, then I got lost reading your comment, my apologies.
| titzer wrote:
| No worries. It's too late for me to edit it now, though,
| but I meant vectorizing="compiler introduces SIMD".
| jcelerier wrote:
| > We are, after all, not even doing experiments with Rust
| vectors,
|
| they are very close to C++ ones, down to the stack
| unwinding to report the panic in case of bound error if
| I'm not mistaken.
|
| > That's just even more confounding variables.
|
| No they are not. We are trying to see how much bound
| checks costs. If you compare between suboptimal programs
| the comparison is meaningless (or rather not interesting
| to anyone) - the starting point has to be the absolute
| best performance that it is possible to get, and add the
| worst-case bound-checking (I'm happy for you if you never
| have to worry about the worst case though!)
|
| > Bounds checks are literally just a couple machine
| instructions, and often eliminated by the compiler anyway
|
| please provide a source for this ? sure, if you use spans
| as other commenters mentioned, that moves the checking at
| the span creation time but that only works for the
| simplest cases where you are going to access linearily -
| and I would say that it's a library feature rather than a
| compiler one. for(double v : vec) { } //
| or any sub-span you can take from it
|
| in C++ also does not need bound-checks by design but this
| kind of construct also utterly does not matter for so HPC
| workloads.
|
| I can look into my pdfs folders and bring out dozens of
| papers where the core algorithms all use funky non-linear
| indexing schemes where you cannot just iterate a range
| (algorithms based on accessing i, i-1, i+1, or accessing
| i and N-i, or accessing even / odd values, etc etc) - how
| would you implement an FFT for instance ? This is the
| code that matters !
| titzer wrote:
| > accessing i, i-1, i+1, or accessing i and N-i,
|
| A compiler can do loop versioning for that. And they do.
| Hotspot C2 (and Graal, too) does a _ton_ of loop
| optimizations, partitioning the index space into in-
| bounds and potentially out-of-bounds ranges, unrolling
| loops, peeling the first iteration off a loop, generating
| code before a loop to dispatch to a customized version if
| everything will be in bounds.
|
| When a compiler is doing that sophisticated of loop
| transforms, you are _not_ measuring the cost of bounds
| checks anymore, you are measuring a whole host of other
| things. And sometimes if a loop is just a little
| different, the results can be disastrously bad or
| miraculously good. Which is why microbenchmarking is so
| fraught with peril. A microbenchmark might be written
| assuming a simple model of the computation and then a
| sophisticated compiler with analyses never dreamed of
| comes along and completely reorganizes the code. And it
| cuts both ways; a C++ compiler might do some unrolling,
| fusion, tiling, vectorization, software pipelining or
| interchange on your loops and suddenly you aren 't
| measuring bounds check cost anymore, but loop
| optimization heuristics that have been perturbed by their
| presence. You end up studying second-order effects and
| not realizing it. And it keeps running away from you the
| more you try to study it.
|
| >> That's just even more confounding variables.
|
| > No they are not. We are trying to see how much bound
| checks costs. If you compare between suboptimal programs
| the comparison is meaningless (or rather not interesting
| to anyone) - the starting point has to be the absolute
| best performance that it is possible to get, and add the
| worst-case bound-checking (I'm happy for you if you never
| have to worry about the worst
|
| We are always comparing suboptimal programs. No compiler
| is producing optimal code, otherwise they would dead-code
| eliminate everything not explicitly intertwined into
| program output, statically evaluate half of our
| microbenchmarks, and replace them with table lookups.
|
| You're going down a linear algebra rabbit hole trying to
| come up with a result that paints bounds checks in the
| worst possible light. If this is the real problem you
| have, maybe your linear algebra kernels would be better
| off just using pointers, or you could even try Fortran or
| handwritten assembly instead, if it is so important.
| Unsafe by default is bad IMHO. For real programs bounds
| checks really don't matter that much. Where this thread
| is going is all about loop optimizers, and they don't
| really get a chance to go nuts on most code, so I think
| we're way off in the weeds.
| tialaramex wrote:
| Note that Rust _has_ unchecked index, you just have to
| explicitly ask for that if you want it. You can even - if
| you just insist on writing cursed code - which it seems
| jcelerier is, write your own type in which the index is
| always unchecked and it is Undefined Behaviour when you
| inevitably make a mistake. Just implement std::ops::Index
| and if you also want to _mutate_ these probably invalid
| indices, std::ops::IndexMut and in your implementation
| say you 're unsafe and just don't bother doing bounds
| checks.
|
| You can shoot yourself in the foot with Rust, it's just
| that you need to explicitly point a loaded gun at your
| foot and pull the trigger, whereas C++ feels like any
| excuse to shoot you in the foot is just too tempting to
| ignore even if you were trying pretty hard not to have
| that happen.
| colejohnson66 wrote:
| If you were using std::vector::at (the proper way), you'd
| have bounds checking as well. One should only direct
| index if they _really_ know that they are inside the
| bounds. And in Rust, there 's
| std::vec::Vec::get_unchecked: https://doc.rust-
| lang.org/std/vec/struct.Vec.html#method.get...
| jcelerier wrote:
| > If you were using std::vector::at (the proper way),
|
| it is absolutely not the proper way and a complete anti-
| pattern
| umanwizard wrote:
| That depends on the project. Anyway, it's irrelevant,
| because you can do it either way in either language.
| gavinray wrote:
| This is the Ossia Score lead dev FYI, he's no stranger to
| either language
| umanwizard wrote:
| I don't know what Ossia Score is. Regardless, I didn't
| mean to imply anything negative about the competence of
| the person I was replying to.
| zozbot234 wrote:
| C++ is really struggling with whether or not to break their
| ABI. But their "identity" is clear enough - much like C, they
| want to support their existing code, while still opening up
| support for newer features if possible. Rust could actually
| benefit from cribbing a few things that are now getting
| defined in C++, such as their support for heterogenous
| executors.
| zabzonk wrote:
| C++ doesn't have a standardised ABI. I'm not aware of any
| languages that do, but I would be interested in any more
| information on this.
| seoaeu wrote:
| There's the widely used Itanium C++ ABI [0], which
| contrary to the name is actually the one used on x86-64.
| Without a standard ABI it wouldn't be possible to
| dynamically link C++ libraries (which includes the
| standard library!), especially if they weren't compiled
| with identical compiler versions.
|
| [0]: https://itanium-cxx-abi.github.io/cxx-abi/abi.html
| kllrnohj wrote:
| C doesn't have a standardized ABI, either, and yet it's
| the most widely used ABI in the world anyway.
|
| Just because it isn't standardized doesn't mean it isn't
| stable & relied upon anyway. Clang & G++ are ABI
| compatible with each other, for example, and more
| importantly (and relevantly) both libstd++ & libcxx
| strive to maintain ABI compatibility (
| https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html
| & https://libcxx.llvm.org/DesignDocs/ABIVersioning.html )
| tambre wrote:
| > both libstd++ & libcxx strive to maintain ABI
| compatibility
|
| They don't, but they do keep it stable.
| mst wrote:
| Alternative phrasing: they strive to maintain "backwards
| compatibility" rather than "compatibility with each
| other".
| kllrnohj wrote:
| For libcxx & libstdc++ yes correct.
|
| Clang, however, does strive to maintain compatibility
| with G++. So you can build a library with clang & use it
| from g++. It strives, and for the most part achieves, to
| be a drop-in replacement for GCC, which includes ABI
| compatibility.
| legalcorrection wrote:
| COM and Windows Runtime (WinRT) provide standardized ABIs
| across multiple languages. With the WinRT language
| projections, it's just like writing 'normal' code.
| pjmlp wrote:
| And with C++/WinRT you even get back the experience of
| how it was like to do ATL development.
| legalcorrection wrote:
| So I gather that, much more than previous COM projections
| and templating libraries, C++/WinRT is much nicer to work
| with. And you can target the WinRT ABI with Rust, .NET
| (C#, F#, etc.), Python, and JS too.
| pjmlp wrote:
| I completely disagree, C++/CX was the best that Microsoft
| ever had closer to C++ Builder and they killed with their
| office politics.
|
| On top of that, the C++/WinRT folks are adamant to push
| their east const preference into every Microsoft
| customer.
|
| If you enjoy editing IDL files without tooling support
| and merging generated files by hand, please do.
| asveikau wrote:
| I really didn't like C++/CX from the perspective of the
| fact that it added weird stuff to the existing syntax of
| the language, and it had no other compiler onboard other
| than Microsoft. These two probably killed it more than
| "office politics".
| pjmlp wrote:
| It is a framework for doing Windows development on Visual
| Studio, follows the same syntax as C++/CLI, in use since
| 2003, it doesn't matter if anyone else is on board.
|
| Even today no other compiler can fit into WinUI/UWP
| workflows no matter what, unless one is willing to do
| lots of under the hood workarounds.
|
| Office politics driven by people without any
| consideration for paying customers.
| Decabytes wrote:
| > In terms of popularity, depending on which survey you look
| at, Rust is stalking or eclipsing C++
|
| The full quote is in reference to popularity. Which if you look
| at the stack overflow developer survey is definitely true
| creatonez wrote:
| Rust is absolutely _dominating_ the very tiny domain of
| greenfield OS development.
| Gigachad wrote:
| It's also made keys steps in to Linux. I haven't checked the
| latest state, but as of last year it was pretty much approved
| as the way to write new drivers on Linux.
| tialaramex wrote:
| Rust for Linux is not landed, and so you can't write Linux
| drivers in Rust even on Linux Next (the git branch where
| in-progress work lives before Linus might ship it in an
| actual Linux kernel)
|
| It's probably of the order of a year out.
| ncmncm wrote:
| Rust is anyway making a better showing there than other
| places.
| jwatt wrote:
| I'm not sure it's "desperately" chasing; its trajectory looks
| positive. For example, in the 2019 Stack Overflow developer
| survey[1] only 3% of professional developers reported using
| Rust. A year later[2] it was 4.8% and a year after that[3] it
| was 6.4%. So based on that metric (of debatable value),
| professional developer use may have doubled in two years. For
| reference, in the SO surveys C++ and C use was about 20% and
| 16%, respectively and, if anything, slowly declining (as a
| percentage of respondents, of course).
|
| 1.
| https://insights.stackoverflow.com/survey/2019#technology-_-...
|
| 2. https://insights.stackoverflow.com/survey/2020#technology-
| pr...
|
| 3. https://insights.stackoverflow.com/survey/2021#technology-
| mo...
| kevin_thibedeau wrote:
| PHP had a positive trajectory too. Rust is riding a wave of
| interest but most are going to revolt against its complexity.
| [deleted]
| wk_end wrote:
| That sounds plausible in the general case...but we're
| putting it up against C++ here, which is at least as
| (probably more) complex.
| stouset wrote:
| Rust is literally the most beloved language in SO's
| developer survey and has been for several years running.
|
| In my own experience, it's allowed me to write software
| that pretty much _doesn 't have bugs_, and I say this as a
| 25 year veteran of being an engineer. The extra bits of
| complexity that are in the language allow me, the
| developer, to not have to write code to deal with that
| complexity in my own projects. It's handled for me.
|
| Rust's primary competition is C and C++. It compares
| favorable on the overwhelming majority of axes against
| either. Not every one, but most.
| stevefan1999 wrote:
| BeOS/Haiku and Redox comes to mind
| paride5745 wrote:
| BeOS and Haiku are mentioned in the article.
| mike_hearn wrote:
| The article didn't list JNode, but it's also a pure Java OS.
|
| I noticed in another thread that a few people seem to think you
| can't implement an entire operating system in a GCd language like
| Java or C#, but that isn't true. You can do it like this:
|
| 1. Use an ahead of time compiler to compile the base
| runtime/kernel image to static machine code. In managed language
| operating systems like JNode or Singularity there isn't a strong
| distinction between kernel and language runtime, so this is the
| same thing. This base needs to contain at least a GC and probably
| a JITC, as well as some basic support services. This can itself
| be written in the managed language.
|
| 2. Write a very short bootstrap loader in assembly like every OS
| has, which sets things up enough to jump into the entry point of
| that runtime.
|
| 3. Writing a compiler in a managed language is easy enough but
| what about a GC? To do this you teach the compiler that some
| methods are 'magic' and shouldn't be treated as normal method
| calls. Instead they become either compiler intrinsics that are
| translated to pure assembly e.g. to read/write raw memory
| locations, or they are compiled in special ways for example to
| remove GC safe points.
|
| The current best example of this in action is the GC in
| SubstrateVM, which is the "VM" compiled into any program AOT
| compiled with the GraalVM native image tool:
|
| https://github.com/oracle/graal/tree/master/substratevm/src/...
|
| If you flick through it you'll see various annotations and types
| you wouldn't normally see in Java, like `Pointer` and
| `@Uninterruptible`. These are recognized by the compiler and
| affects how the machine code is generated. The language is the
| same, so all existing tools continue to work - it's not a dialect
| or subset of Java, it's the full thing, just with slightly
| modified rules for how the final generated code behaves.
|
| SubstrateVM has one more trick up its sleeve to break the
| circularity: some objects can be initialized and persisted to an
| "image heap" at build time. In other words, the GC code can use
| Java classes to structure itself, despite being unable to
| allocate.
|
| And that's all it needs.
|
| There have been efforts to do things like this in the past for
| full operating systems. They have nice properties: for example
| you can sandbox drivers, IPC overheads goes away because it's all
| in a single address space, capabilities actually work and are
| pervasive, and it's quite easy to keep ABIs stable. There are
| usually a few sticking points that prevent them taking off:
|
| 1. Historically, GCs have either been good at latency _or_
| throughput but not both simultaneously. Some of them also had
| trouble with large heaps. That 's a problem because virtually all
| reasonable computing requires a mix of programs with wildly
| different sensitivity, most obviously, developers want latency to
| be prioritized for editing in their IDE but throughput to be
| prioritized for their build system. If you have one GCd heap for
| the entire computer then you have two options:
|
| 1a. Pick one algorithm to manage it.
|
| 1b. Do what Singularity did and come up with a quasi-process
| notion in which each unit has its own GCd heap. Singularity had a
| nice concept called an 'exchange heap' which allowed objects to
| be passed between these quasi-processes very fast and cheaply,
| whilst ensuring that object graph could only be pointed to by one
| unit at once. This made IPC tremendously cheap, allowed one unit
| to be paused for GC whilst other units ran, and let them use IPC
| all over the place. However it did reduce the benefits of using
| managed languages somewhat as it reintroduced complex data
| ownership rules.
|
| NB: This is changing now with tech like HotSpot ZGC and
| Shenandoah (which are written in C++ though). They drive
| latencies through the floor, it's basically pauseless, and the
| new not yet released fully generational variants have very good
| throughput too. Also, G1 GC has monstrous throughput even with
| low pause times, they just aren't quite as low as ZGC/Shenandoah.
|
| 2. Overheads of managed languages are higher. The successor at MS
| Research to Singularity was codenamed Midori and not much was
| ever published about it publicly, but from what was written (by
| Joe Duffy) it seemed apparent that they went down a rabbithole of
| trying to make Midori have the same raw efficiency and overhead
| as C++ based Windows. They got a long way but ended up not having
| any interesting enough new features to justify the investment and
| the project was eventually canned.
|
| 3. All the same problems new operating systems always have: no
| apps, drivers etc.
|
| 4. Spectre attacks make single address space operating systems
| more complex. The new support for memory protection keys in Intel
| CPUs could re-awaken this area of OS research however because
| MPKs let you block speculation attacks within a single address
| space.
| maxloh wrote:
| Which was the another thread you read?
| Yhippa wrote:
| Mike, this is very well-written. Thank you. I grok maybe 40% of
| what you are saying which is a testament to what you typed out.
|
| Is there a situation where if your sticking points could. E
| addressed or tolerated where using a GC'd language could shine
| vs a non one?
| mike_hearn wrote:
| Sorry it's only 40% :) If you like I can elaborate.
|
| I think there are many cases where GC'd languages can benefit
| tasks traditionally thought of as 'systems' tasks (I don't
| personally recognize much distinction between systems
| programming and non-systems programming). The Graal project
| is probably the most successful in this area. Like Midori it
| came out of long term corporate R&D, but unlike Midori it
| successfully shipped impactful software. They also published
| papers which the Midori guys never did.
|
| Graal is an implementation of a JIT and AOT compiler for many
| languages, both source code and byte code based. It can run
| not only JVM bytecode but e.g. Python, Ruby, LLVM bitcode,
| WASM, and a whole bunch more, which makes a JVM powered by it
| easily the most polyglot compiler and runtime systems in
| history. Languages aren't translated to JVM bytecode, which
| would introduce impedance mismatches, but rather input to the
| compiler via a rather interesting technique called partial
| evaluation of interpreters.
|
| Graal came out of asking the question, what if we wrote a JVM
| in Java? How can we do that, and if we can, are there
| benefits? Well, just rewriting something in Java doesn't give
| any benefits to the end users, just the developers, so to
| justify this you really have to find ways to really boost
| productivity a lot and then use that to yield interesting
| end-user features as well. Midori seems to have failed at
| this, Graal succeeded.
|
| IMO the primary benefits Graal gets out of being written in a
| high level statically typed managed language are:
|
| 1. All the usual benefits like GC, great libraries, rock
| solid refactoring IDEs.
|
| 2. Annotations and the associated annotation
| processing/reflection infrastructure. The part that makes
| Graal polyglot is called Truffle and Truffle relies on this
| feature extensively.
|
| 3. Medium-level program input representation. One thing the
| Java world got really right is the clean separation between
| frontend and backend compilers via a stable bytecode format.
| Bytecode is not too high level, so you don't have to think
| about the complexities of evolving syntax when reading it,
| but also not too low level. Graal is of course itself
| expressed using bytecode and the compiler exploits this all
| over the place, for instance, it can parse bits of itself
| into compiler graphs at runtime and then use them as
| templates. They call this snippets and they use it as a way
| to lower high level constructs into lower level ones. It's
| really neat and a boost to compiler development productivity.
|
| 4. Related to that, Truffle relies heavily on that sort of
| meta-circular reflection capability. Truffle is the way Graal
| can compile languages that aren't JVM bytecode based. You
| write an AST interpreter for the language in any language
| that can produce bytecode (but in reality it's always Java
| because you need fairly precise control over the output;
| Kotlin might also work). The interpreter uses the Truffle API
| to express the AST, in particular it uses lots of
| annotations. The resulting interpreter is a normal Java
| program that can run on anything that can interpret bytecode,
| but the Graal compiler has plugins that recognize when it's
| compiling a Truffle program and handle it in special ways.
|
| This ability to construct large, complex API surfaces that
| trigger special compiler behavior is one of the huge
| productivity wins that allowed the Graal team to add so many
| languages with such high runtime performance, so fast and so
| cheaply. It's like compiler intrinsics but taken to the next
| level, and then the next level again. And the end result is
| real benefits delivered to end users, for instance, Shopify
| uses TruffleRuby to get better performance for components of
| their web app.
| yvdriess wrote:
| Truffle/Graal sounded almost like magic when we first
| looked at it. Its partial evaluation was one of the best
| efforts we've seen for speeding up R, which is notoriously
| averse to being compiled. (https://www.graalvm.org/r/)
| titzer wrote:
| > 4. Spectre attacks make single address space operating
| systems more complex.
|
| I will say, unequivocally, that Spectre actually makes process
| isolation in single address space operating systems impossible
| on modern hardware. There is too much speculation and too many
| leaks, and it's not just branches. We wrote a whole paper about
| it a few years back.
|
| https://arxiv.org/abs/1902.05178
| mike_hearn wrote:
| I believe that paper predates the introduction of
| speculation-blocking MPKs. Could you build a single-address
| space OS out of those, without hitting problems with Spectre
| attacks? It's an open research question but my gut says yes.
| MPKs are limited so you may need an equivalent of swapping
| with fallback to page table based isolation, but it's worth
| noting that in a SASOS the notion of a process is unpacked,
| so you can then add on top newly defined hardware enforced
| privacy domains that don't cleanly map to any existing notion
| of a process.
|
| For example all code from a particular vendor (origin) could
| share a single MPK whilst running, even if the failure domain
| for things like fault isolation is finer grained.
| titzer wrote:
| > I believe that paper predates the introduction of
| speculation-blocking MPKs
|
| That isn't enough, because you can induce misspeculation
| through paths that _do_ (or would) have access to
| appropriate MPKs and do almost anything you want, including
| disclosing information through sidechannels you do have
| access to. Loading an MPK is akin to changing address space
| protections; it has to be a hard barrier that cannot be
| speculated through. You cannot even have code _mapped_ that
| would have access to those MPKs, as you can induce
| misspeculation into this code.
|
| > It's an open research question but my gut says yes.
|
| My gut says no. There is just too much dark stuff going on
| in hardware. Sidechannels are outside of models. You can't
| verify anything until you have not the model, but the whole
| design of the chip.
|
| Also, variant 4 is not addressed much in the literature. I
| couldn't write what I wanted to write because of NDAs, but
| I have personally written PoCs that basically say hardware
| has to turn off memory speculation or you end up with a
| universal read gadget again. There is no software solution
| for variant 4.
| mike_hearn wrote:
| I'm told, but haven't verified, that in older Intel CPUs
| loading an MPK wasn't a speculation barrier, but in newer
| CPUs it is. In other words changing your current MPK is
| like changing address spaces but much faster because
| there's no TLB flush or actual context switch.
|
| I think there are also other caveats to consider. A lot
| of Spectre research (like your paper) is implicitly
| focused on the web renderer/V8 use case but here we're
| discussing theoretical operating system designs. Let's
| say you sandbox an image decoder library in-process,
| using type security and without using MPKs. Is this
| useless? No, because even if the image decoder is:
|
| a. Maliciously doing speculation attacks.
|
| b. Somehow this doesn't get noticed during development.
|
| ... the sandbox means it won't have any ability to make
| architectural-level changes. It can spy on you but its
| mouth is sealed; it doesn't have the ability to do any IO
| due to the architectural sandbox. To abuse Spectre in
| this context would require something really crazy, like
| trying to speculatively walk the heap, find the data
| you're looking for, encrypt it, steganographically encode
| the result into the images it decodes, and then hope that
| somehow those images make it back to the attackers even
| though the destination is probably just the screen. This
| isn't even NSA-level stuff, it's more like Hollywood at
| that point.
|
| Compare to the situation today: the image decoder (or
| whatever) is a buggy C library running in a process with
| network access because it's a web origin. Game over.
|
| I worry that the consequence of Spectre research has been
| that people conclude "in-process sandboxing is useless,
| cross-process is too hard, oh well, too bad so sad".
| Whereas in reality in-process sandboxing even without
| MPKs or equivalent would still be a massive win for
| computer security in many contexts where Spectre is
| hardly your biggest problem.
| titzer wrote:
| > I worry that the consequence of Spectre research has
| been that people conclude "in-process sandboxing is
| useless, cross-process is too hard, oh well, too bad so
| sad". Whereas in reality in-process sandboxing even
| without MPKs or equivalent would still be a massive win
| for computer security in many contexts where Spectre is
| hardly your biggest problem.
|
| Well, I agree that in-process sandboxing is still quite
| useful; it at least closes the barn door. But the rest of
| the conclusion is not what we made in Chrome; we had to
| go whole-hog multi-process for site isolation. That and
| just moving as much as possible out of the renderer
| process so that there aren't many secrets left to steal.
|
| It's really an issue for situations where a process (or
| platform) is _required_ to run untrusted code from lots
| of different sources. There isn 't a software solution
| that is robust to side channels yet. They can still spy
| on each other. Clearly, two import cases that Google
| cares about are Cloud and the web.
| mprovost wrote:
| > I noticed in another thread that a few people seem to think
| you can't implement an entire operating system in a GCd
| language like Java or C#, but that isn't true.
|
| Both Smalltalk and LISP were used to write operating systems
| decades ago.
| sigzero wrote:
| jOS was another Java OS.
|
| http://jos.sourceforge.net/
| pjmlp wrote:
| Regarding Midori, besides the blog posts, Joe Duffy did two
| talks about the subject,
|
| "RustConf 2017 - Closing Keynote: Safe Systems Software and the
| Future of Computing by Joe Duffy"
|
| https://www.youtube.com/watch?v=CuD7SCqHB7k
|
| "Safe Systems Programming in C# and .NET"
|
| https://www.infoq.com/presentations/csharp-systems-programmi...
|
| In one of them, he mentions that even with Midori proving its
| value to the Windows team, they were quite dimissive of it.
|
| It appears to have also been yet another victim of the usual
| DevDiv vs WinDev politics.
| zozbot234 wrote:
| > you can sandbox drivers, IPC overheads goes away because it's
| all in a single address space, capabilities actually work and
| are pervasive, and it's quite easy to keep ABIs stable.
|
| You can do these things with WASM, which has no GC and simply
| manages memory segments on a per-module basis.
| mike_hearn wrote:
| The point of a single address space OS is that you can pass
| pointers/references between modules without requiring any
| form of marshalling, nor introducing memory safety problems.
|
| WASM cannot do this because it's meant to be a target for C
| and C-like languages. Either you have to make one WASM
| address space contain all your software, in which case it can
| easily corrupt itself internally (the sandbox is useless
| because it contains everything), or, you have to re-introduce
| a process-like concept that gives you isolated memory spaces,
| at which point it's not a single address space OS anymore.
| zozbot234 wrote:
| Address space is entirely orthogonal to memory protection.
| You can have multiple protected tasks in a single address
| space, or multiple address spaces sharing blocks of
| physical memory among themselves with different virtual
| addresses, or any combination.
| mike_hearn wrote:
| Yes, you could configure your memory maps so they never
| overlap and then call it a single address space, but if
| passing pointers between realms doesn't work then why
| bother? You didn't get any real benefit. The point of
| using a unified GC is that you can actually do this: just
| call a method in another protection domain, pass a
| pointer to a huge object graph, and you're done. There's
| no need for concepts like SHM segments or IPC
| marshalling. Even if you segmented your address space and
| then used classical process-like context switching, you'd
| still need all those things.
| zozbot234 wrote:
| > but if passing pointers between realms doesn't work
| then why bother?
|
| Because then it can work? It's a matter of what virtual
| addresses each "realm" has access to, either reading,
| writing or both.
| mike_hearn wrote:
| I don't think I quite follow what you have in mind.
|
| If there are two realms or protection domains or whatever
| we want to call them, but there is memory protection in
| place to prevent reading/writing of others when one is
| active, you can pass a pointer from one to the other and
| the other knows it's not belonging to itself. But the
| moment that receiver tries to read it, it'll segfault. Or
| what are you imagining happens here?
|
| It seems to me like to solve that you have to copy data,
| not pointers. Now you have marshalling.
|
| There's a second problem with trying to solve this with
| WASM - C and the associated ABIs don't handle evolution
| all that well. But part of what you need in a single
| address space OS is the ability for components to evolve
| independently, or semi-independently. In particular you
| need the ability to add fields to structures/objects
| without needing to recompile the world. Ideally, you'd
| even be able to modify structures in memory without even
| restarting the software. Higher level VMs than WASM can
| do this because they provide higher level semantics for
| memory layouts and linkage. You can evolve a JAR in much
| more flexible ways than you can evolve a binary C/rust
| module, or at least it's a lot less painful, which is why
| Win32 is full of reserved fields, and most attempts at
| long-term stable C APIs are full of OO-style APIs like
| GObject or COM in which structs are always opaque and
| everything has to be modified via slow inter-DLL calls to
| setters.
| legalcorrection wrote:
| I think the piece you're missing is the continuing role
| of the page tables or similar functionality in such
| systems. You can have a single address space, i.e. a
| particular address can only ever refer to the same
| memory, while still determining that only certain
| processes are allowed to access that address. In such a
| system, the page tables would always have the same
| mapping to physical addresses no matter what process
| you're in, but the read/write/execute bits on the page
| table would still change as you context switch.
| mike_hearn wrote:
| That's exactly what I understood from the proposal too,
| but I don't see why that is useful, nor why it'd be worth
| implementing with WASM.
|
| Perhaps it's worth stepping back. The reason SASOSs are
| always written in managed languages like C# or Java
| [dialects] is that they're trying to solve several
| problems simultaneously:
|
| 1. IPC is slow due to all the copying and context
| switching.
|
| 2. Beyond slow it's also awkward to share data structures
| across processes. You need SHM segments, special
| negotiations, custom allocators that let you control
| precisely where data goes and which are thread safe
| across processes etc. Even if you do it, you need a lot
| of protocols to get memory management right like
| IUnknown. So in practice it's rarely done outside of
| simple and special cases like shared pixel buffers.
|
| 3. Hardware processes conflate several different things
| together that we'd like to unpack, such as fault
| isolation, privacy, permissions etc.
|
| 4. Hard to evolve data structures when code is compiled
| using C ABIs.
|
| and so on.
|
| Simply creating non-overlapping address spaces doesn't
| help with any of these things. Even if all you do on a
| context switch is twiddle permission bits, it doesn't
| matter: you still need to do a TLB flush and that's the
| bulk of the cost of the context switch, and at any rate,
| you can't just call a function in another address space.
| Even if you know its address, and can allocate some
| memory to hold the arguments and pointers, you can't jump
| there because the target is mapped with zero permissions.
| And even if you go via a trampoline, so what, the stack
| holding your arguments also isn't readable. If you fix
| that with more hacks, now any structures the arguments
| point to aren't readable and so on recursively.
|
| So you end up having to introduce RPC to copy the data
| structures across. Well, now what if you want a genuinely
| shared bit of state? You need some notion of handles,
| proxies, stubs, and that in turn means you need a way to
| coordinate lifetime management so different quasi-
| processes don't try to access memory another process
| freed. That's COM IUnknown::AddRef. Then you need ways to
| handle loosely coupled components that can be upgraded
| independently. That's COM IUnknown::QueryInterface and
| friends. And so on and so forth.
|
| In a SASOS all that goes away because the compiler and GC
| are tightly integrated, and they don't let you
| manufacture arbitrary pointers. You don't have to do
| refcounting or marshalling as a consequence, you can
| evolve the ABIs of components without breaking things,
| you can create capabilities easily and cheaply, and so
| on.
|
| As discussed above, speculation is a pain because it lets
| you break the rule of not crafting arbitrary pointers,
| but there are caveats to that caveat. I'm not actually
| convinced Spectre kills SASOS though you do need to do
| things differently.
| legalcorrection wrote:
| Why can't it be that whatever procedure you use to
| give/get a pointer into another process also makes the
| necessary modifications to the page table? As you point
| out, this would become very tedious to the programmer if
| you just tried to bolt it on to current languages as a
| library, but I can imagine, e.g., a version of Java or C#
| that makes this all mostly seamless.
|
| As for what the benefit is, I think you can at the very
| least get rid of needing to copy data back and forth.
|
| Not that I'm an advocate for single address space OS's.
| I'd have to think about this more. You might be right.
| I'm playing devil's advocate to think it through, not to
| defend a position, if that makes sense.
| mike_hearn wrote:
| Sure.
|
| Well, pages are coarse grained so you'd end up giving the
| other quasi-process access to more stuff than it should
| have. And you'd have to flush the TLB so you pay the
| context switch cost anyway, at which point why bother?
| The reason operating systems make overlapping mappings is
| (classically) to enable better memory sharing due to not
| needing relocations or GOT/PLTs. That's all irrelevant
| these days due to ASLR (which doesn't even work that well
| anymore) but that was the idea.
|
| You can do some tricks with using special allocators for
| the stuff you want to reveal that places the data in SHM
| segments, then blit the stack across to the other process
| and it can work. I know this because I built such a
| system as my undergrad thesis project :)
|
| https://web.archive.org/web/20160331135050/plan99.net/~mi
| ke/...
|
| It's very ugly though. And again, you need to coordinate
| memory management. For FastRPC it didn't matter because
| it was used mostly for holding stuff temporarily whilst
| you called into a library, so the 'outer' process owned
| the memory and the 'inner' process just used it
| temporarily. I never did anything with complex lifetimes.
|
| One way of thinking about it is to study the issues that
| cause people to write large JVM apps instead of large C
| apps. It's not any different at the systems level. They
| want GC that works across all the libraries they use,
| they want to be able to upgrade the backend compiler
| without frontend-recompiling everything, they want to be
| able to upgrade a library or change a memory layout
| without recompiling the world, and they don't want all
| the goop that is triggered by allowing libraries to use
| arbitrary compilers and memory management subsystems.
| Forcing a unified compiler and GC simplifies the inter-
| module protocols so drastically, the wins are enormous.
| legalcorrection wrote:
| > _Well, pages are coarse grained so you 'd end up giving
| the other quasi-process access to more stuff than it
| should have._
|
| Good point. Embarrassing oversight on my part. My whole
| mental model of how this would work has come crashing
| down. Now obvious to me that to have it work the way I
| envisioned, you would need a managed-code only
| environment that supervises all memory accesses.
|
| > _I know this because I built such a system as my
| undergrad thesis project_
|
| Very cool!
| rev_d wrote:
| There's a USENIX article from a year or two ago about the
| "Biscuit" research OS, which was written in golang.
|
| There were clear compromises & performance degradations related
| to using a GC'ed language, but it definitely was interesting
| enough that you hope for more.
| [deleted]
| [deleted]
| jleyank wrote:
| If they're discussing historical OS's, which I think they were, I
| thought VMS was written in BLISS. That one was sorta important,
| and I recall "Ignorance is BLISS, BLISS is ignorance" T-shirts.
| lproven wrote:
| Hi. Author here.
|
| It's not all historical stuff by any means; in fact, I tried to
| skew it towards stuff from the recent or modern era and things
| that are still in maintenance, still being worked on, or
| actively sold.
|
| And, yes, I did consider VMS, but the article was getting too
| long already and while BLISS does qualify, it's also relatively
| obscure. Maybe I made the wrong call there: it's in current
| use, on sale, and about to become generally available on
| x86-64. Ah, well.
| jleyank wrote:
| Google sea that openvms is still used. Talk about inertia...
| lproven wrote:
| "Google sea"?
|
| VMS is alive, well, and about to ship a new version.
|
| https://vmssoftware.com/about/openvmsv9-1/
|
| It is exceptionally solid, it has the best clustering ever
| invented for any OS in history, and above all: if it ain't
| broke, _don 't fix it._
| namdnay wrote:
| I wouldn't say Symbian was C++ - I remember it being a sort of
| intermediate step between C and C++. There were no exceptions, no
| STL, and many other weird quirks
| t43562 wrote:
| C is still C without libc and python is still python even if
| you can't "import os, sys" so why does lack of the STL matter
| to C++ ? Symbian used a standard C++ compiler and mandated a
| certain style of programming because C++ with STL and
| exceptions wasn't actually safe on small machines with limited
| memory. No exceptions were allowed in the kernel IIRC and new
| had to return an error code rather than throw an exception
| because handling an out of memory exception required....spare
| memory.
| ok123456 wrote:
| Very big omission of TempleOS. It was written in HolyC.
| trashburger wrote:
| Hmm, they didn't list SerenityOS[0] under the C++-based operating
| systems for some reason... maybe it's still too under the radar?
|
| [0]: https://serenityos.org/
| eatonphil wrote:
| I don't think it's any more under the radar than Redox or Tock
| listed in the article.
| dang wrote:
| Discussed yesterday:
|
| _SerenityOS Browser now passes the Acid3 test_ -
| https://news.ycombinator.com/item?id=30853392 - March 2022 (141
| comments)
|
| Lots more at https://news.ycombinator.com/item?id=30858768
| LordDragonfang wrote:
| Nor do they mention Google's upcoming Fuchsia OS, whose Zircon
| Kernel is written almost entirely in C++. While not quite
| "beloved" yet, it generates substantial buzz every time it hit
| the HN front page.
| [deleted]
| mal10c wrote:
| This took me down memory lane but in a weird way. One of the
| first languages I really took to was vb6. I was absolutely
| convinced I could write an OS with that language... I tried and
| tried - really not knowing what I was doing and finally realized
| its limitations. Such a good lesson on using the right tool for
| the job.
| Tozen wrote:
| Article forgot to mention that Niklaus Wirth was a consultant for
| the team at Apple that created Object Pascal. Apple had used
| variations of Pascal and Object Pascal for early Macs. This was
| in addition to companies using Wirth's Modula-2 and Oberon.
|
| Part of the reason why C became so prevalent is it being created
| within and backed by AT&T (one of the largest companies in the
| world back then). They pushed C and Unix everywhere, it took
| hold, and here we are today.
| glouwbug wrote:
| We could have as easily lived in a world where we all
| programmed Go because Google pushed gUnix should they have been
| the industry leaders in 80s, or something like that.
| tremon wrote:
| Do these operating systems also use a non-C calling convention
| for libraries?
| ncmncm wrote:
| It is common for an OS to use a non-C protocol for actual
| system-call traps, but also unfortunately common not to
| document, support, or maintain those, so users are obliged to
| call a provided C library for to use them.
| steveklabnik wrote:
| Hubris, the Rust OS I work on, is... aggressively static and
| doesn't have dynamic linking (or loading, in fact), but the
| kernel ABI is non-C, yes:
| https://hubris.oxide.computer/reference/#_syscall_abi
|
| (We use it in Rust via inline assembly stubs.)
| jnxx wrote:
| I think Modula-2 was also quite influential in one of the main
| programming languages for modern industrial PLCs, called
| Structured Text. One of its features are lightning fast compile
| times compared to C++.
| Decabytes wrote:
| It would be fun to try and make a scheme based Kernel/OS. My
| reasoning is that older revised reports document a much smaller
| Scheme language than r6rs and r7rs so would be easier to
| implement in asm. Then once you have a working Scheme you can
| build on top of it
| wglb wrote:
| And start bootstrapping with sector lisp
| https://justine.lol/sectorlisp/
| greenyoda wrote:
| The article notably omits Multics, which was written in PL/I:
| https://en.wikipedia.org/wiki/Multics
|
| Multics, first released in 1969, was a major influence on Unix.
| From the Wikipedia article:
|
| > _Multics was the first operating system to provide a
| hierarchical file system, and file names could be of almost
| arbitrary length and syntax. A given file or directory could have
| multiple names (typically a long and short form), and symbolic
| links between directories were also supported. ... It was also
| the first to have a command processor implemented as ordinary
| user code - an idea later used in the Unix shell. It was also one
| of the first written in a high-level language (Multics PL /I),
| after the Burroughs MCP system written in ALGOL._
| Koshkin wrote:
| > _Wirth a go_
|
| They should rename Oberon to Wirth.
| butlerm wrote:
| > C fans tend to regard BCPL as an unimportant transitional step,
| but it was used in two celebrated OSes. One was the TRIPOS
| operating system, later used as the original basis of AmigaOS.
|
| The latter is a bit of an exaggeration. Tripos related code
| written in BCPL was used for "AmigaDOS" - meaning the filesystem
| driver, the command line shell, a few bits of process control,
| and a variety of generic command line utilities. It was not used
| for the kernel (Exec), or graphics support, or the gui library
| (Intuition), or the graphical shell (Workbench), or lower level
| device drivers, which were all written in C or 68K assembly.
|
| The Tripos derived parts were nice though, generally nicer than
| the MSDOS equivalent and somewhat friendlier than the Unix
| command line, if the filesystem was a little slow, comparatively
| speaking.
| hulitu wrote:
| As far as i know MS windows was written in C++. DomainOS (Aegis)
| was written in Pascal.
| abnercoimbre wrote:
| We asked Walter Bright at Handmade Seattle [0] what he thinks of
| a future with non-C ABIs. He makes the case we must all accept: C
| is deeply entrenched and all-encompassing.
|
| That's not to discourage creative efforts -- it's more like "be
| aware this is 10x bigger than Mt. Everest"
|
| [0] https://vimeo.com/652821195#t=10m52s
| skissane wrote:
| Also z/OS (formerly known as MVS) -large chunks of it are written
| in an IBM-internal-use-only PL/I dialect called PL/X.
|
| The core of the AS/400 operating system, OS/400, the Licensed
| Internal Code (LIC), used to be written in another IBM-only PL/I
| dialect, PL/MP. In, the 1990s, while porting the AS/400 from CISC
| to RISC, they rewrote all the PL/MP code into C++. But IBM has
| another private dialect of PL/I called PL/MI, which was used to
| write the higher level part of OS/400, the XPF - basically PL/MI
| compiled to the virtual machine bytecode but PL/MP compiled to
| the machine code of the physical hardware. From what I
| understand, parts of the XPF are still written in PL/MP even in
| the latest versions of OS/400 (now called IBM i OS), although
| newer components tend to be developed in C/C++ instead. Other
| parts of the XPF were written in Modula 2, and I believe there is
| still some Modula 2 code surviving in the most recent versions as
| well.
|
| IBM has had even more secret proprietary PL/I variants. The long
| defunct IBM 8100 line's DPPX operating system was a written in
| PL/DS. Another dialect, PL.8, used to be used to write IBM's
| compilers (although I believe it has now been replaced by C++)
| and also some of the firmware for the mainframes. IBM even wrote
| a GCC front-end for PL.8, although it appears they never released
| it.
| deepspace wrote:
| Speaking of PL/I, Intel made a few operating systems in the
| late 70s / early 80s written in their dialect of PL/I called
| PL/M. There was ISIS, which ran on their microprocessor
| development stations, and iRMX, the real time O/S running on
| the target microprocessors.
|
| Having used both, ISIS was quite a bit more advanced than
| 'competitors' like DOS and CP/M at the time.
| skissane wrote:
| > Speaking of PL/I, Intel made a few operating systems in the
| late 70s / early 80s written in their dialect of PL/I called
| PL/M
|
| PL/M was created by Gary Kildall, creator of CP/M, and CP/M
| 1.x was written in PL/M. In CP/M-80 2.x, they rewrote the
| kernel (BDOS) and command processor (CCP) in 8080 assembly to
| improve performance, but utility programs were still written
| in PL/M. Some later CP/M ports, such as CP/M-68K and
| CP/M-8000, were written in C instead.
|
| Wikipedia says that IBM used PL/M too - to write some of the
| firmware for the CISC AS/400s (the Service Processor).
| wglb wrote:
| Here is a trivia question for you. What language was the first
| optimizing compiler written in?
| pinewurst wrote:
| It was the initial IBM FORTRAN written in 704 assembler. Very
| much an optimizing compiler.
| wglb wrote:
| Not quite.
|
| The idea of "basic blocks", key to optimizing compiler, was
| invented by the team that wrote the Fortran I compiler in
| Fortran I. See https://ucla-
| biostat-257-2020spring.github.io/readings/fortr...
| rst wrote:
| The first Fortran I compiler is the one pinewurst was
| referring to, and it was written for the 704 -- in 704
| assembly. Source code is here -- this is Fortran II
| (basically Fortran I plus provision for separately
| compiled subprograms), but obviously still assembly: http
| ://www.softwarepreservation.org/projects/FORTRAN/index.h.
| ..
|
| (The article in the course reading you cited also
| correctly states that the compiler was written in
| assembly, for the 704.)
| Tao331 wrote:
| IIRC this is the punchline of a story about Seymour Cray
| programming a BIOS in octal via front switches and
| blinkenlights. Or somewhere in that ballpark. It's a person,
| so the answer is DNA.
| wglb wrote:
| Nope. This happened before Seymour was at Control Data.
| javajosh wrote:
| Surely a more accurate answer would be "Cray's bookshelf".
| the_only_law wrote:
| I actually don't mind PL/I. Always wondered one they needed a
| ton of derivative proprietary languages. I think I'd almost
| prefer language bloat.
| zozbot234 wrote:
| PL/I is a kitchen sink language already.
|
| Modern languages tend to minimize the amount of features that
| they provide in the language itself, and do as much as
| possible in their standard libraries.
|
| (This also goes together with simplified syntax, because
| user-defined complex syntax is hard, slow and potentially
| ambiguous, whereas limiting verbose, "readable" syntax to
| just a handful of language-provided features is just silly.)
| nobody9999 wrote:
| >PL/I is a kitchen sink language already.
| Speaking as someone who has delved into the
| intricacies of PL/I, I am sure that only Real Men
| could have written such a machine-hogging, cycle-
| grabbing, all-encompassing monster. Allocate
| an array and free the middle third? Sure! Why not?
| Multiply a character string times a bit string and
| assign the result to a float decimal? Go ahead!
| Free a controlled variable procedure parameter and
| reallocate it before passing it back? Overlay
| three different types of variable on the same
| memory location? Anything you say! Write a
| recursive macro? Well, no, but Real Men use rescan.
| How could a language so obviously designed and
| written by Real Men not be intended for Real Man
| use?
|
| Heh. Heh. Heh.
|
| [Source: http://www.anvari.org/fortune/Miscellaneous_Collec
| tions/3620... ]
| Koshkin wrote:
| Looks like a perfect language to write an OS in!
| kps wrote:
| The design criteria are as follows: 1.
| Anything goes. If a particular combination of symbols
| has a reasonably sensible meaning, that meaning will be
| made official.
|
| https://dl.acm.org/doi/10.1145/363707.363708
| shakna wrote:
| > IBM even wrote a GCC front-end for PL.8, although it appears
| they never released it.
|
| Oh, how I had hoped I had scrubbed PL.8 from my memory.
|
| "80% of PL/I." Only missing everything that made it bearable to
| use PL/I. No interrupts. You get ITERATE but no LEAVE
| statement. No STATIC, EXPORT or PACKAGE. You do get DEFINE...
| But not the one in other IBM PL/? langs. Incompatible.
|
| The two weeks I spent debugging an interbank transferring
| program IBM wrote for them decades ago was one of the least
| pleasant experiences I've had when actually having access to
| the source code (albeit printed on five reams of paper, rather
| than digital). Give me pre-standard COBOL or FORTRAN over PL.8,
| any day.
|
| Though, to be fair, the version of PL.8 I was granted access to
| was the original compiler, only. Not the updated one used for
| z/Arch firmware, today. Which I think solves a lot of those
| things. And I do mean the original compiler. The very, very
| first version.
|
| I said no interrupts? Most of IBM's PL/? langs will signal when
| they run out of memory, so you can do something intelligent to
| kick the never-allowed-to-go-down mainframe into gracefully
| continuing to crunch endlessly (like clearing up part of a
| queue, and re-requesting it from a previous point). That signal
| is missing. So instead, you have developers build their own
| systems of heuristics to guess if a memory allocation had
| failed. Heuristics that had begun to fail, after an old network
| card burned out and was replaced with a similar one... But a
| faster, more modern, one. Faster hardware is the bane of old,
| hard-coded software. DCL IntVar INTEGER BASED
| (Cursed)
| skissane wrote:
| > The two weeks I spent debugging an interbank transferring
| program IBM wrote for them decades ago was one of the least
| pleasant experiences I've had when actually having access to
| the source code (albeit printed on five reams of paper,
| rather than digital). Give me pre-standard COBOL or FORTRAN
| over PL.8, any day.
|
| Interesting. I didn't realise PL.8 was used to write
| application software. I knew about it being used to write
| compilers, firmware, etc. If customer applications were
| written in it, did this mean they actually shipped PL.8
| development tools to customers? I thought it was always IBM-
| internal-only.
___________________________________________________________________
(page generated 2022-03-31 23:00 UTC)