[HN Gopher] Kerla: Monolithic kernel in Rust, aiming for Linux A...
       ___________________________________________________________________
        
       Kerla: Monolithic kernel in Rust, aiming for Linux ABI
       compatibility
        
       Author : Klasiaster
       Score  : 400 points
       Date   : 2021-10-25 10:29 UTC (12 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | jacquesm wrote:
       | Rewriting the Linux kernel in rust would be a rehash of why we
       | ended up with Linux in the first place: rewriting an older OS
       | from scratch.
       | 
       | I would love to see a solid implementation of a real micro kernel
       | based OS based on a message passing core in Rust to become a
       | viable alternative to the now 50 year old Unix architecture. This
       | would give us a possibility of some real progress on the security
       | and process scheduling front, both of which are severely lacking
       | in the Unix model.
        
         | chasil wrote:
         | Fuchsia and QNX provide a lot of what you want.
         | 
         | https://en.wikipedia.org/wiki/QNX
         | 
         | https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
         | 
         | https://lobste.rs/s/p8bizb/getting_know_fuchsia_google_s_ope...
        
         | abendstolz wrote:
         | Just in case you don't know about it: https://redox-os.org/
         | could be interesting to you :)
        
         | dnautics wrote:
         | I believe someone is building something like that on top of
         | sel4, which gives even stronger correctness guarantees than
         | Rust, even though it's C.
        
         | afr0ck wrote:
         | Why do people implicitly assume that micro kernels are the best
         | OS design?
         | 
         | In fact, micro kernels have a lot of issues and limit the
         | possibilities of what an OS can do! One of the most important
         | issues is they prevents resource sharing in a rich and
         | efficient way between the components of the kernel. They make
         | building rich relationships between kernel data structures and
         | kernel subsystems very messy, complicated, impracticable and
         | very inefficient.
         | 
         | Micro kernels are delusional and people should stop implicitly
         | assuming their superiority.
        
           | sumtechguy wrote:
           | Message passing at first seems interesting and neat and
           | useful. It is for some cases. But it comes at a cost of
           | complexity and latency.
        
           | PaulDavisThe1st wrote:
           | This is a limitation of the memory model, not the
           | microkernel. If you had a single hardware-protected address
           | space, kernel modules could share resources freely.
           | 
           | But this does argue against message passing, since such
           | systems preferentially try to use messages over sharing
           | whenever it's possible.
        
         | pcwalton wrote:
         | Written properly, a monolithic kernel in Rust would provide
         | many of the benefits of a microkernel, due to the memory
         | safety. It'd be much along the same lines as Microsoft
         | Research's Midori, with software isolation instead of hardware
         | isolation.
        
         | PaulDavisThe1st wrote:
         | Mach's internals didn't follow the Unix model, and was by
         | almost any metric a failure.
         | 
         | There have been several other "real micro kernel based OS based
         | on a message passing core", and they too, by almost any metric,
         | have been a failure.
         | 
         | Process scheduling in Linux in 2021 is far, far ahead of
         | anything in any other OS, ukernel or MP-based. The idea that
         | there hasn't been progress in this area is really completely
         | absurd. In fact, the general idea that Linux follows "the now
         | 50 year old Unix architecture" is also pushing the boundaries
         | quite a lot.
        
           | dnautics wrote:
           | > "real micro kernel based OS based on a message passing
           | core", and they too, by almost any metric, have been a
           | failure.
           | 
           | BeOS wasn't half bad. The failure was probably mostly
           | commercial. It is true that they moved their networking stack
           | into the kernel, but it's not entirely clear to me that they
           | _had_ to do that, it was just the most expedient way to get
           | acceptable performance and stability given the (programmer-
           | time) resourcing constraints of the project.
        
             | PaulDavisThe1st wrote:
             | > BeOS wasn't half bad. The failure was probably mostly
             | commercial.
             | 
             | Fair point. You could probably say this of other attempted
             | microkernels too, to be even fairer. But that doesn't
             | really change the fundamental point that just cooking up "a
             | better OS design" doesn't lead to it being successful.
             | There's a lot more in play that "mere quality".
        
               | dnautics wrote:
               | well if that sort of "successful" is your key metric,
               | this is true for a lot of things and has been known since
               | biblical times: "a person may labor with wisdom,
               | knowledge and skill, and then they must leave all they
               | own to another who has not toiled for it"
               | 
               | you did say "by almost any metric".
        
               | PaulDavisThe1st wrote:
               | the metric i was mostly thinking was inspired by the
               | parent to my post, which seemed to me to imagine that
               | Rust+microkernel => widescale deployment.
        
       | exabrial wrote:
       | Dunno: just a hobby, won't be big and professional like linux.
       | 
       | :)
        
       | rvz wrote:
       | A star has been born. A Linux alternative needs to be compatible
       | with Linux binaries and also support modern architectures.
       | 
       | If you want Linux to survive, it needs to be _re-written_ from
       | scratch, optimised in Rust and compatible with the existing Linux
       | binaries or drivers.
       | 
       | This project might have a chance.
       | 
       | From: [0]
       | 
       | > TL;DR: I'm writing a Linux clone in Rust just for fun. It does
       | NOT aim to replace the Linux kernel.
       | 
       | A certain someone thought that their hobby OS Minix clone won't
       | be 'big or professional like GNU'.
       | 
       | It's time for 'change'.
       | 
       | [0] https://seiya.me/writing-linux-clone-in-rust
        
         | tonetheman wrote:
         | sigh...
         | 
         | Linux is going to survive. And no it does not need to be re-
         | written from scratch in Rust. Someone lied to you at the church
         | of rust.
        
           | thawkins wrote:
           | But its a fun thing to do, and that's a good enough reason to
           | do it. After all is that not exactly how Linux came into
           | being anyway.
        
             | yjftsjthsd-h wrote:
             | Linux came into being as a fun hobby project. It was
             | _successful_ because it gained traction with the wider
             | community and got huge commercial backing because it was
             | vastly superior to most of the alternatives from a user
             | perspective ( "oh hey, I can run my web servers on this
             | 100% free kernel plus the GNU userland and have a mostly
             | good server for zero software cost on commodity hardware").
             | But today, Linux itself exists, and while there are
             | underlying technical changes that could improve on it, from
             | an end-user perspective (especially potential commercial
             | backers) anything else would be barely different, if they
             | could tell at all. You'd need some huge compelling
             | advantage - that a user cares about, nicer code and 5 bugs
             | a year instead of 10 bugs a year isn't going to cut it - to
             | actually gain any ground.
        
               | pjmlp wrote:
               | It gained commercial tracking because it represented a
               | way for IBM, Oracle and Compaq to reduce development
               | costs on their own systems, and thus started sponsoring
               | Linux development efforts around 2000.
        
             | simion314 wrote:
             | Maybe I will write a TODO app in Rust, should it be on top
             | of HN because is some unoriginal,incomplete,bug filled,toy
             | stuff just because the reason is written in Rust? 1
             | 
             | My assumption is that all titles with Rust get blind
             | upvotes, sure a new kernel is cool but would be more cool
             | if is more then someone toy project, this should probably
             | be posted and upvoted in Rust forums so it gets support
             | from the fans and hopefully the Rust guys will decide what
             | kernel should they focus on and stop promoting every
             | student weekend project here.
        
               | skocznymroczny wrote:
               | I mean, if you had a headline like "Show HN: org-mode in
               | Rust" you'd get a ton of clicks
        
         | breakfastduck wrote:
         | You think Linux, which the vast majority of web servers run
         | today, is going to die out if it's not rewritten from scratch
         | in Rust?
         | 
         | How on earth have you come to that one?
        
         | MrBuddyCasino wrote:
         | Sometimes, just sometimes, I wished one could post soyjaks on
         | HN.
        
         | _joel wrote:
         | Absolute codswallop. Linux will be here for many, many years to
         | come yet.
        
           | rvz wrote:
           | Absolutely! But in _Rust_.
           | 
           | Why do you think the Linux developers are bolting on Rust in
           | the Linux kernel themselves? [0]
           | 
           | It has already been admitted.
           | 
           | [0] https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda
           | @ke...
        
             | gizdan wrote:
             | It'll take decades for Linux to have everything re-written
             | in Rust. Even then it'll take even longer for _every_
             | machine to run RLinux.
        
             | _joel wrote:
             | They've been writing drivers and such for years now and
             | they've indeed levelled up the tooling integration. Your
             | original post said along the lines of "if linux is to
             | survive". Even if it was a full rewrite in rust (not going
             | to happen) then it's _still_ be Linux, with the same
             | leadership. Most users wouldn 't be able to tell the
             | difference.
        
         | fnord123 wrote:
         | Not with Apache v2 License.
        
       | ordu wrote:
       | Just to clarify goals of the project:
       | 
       | "I completely agree that rewriting existing large, feature-rich,
       | and robust operating system kernels is not a good idea. However,
       | a question came to my mind: what are the pros and cons of writing
       | a practical UNIX-like OS kernel in Rust from scratch? How does it
       | look like? This is why I started this project."[1]
       | 
       | [1] https://seiya.me/writing-linux-clone-in-rust
        
       | gabrielhidasy wrote:
       | For each piece of software, there just be a version in Rust, and
       | that version must advertise it's rustiness
        
         | jatone wrote:
         | I mean honestly who cares? its their time. I use rust over
         | c/c++ these days for practical reasons.
         | 
         | 1. the toolchain is better. (cargo ala)
         | 
         | 2. maintenance of the software is better. (static linked
         | binaries, easier deploys)
         | 
         | 3. memory safety.
         | 
         | 4. resource consumption (cpu/ram)
         | 
         | when I don't need 4 I use golang (most cases) because of 1, 2,
         | and 3.
         | 
         | and I choose tools (all other things being equal) written in
         | rust for the same reasons.
        
           | gspr wrote:
           | Me too, but in my case it's _despite of_ point 2.
        
         | Klasiaster wrote:
         | The type of software makes the difference - does it process
         | untrusted input, maybe even over the network? This here is a
         | kernel including a memory-safe TCP/IP stack
         | (https://github.com/smoltcp-rs/smoltcp/), and not having it
         | crash or be full of security vulnerabilities due to preventable
         | memory corruption is a quality beyond personal language
         | preferences.
        
           | fivelessminutes wrote:
           | It is possible to overstate a valid case until it becomes
           | meaningless... are modern OS IP stacks written in C really
           | 'crash'ing or 'full of security vulnerabilities'? No.
        
             | roblabla wrote:
             | There has actually been a few crashes in the _core_ TCP /IP
             | stack of both Linux and Windows in the last 5 years, though
             | I'm not sure any of them were due to memory safety (most
             | were logic bugs as far as I can tell). That said, rust's
             | approach to error handling could help avoid crashes here as
             | well.
             | 
             | However, we have seen *many* memory-safety crashes in the
             | drivers to talk to network interfaces. Those really are
             | full of security vulnerabilities. The most prominent recent
             | one was broadcom's wifi driver having heap buffer overflows
             | allowing remote code execution by anyone within wifi range
             | in 2019[0].
             | 
             | [0]:
             | https://www.bleepingcomputer.com/news/security/broadcom-
             | wifi...
        
             | ls65536 wrote:
             | The fact is that it is overwhelmingly more difficult to
             | write and guarantee that your C code is "safe" to the same
             | extent that a naively-written Rust program which
             | accomplishes the same thing would be, especially as your
             | software's complexity increases.
             | 
             | Is it possible to write C code that has the safety
             | properties of Rust code which provides the same
             | functionality? Absolutely, in theory. But will you be able
             | to accomplish this and then guarantee this safety and have
             | the confidence that future code changes won't break those
             | guarantees? That becomes exceedingly more difficult to do,
             | nevermind the extra time and effort it would take. In most
             | cases this would be prohibitively expensive, and you'd just
             | get on with life hoping for the best and mitigating what
             | you can.
             | 
             | In my view, one of the killer features of Rust is its
             | ability to be easily ABI-compatible with C as needed, which
             | I see as a pragmatic feature of Rust that acknowledges the
             | messy software reality of our world and the fact that C (or
             | any other language) isn't just going to be "replaced" by
             | rewriting stuff everywhere, neither anytime soon nor
             | probably ever in totality. This lets us more readily
             | combine the great capabilities of Rust with the enormous
             | volumes of useful code that already exist today without
             | resorting to dogmatic approaches of always thinking we need
             | to rewrite the latter. The "rewrite everything in Rust"
             | idea is fun in an academic kind of way, but in the real
             | world economics will prevent that from happening
             | everywhere, in a similar way that economics usually
             | prevents you from validating and providing certain safety
             | guarantees about your C code.
        
             | Klasiaster wrote:
             | The TCP/IP stacks of the few popular OSes is not full of
             | security issues, but that took a lot of effort that a small
             | team can't do. Also, it's a matter of updating: after
             | reviewing the CVE database, would you be comfortable
             | running a 5 year old kernel without updates?
        
         | 29athrowaway wrote:
         | Now ask yourself why.
         | 
         | Mozilla decided to create Rust after years of using C/C++. Why?
         | 
         | Because they spent a fortune fixing the same problems over and
         | over again, and they realized that no matter how clever their
         | developers were, those problems were always going to arise as
         | long as they kept using C/C++.
         | 
         | Not because C/C++ is inherently bad, but because is more
         | flexible than strictly needed in some ways.
         | 
         | Rust was designed to prevent many classes of problems.
        
           | gabrielhidasy wrote:
           | I'm ok with things rewritten in Rust, really looks awesome as
           | a language. And the fact every rust project advertises rust
           | front and center compounds that image.
        
         | option_greek wrote:
         | For each rust submission, there shall be a comment at top
         | lamenting the gradual rusting of hackernews.
        
           | gabrielhidasy wrote:
           | Not lamenting in this case, the project is awesome and rust
           | is made for this. I just find amusing that almost all popular
           | rust projects I see have "in Rust" front and center
           | 
           | Must be an awesome language to work on.
        
         | wongarsu wrote:
         | After looking at the source I find this one to be much easier
         | to read than equivalent C, because Rust is a more expressive
         | language. So looking at what an all-rust kernel would look like
         | seems quite worthwhile. You can't reach anything new if you're
         | not willing to experiment.
        
       | fork-bomber wrote:
       | Very cool!
       | 
       | I think a killer feature would be to emulate the Linux kernel
       | device driver model such that existing C driver modules could be
       | used with Kerla. Further, borrow (no pun intended!) the Linux
       | device driver Rust wrappers from the Rust for Linux kernel
       | project to then enable writing Linux device drivers in Rust.
       | 
       | The question is whether emulating the Linux device driver model
       | (by which I kind of mean mimicking the set of core kernel
       | services needed to support drivers - kernel threads, sync
       | primitives, allocators etc) ends up being a massive task akin to
       | writing the core kernel itself.
       | 
       | I think there might be reasonable trade offs that make the
       | problem more tractable.
       | 
       | Having the ability to have a significantly simpler kernel with
       | Rust's memory and thread safety guarantees plus the ability to
       | use even a small set of Linux's massive set of drivers would be
       | very compelling.
        
         | Klasiaster wrote:
         | Maybe starting with the rumpkernel interface is easier and one
         | could already use some of the NetBSD drivers:
         | https://en.wikipedia.org/wiki/Rump_kernel
        
           | fork-bomber wrote:
           | That's a very good point indeed.
           | 
           | While I completely agree that NetBSD's Rump kernel would be a
           | great way to go (I've played with it in the past and quite
           | like it!) I do wish there was a similar initiative bound to
           | the Linux kernel.
           | 
           | The closest I've seen is Octavian Purdila's Linux-Kernel-
           | Library project: https://lkml.org/lkml/2015/11/3/706
           | 
           | Not sure how far they've come though. Perhaps Kerla could be
           | a prime mover for them ?
        
         | roblabla wrote:
         | Linux drivers don't have a fixed ABI (or even API), making such
         | a feat somewhere between hard and impossible. You'd have to
         | constantly play catch up with the latest kernel refactorings,
         | which would require a tremendous amount of engineering effort.
         | 
         | Heck, it'd probably be easier to emulate the Windows driver
         | architecture - at least those have proper ABI compat.
        
           | fork-bomber wrote:
           | I asked this question during the recent Linux kernel plumbers
           | conference: "How often in practice do the kernel A{B,P}Is fro
           | device drivers change ?". I got a very mixed response.
           | 
           | My take away was that while 'new' driver classes/subsystems
           | do involve more churn in the core kernel support subsystems,
           | there is a clear trend towards convergence in terms of
           | significant change.
           | 
           | If the change truly was significant and continuous then that
           | would impede productisation with the Rust kernel. I'm not
           | saying that the A{B,P}Is are in any way stable but I
           | personally don't think that keeping up will be an intractable
           | problem.
           | 
           | In the spirit of Kerla's authors prime motive - which appears
           | to be hacking for knowledge - this might be something to try,
           | perhaps with a driver class that is known to be more stable.
        
           | krater23 wrote:
           | >Heck, it'd probably be easier to emulate the Windows driver
           | architecture - at least those have proper ABI compat.
           | 
           | And better drivers.
        
             | robocat wrote:
             | Actually many Linux drivers are far better than their
             | Windows counterparts - many Windows drivers are complete
             | shit. It is entirely dependant on your particular hardware.
             | I know the driver for my WiFi in my laptop is far crappier
             | in Windows for example.
        
           | trasz wrote:
           | FreeBSD's linuxkpi - bits and pieces of Linux KAPI provided
           | to make porting Linux drivers to FreeBSD easier - works just
           | fine, so this assumption is wrong.
        
         | pjc50 wrote:
         | > The question is whether emulating the Linux device driver
         | model (by which I kind of mean mimicking the set of core kernel
         | services needed to support drivers - kernel threads, sync
         | primitives, allocators etc) ends up being a massive task akin
         | to writing the core kernel itself.
         | 
         | The kernel has (deliberately) never had a simple or stable
         | driver ABI, so I think this is potentially a huge task. And
         | then you've also left a C-based attack surface in place.
         | 
         | I'm not sure what the minimal set is if you decide to only
         | support 2021-era hardware.
         | PCIe+SATA+framebuffer+USBhost+USBHID?
        
           | fork-bomber wrote:
           | Towards the C-based attack surface:
           | 
           | I think using the Rust-for-linux wrappers to implement the
           | drivers in Rust would eventually help.
           | 
           | Also, even if initially Linux C drivers were transplanted
           | into Kerla that could be a net benefit if the tradeoff of
           | wide device driver availability vs increase in the trusted
           | compute base was acceptable.
           | 
           | I personally think that tradeoff would be acceptable - if the
           | transplanting ability was tractable. It's a hard one to
           | absolutely agree on, admittedly.
           | 
           | Towards the minimal set of hardware, fair point! I think that
           | one should be able to get quite far with judicious selection
           | of device driver classes and leave the truly in-flux classes
           | alone ?
        
       | nivenkos wrote:
       | Post it when it's production ready.
       | 
       | A big problem in the Rust ecosystem is the lack of continued work
       | and support on libraries (understandably, since many are new and
       | aren't commercially supported).
       | 
       | For example, pyroute2 is still better for Linux admin that the
       | equivalent Rust crates. Whilst the latter often fully support
       | async, none of them cover all of the features of pyroute2 nor the
       | easy-to-use API.
       | 
       | Hopefully things will improve over time, it's always a shame to
       | see great projects be abandoned.
        
         | kryptiskt wrote:
         | > Post it when it's production ready.
         | 
         | This is Hacker News, not Enterprise IT News.
        
           | nivenkos wrote:
           | I just meant that even big projects posted here in the past
           | like the Headcrab debugger: https://github.com/headcrab-
           | rs/headcrab have been seemingly abandoned now.
           | 
           | Even Rocket https://github.com/SergioBenitez/Rocket seems to
           | have greatly slowed development unfortunately.
        
             | fbanon wrote:
             | Maybe those projects are considered "done".
        
             | __s wrote:
             | Nowadays the Rust http libraries to use are warp or axum
             | _(but Rocket works as is)_
             | 
             | https://github.com/seanmonstar/warp
             | 
             | https://github.com/tokio-rs/axum
             | 
             | re headcrab, https://github.com/headcrab-
             | rs/headcrab/issues/132
        
               | ibraheemdev wrote:
               | > Nowadays the Rust http libraries to use are warp or
               | axum
               | 
               | Warp and axum are certainly not the only choices, and
               | saying they are the best choices is a matter of opinion.
               | Actix-web, rocket, tide, warp, axum are all great web
               | frameworks.
        
         | Klasiaster wrote:
         | The project author asks for help at the end of the blog post1
         | announcing it, if you want to have a production ready kernel,
         | maybe invest some time?
         | 
         | 1 https://seiya.me/writing-linux-clone-in-rust
        
       | Klasiaster wrote:
       | A minimal interactive demo running busybox is here:
       | ssh root@kerla-demo.seiya.me
        
         | thawkins wrote:
         | is there not an opportunity to implement the posix filesystems
         | interface on top of something other than a block device system.
         | That would be fun.
        
         | codetrotter wrote:
         | That's very cool! Could you share some details on how to set it
         | up for oneself with AWS Firecracker the same way that this demo
         | was set up?
         | 
         | Also, as owner of the instances are you able to see what
         | commands people are running? First thing I did was to type
         | hello, and it said hello world to me :)
        
           | nuta wrote:
           | Author here. I've made the demo system public [1] but it's
           | written just for me so it should be painful to set up the
           | same environment.
           | 
           | The mechanism is pretty simple: a Node.js server (running on
           | genuine Linux) listens on tcp:22. Once you connect, it boots
           | a dedicated Firecracker microVM instance and forwards network
           | packets between your SSH client and VM.
           | 
           | Regarding the command history, others (including I) can't see
           | what you type. If you could, it must be a vulnerability.
           | 
           | [1] https://github.com/nuta/kerla-demo
        
           | Klasiaster wrote:
           | I'm not the project author, maybe you can ask via e-mail, see
           | https://seiya.me/
        
             | codetrotter wrote:
             | Ok, thanks :)
        
       | EdSchouten wrote:
       | This is awesome. Imagine the potential if it was just complete
       | enough to run Kubernetes' kubelet on a cloud instance. Lots of
       | security minded folks would love it.
        
         | JakaJancar wrote:
         | I wonder, if you run every container in it's own MicroVM a-la
         | Firecracker like Fargate does (or Kerla's demo host), does the
         | security of the kernel still matter as much?
        
         | phkahler wrote:
         | That's IMHO how you get something like this off the ground.
         | Make it useful for a simple use-case while offering something
         | of value - in this case the security of Rust. It can then
         | become a preferred solution to _some_ problem and that will
         | bring users and developers looking to incrementally expand it.
        
       | nuta wrote:
       | Author here. I'm surprised to see my _hobby_ project on Hacker
       | News.
       | 
       | I know this kind of stuff spark the ``it's meaningless to rewrite
       | everything (especially Linux) in Rust'' debate. I agree 100% that
       | rewriting Linux in Rust (or your favorite language) is just a
       | waste of time and such a project won't live long.
       | 
       | That said, what I'd say here is that it's fun. Really fun.
       | Implementing ABI compatibility requires you to understand under
       | the hood of the kernel. You can learn how printf(3) works. You
       | can learn what happens before main().
       | 
       | Especially, in Rust, it's inspiring to explore ways to write a
       | kernel safely and expressively. One of my favorite part is the
       | implementation of read system call [1].
       | 
       | Lastly, You can try this OS on SSH. Hope you enjoy :)
       | ssh root@kerla-demo.seiya.me
       | 
       | [1]:
       | https://github.com/nuta/kerla/blob/main/kernel/syscalls/read...
        
         | jedimastert wrote:
         | Aside from "rewrite in rust" debate that I was unaware of,
         | there seems to be this pervasive attitude among the HN hivemind
         | that the end-goal for all projects, side- or main-, is
         | "launch", and therefore needs a market analysis to decide the
         | "worth" of such an idea, and it ends up being rather silly.
         | 
         | For example, I've written a Mandelbrot visualizer so many times
         | I've lost count. Not because the world needs another poorly
         | written or optimized rainbow-ladybug-simulator, but because it
         | serves as a slightly-non-trivial hello-world. For example, it's
         | the first end-to-end thing I made in Common Lisp.
         | https://git.sr.ht/~amtunlimited/mandelbrot-plot
        
           | pjmlp wrote:
           | I have a A* search algorithm and a toy compiler that I use
           | exactly for the same purpose.
           | 
           | I just rewrite them all the time as means to get a feeling
           | about programming languages.
           | 
           | I dumped a couple of other stuff on GitHub so that HR people
           | are happy to get a link that they never read anyway.
           | 
           | Then I get back to Java and .NET at the office. :)
        
             | jedimastert wrote:
             | I've actually been considering adding a toy compiler to my
             | collection of getting-up-to-speed projects, do you mind
             | sharing what you think are good features?
        
               | pjmlp wrote:
               | Try to follow along the Tiger Book, in the variant that
               | appeals to you.
               | 
               | https://www.cs.princeton.edu/~appel/modern/
               | 
               | It covers most areas quite alright even if a bit aged.
        
               | spoiler wrote:
               | Not OP, but there's also "crafting interpreters". In the
               | second half of the book you emit bytecode for whatever
               | language you designed in the first half, and also
               | implement a VM for said bytecode.
        
         | a-dub wrote:
         | nice project! curious how the rust ownership model works in a
         | monolithic kernel context. are the lifetimes of kernel
         | structures associated with a process "owned" by the process
         | itself somehow?
        
         | rakoo wrote:
         | > I know this kind of stuff spark the ``it's meaningless to
         | rewrite everything (especially Linux) in Rust'' debate. I agree
         | 100% that rewriting Linux in Rust (or your favorite language)
         | is just a waste of time and such a project won't live long.
         | 
         | Considering that this is exactly how Linux was born (just a
         | hobby project for fun), I wouldn't assume so fast that it's
         | useless. Moreover, you don't need to justify yourself if you
         | just want to have fun !
        
           | hnlmorg wrote:
           | I don't see why we need to pretend that every hobby has the
           | potential for greatness. Being a hobby is a good enough end
           | to itself.
           | 
           | In fact in this instance I think it's a little disingenuous
           | to quote Linux and say it could happen again. The industry is
           | totally different now. There's much more competition than
           | there was when Linux was released and that competition is
           | much more mature too. Plus the bar for a production-quality
           | kernel is a lot higher than it was when Linux was released.
        
             | _joel wrote:
             | > Being a hobby is a good enough end to itself.
             | 
             | Indeed, there's a word for it which I find particularly
             | lovely, autotelic.
        
             | sennight wrote:
             | > In fact in this instance I think it's a little
             | disingenuous to quote Linux and say it could happen again.
             | 
             | Seems apropos to me, given the fact that a Linux ABI
             | compatible hobby project is under discussion - and that
             | everyone here is familiar with the famous Usenet
             | announcement.
             | 
             | > The industry is totally different now. There's much more
             | competition than there was when...
             | 
             | I wonder how you define "competition"? Because there were
             | way more operating systems in use then, and the industry
             | was far more fractured. Fractured in a way that was
             | meaningful - not like today where you can spin up a VM and
             | be productive in short order, thanks to the significant
             | lack of distinguishing difference. That is the really
             | interesting thing about these hobby projects - they
             | introduce possibilities that are either completely ignored
             | by organizations suffering from inertia constraints, or
             | can't be mimicked because they're diametrically opposed to
             | present designs.
        
           | uuddlrlr wrote:
           | Any rust project like this inevitably gets sardonic replies
           | about rewrite-in-rust fanaticism.
           | 
           | I agree they shouldn't have to justify themselves, but it is
           | handy to preempt that.
        
             | Iv wrote:
             | Also, the rust community is too nice to openly admit it,
             | but it's goal IS to displace C as the main low level
             | language especially for the kernel. This is doomed to
             | attract a lot of attention
        
           | nuta wrote:
           | Thanks! I'll keep doing Just for Fun :D
        
         | monopoledance wrote:
         | Thanks for checking in. Makes me happy to see your excitement
         | for learning the inner workings of things.
         | 
         | What do you think about other Rust OS projects like Redox?
         | 
         | https://www.redox-os.org/
        
           | nuta wrote:
           | In addition to its huge contribution to OS development in
           | Rust, as a microkernel enthusiast, it sounds exciting to
           | writing a microkernel in Rust, in the "Everything is a URL"
           | principle. Moreover, it can run a good-looking GUI on a real
           | machines [1]! I know it's very hard to be realized.
           | 
           | Aside from Redox, I'd mention Tock [2], an embedded operating
           | system. It introduces a novel components isolation using
           | Rust's power. I believe this field is where Rust shines and
           | am looking forward to seeing it in production.
           | 
           | [1]: https://www.redox-os.org/screens/ [2]:
           | https://www.tockos.org
        
             | panick21_ wrote:
             | There are more good efforts, the BeTrusted guys are working
             | on Xous, its a microkernel for a phone like device called
             | the Precurser.
             | 
             | https://github.com/betrusted-io/xous-core
             | 
             | As a embedded service processor OS for a big server rack,
             | Oxide Computer is working on 'HubrisOS'. They seem to have
             | not released it yet, but that will be open sourced.
             | 
             | https://github.com/oxidecomputer
             | 
             | Those are two efforts where I know real resources are going
             | into.
        
               | surajrmal wrote:
               | Is Hubris targeted at BMCs? Or is something different?
        
               | panick21_ wrote:
               | I think its an FPGA with a costume Open-Source Titan chip
               | on it (RISC-V). It is not really a traditional BMC, its
               | more like a service processor that does secure boot and
               | gets you in the OS. It does a few other things but I
               | think they really want it to have minimum functionality.
               | 
               | This is a great talk about what they do and why:
               | https://www.youtube.com/watch?v=vvZA9n3e5pc
        
               | bcantrill wrote:
               | This is a very reasonable inference, as it absolutely was
               | when I gave that talk. ;) Very shortly after that talk,
               | however, we came to the realization that the OpenTitan
               | was not going to be what we needed when we needed it, and
               | moved to a Cortex M7-based microcontroller for our
               | service processor (and a separate M33-based
               | microcontroller for our root of trust); Hubris is the
               | operating system that runs on those two MCUs.
        
               | panick21_ wrote:
               | Not that it matters all that much, but why a Cortex when
               | you can get RISC-V chips form SiFive (or whoever) that do
               | about the same stuff?
               | 
               | Does that mean the product will not have an FPGA? I kind
               | of liked that idea of updating the hardware.
               | 
               | P.S: Twitter spaces about the history of computing are
               | really fun. Love to hear more about all the dead computer
               | companies you researched. There is not enough content
               | about computer history out there.
        
               | bcantrill wrote:
               | Yeah... a bunch of reasons. We definitely have a couple
               | of FPGAs in the product (Bluespec FTW!), and we
               | anticipate that we will have more over time -- but not
               | for the RoT or SP, or at least not now. The reasons are
               | different for each, but include:
               | 
               | 1. ASICs are out of the question for us for a bunch of
               | economic reasons
               | 
               | 2. FPGAs by and large do not have a good security story
               | 
               | 3. The ones that do (or rather, the one that does) has an
               | entirely proprietary ecosystem and a terrible toolchain
               | -- and seems to rely on security-through-obscurity
               | 
               | 4. Once you are away from a softcore, the instruction set
               | is frankly less material than the SoC -- and the RISC-V
               | SoC space is still immature in lots of little ways
               | relative to the Cortex space
               | 
               | 5. Frankly, there's a lot to like about ST: good parts,
               | good docs, good eval boards, pretty good availability
               | (!), minimum of proprietary garbage.
               | 
               | We tried really hard (and indeed, I think my colleagues
               | would say that I probably tried a little _too_ hard) to
               | get FPGAs to be viable, and then to get FPGAs + hardcores
               | to be viable, and then multicore hardcores to be viable
               | (one for the SP, one for the RoT). Ultimately, all of
               | these paths proved to be not yet ready. And while we 're
               | big believers in FPGAs and RISC-V, we're even bigger
               | believers in our need to ship a product! ;)
        
               | bcantrill wrote:
               | Also, where are my manners?! Really glad you're enjoying
               | our Twitter Spaces[0] -- and thank you for the kind
               | words!
               | 
               | [0] https://github.com/oxidecomputer/twitter-spaces
        
               | bcantrill wrote:
               | Hubris is targeted at microcontrollers. Much more
               | information (and, importantly, the source code!) will be
               | available in our talk at the Open Source Firmware
               | Conference[0], the abstract for which elaborates on our
               | motivations and use case:
               | 
               |  _On Hubris and Humility: when "write your own OS" isn't
               | the worst idea_
               | 
               |  _Hubris is a small open-source operating system for
               | deeply-embedded computer systems, such as our server 's
               | replacement for the Baseboard Management Controller.
               | Because our BMC replacement uses a lower-complexity
               | microcontroller with region-based memory protection
               | instead of virtual memory, our options were limited. We
               | were unable to find an off-the-shelf option that met our
               | requirements around safety, security, and correctness, so
               | we wrote one._
               | 
               |  _Hubris provides preemptive multitasking, memory
               | isolation between separately-compiled components, the
               | ability to isolate crashing drivers and restart them
               | without affecting the rest of the system, and flexible
               | inter-component messaging that eliminates the need for
               | most syscalls -- in about 2000 lines of Rust. The Hubris
               | debugger, Humility, allows us to walk up to a running
               | system and inspect the interaction of all tasks, or
               | capture a dump for offline debugging._
               | 
               |  _However, Hubris may be more interesting for what it
               | doesn 't have. There are no operations for creating or
               | destroying tasks at runtime, no dynamic resource
               | allocation, no driver code running in privileged mode,
               | and no C code in the system. This removes, by
               | construction, a lot of the attack surface normally
               | present in similar systems._
               | 
               |  _This talk will provide an overview of Hubris 's design,
               | the structure of a Hubris application, and some
               | highlights of things we learned along the way._
               | 
               | [0] https://talks.osfc.io/osfc2021/featured/
        
             | monopoledance wrote:
             | Thanks for sharing!
             | 
             | I have to say, I feel a bit "dirty" carrying so much unused
             | and legacy code around with Linux, so I like people trying
             | to reinvent the wheel just for the pleasure of a fresh
             | start. For the aesthetics. Even if it's merely a fantasy
             | and not replacing anything soon, realistically. They are
             | also keeping OS development accessible to new generations
             | of geeks. The unfriendliness of C, the gigantic codebase
             | and seemingly distinct culture make the Linux kernel quite
             | off putting, filtering possible engagement by unfortunate
             | parameters IMO. Novel OS development in Rust takes away at
             | least some of those barriers and some of the gained
             | knowledge may be applicable with the Linux kernel later.
        
         | stjohnswarts wrote:
         | It doesn't matter if it's meaningless to the hackernews crowd,
         | it really only matters if it has meaning to you and if you're
         | learning.
        
         | matheusmoreira wrote:
         | > That said, what I'd say here is that it's fun. Really fun.
         | Implementing ABI compatibility requires you to understand under
         | the hood of the kernel.
         | 
         | > You can learn how printf(3) works. You can learn what happens
         | before main().
         | 
         | Yes! It's such a wonderful experience. This is exactly what I
         | most enjoy doing, just seeing how things work, maybe making my
         | own version. I hope you have lots of fun.
         | 
         | You're reimplementing Linux's kernel-userspace binary
         | interface, right? The system call interface is stable and
         | language agnostic, it's really nice. Some pointers for anyone
         | who'd like to know more:
         | 
         | https://man7.org/linux/man-pages/man2/syscalls.2.html
         | 
         | https://man7.org/linux/man-pages/man2/syscall.2.html
         | 
         | https://github.com/torvalds/linux/blob/master/Documentation/...
        
         | sophacles wrote:
         | This is really cool. Ignore the haters:
         | 
         | * Some people bake bread even though there's a good bakery
         | nearby.
         | 
         | * Some people grow gardens even though there's a farmers'
         | market down the street.
         | 
         | * Some people restore old cars even though there's a good
         | restoration professional in town.
         | 
         | * Some people make k8s clusters on rpi even though they could
         | rent that for cheap.
         | 
         | You're making an OS that's linux compatible even though there's
         | already linux. And that is awesome!
        
           | teekert wrote:
           | Yes, and some people write an OS that is similar to Minix and
           | won't ever be ported to anything beyond 386 and AT harddisks
           | ;)
        
         | nix23 wrote:
         | Really cool that you mentioned OSv on your github, really not
         | enough eye's on it, i think.
        
         | 0xTJ wrote:
         | Wow, I really like the implementation of that syscall. I've got
         | my own toy OS project in C, for the M68k, and my `read` is "a
         | little" less clean [0, 1].
         | 
         | [0]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L469
         | 
         | [1]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L907
        
           | nuta wrote:
           | Thanks! By the way, your MUTEX_WITH macro looks pretty
           | interesting to me. I've never seen the idea.
        
         | baktubi wrote:
         | Spending your time how you want to is never meaningless.
         | Especially if you're creating value in the world.
        
         | [deleted]
        
         | bitcoinmoney wrote:
         | How long did it take you to write this?
        
         | bpye wrote:
         | I honestly think there is real value here for firecracker style
         | containers. Running memory safe code for your whole stack, with
         | a minimal number of virtio devices? Fantastic!
         | 
         | The permissive license would also be interesting in some
         | applications.
        
         | qwerty456127 wrote:
         | Even "rewriting Linux in Rust" is far from a waste if you make
         | it run and if your code quality is good enough for others to
         | join.
         | 
         | If only you could write an entirely new (meant to be better,
         | for some cases at least) kernel compatible with Linux hardware
         | drivers - this would be not waste at all but in fact fantastic.
        
           | nuta wrote:
           | The most challenging point is, as others said, the lack of
           | the compatibility with Linux Driver API. I believe it would
           | be really hard to implement and keep following changes in
           | Linux.
           | 
           | An idea in my mind is to use Kerla for virtualized
           | environments like KVM where only limited device drivers are
           | needed (virtio for example).
        
             | phkahler wrote:
             | >> The most challenging point is, as others said, the lack
             | of the compatibility with Linux Driver API.
             | 
             | IMHO getting a minimal set that works is good enough to get
             | people on board. This is still nontrivial. But for example,
             | getting FUSE working would be useful. Even if the driver
             | itself was not ABI compatible, it would bring functionality
             | and someone might then aim for ABI compatibility afterward
             | which would open even more doors.
        
             | yjftsjthsd-h wrote:
             | > The most challenging point is, as others said, the lack
             | of the compatibility with Linux Driver API. I believe it
             | would be really hard to implement and keep following
             | changes in Linux.
             | 
             | Especially since, as I understand it, even _Linux_ isn 't
             | compatible with the Linux driver API across versions; they
             | can and will change internals at will and just update in-
             | tree drivers to match. Hence some of the difficulty doing
             | things like getting a newer kernel on assorted embedded
             | devices (ex. 99% of phones) because you have to port the
             | vendor's drivers to the new version and both sides changed
             | stuff.
        
         | allochthon wrote:
         | I'm new to Rust and am far less along than you are, as
         | evidenced by this project. I noticed that in the few files I
         | spot checked, there aren't many/any tests. Without any context
         | or opinion, I'm curious whether unit and integration testing
         | are hard for a project like this.
        
       | jll29 wrote:
       | Kudos, it works and thee source code is a beautiful read.
       | 
       | Thanks so much for sharing!
        
       | ativzzz wrote:
       | What are some ways I can increase my knowledge in this domain
       | that the OP is very skilled at, meaning low level OS development?
       | 
       | I've taken an intro to OS class and am currently going through
       | Linux From Scratch [1], which is interesting and is teaching me a
       | lot, but it's more about how to setup a Linux distro using
       | existing packages and not really about reading/writing the code
       | involved.
       | 
       | Any recommendations?
       | 
       | [1] https://www.linuxfromscratch.org/
        
         | 0x0203 wrote:
         | One of the easiest ways to get into writing your own kernel is
         | to start with the OS Dev wiki [0]. Both the wiki and their
         | forums are a great place for getting your feet wet.
         | 
         | For more general information, there are also a handful of
         | articles and/or blog posts that I've come across when I was
         | getting started that gave a very good introduction to many of
         | the low-level components of how computers work and what the OS
         | needs to do to run them. One of my favorites was Many But
         | Finite [1]. Check out a bunch of his posts about the memory
         | map, the computer boot process [2], cpu rings, etc... Not
         | exhaustive by any means, but good, well-presented info. Many
         | other such sources exist; maybe others will share their
         | favorites.
         | 
         | And if you'd just like to get a more gentle introduction into
         | actual OS code than the monster that is Linux, maybe try
         | looking at some of the BSD's. NetBSD [3] and OpenBSD [4] are
         | remarkably easy to navigate and follow from a code flow
         | perspective, and if you just want to see how things work in a
         | real OS, it's both a good reference and a good place to start.
         | 
         | [0] https://wiki.osdev.org/Main_Page
         | 
         | [1] https://manybutfinite.com/archives
         | 
         | [2] https://manybutfinite.com/post/how-computers-boot-up
         | 
         | [3] http://netbsd.org/
         | 
         | [4] https://www.openbsd.org/
        
           | ativzzz wrote:
           | Thank you!
        
       | krylon wrote:
       | This may be a naive question, but I cannot help but wonder: As C
       | and Rust aim to be link-compatible, wouldn't it be easier to
       | gradually replace parts of the Linux kernel with Rust code?
       | 
       | The only technical problem I can think of is that Rust may not be
       | available for all CPU architectures Linux supports, but this is
       | just speculation on my part having done no research on the
       | matter.
        
         | fork-bomber wrote:
         | The Rust for Linux kernel project aims to enable writing Linux
         | kernel device drivers in Rust.
         | 
         | See: https://lwn.net/Articles/862018/
         | 
         | The issue of Rust's LLVM based compiler toolchain not targeting
         | all CPU archs is intended to be solved by the gcc-rs project.
         | 
         | See: https://lwn.net/Articles/871283/
         | 
         | I think the approach the Rust for Linux project is taking is
         | wise: Not about outright re-writes but more about focusing on
         | those subsystems where Rust's intrinsic safety and security
         | properties helps the most.
        
           | maleldil wrote:
           | Last I checked, one of the big problems was the Rust panics
           | when you run out of memory, which is unacceptable in the
           | kernel. Is there any progress on that?
        
             | geofft wrote:
             | Yes, this is called "fallible allocations." You add methods
             | with the "try_" prefix that work like the existing methods,
             | except they return a Result which fails if it's out of
             | memory instead of panicking.
             | 
             | We have a light / temporary fork of the Rust stdlib
             | allocator with fallible allocation support:
             | https://github.com/Rust-for-
             | Linux/linux/tree/rust/rust/alloc
             | 
             | See e.g. https://github.com/Rust-for-
             | Linux/linux/commit/487d7578bd036...
        
             | phkahler wrote:
             | >> one of the big problems was the Rust panics when you run
             | out of memory, which is unacceptable in the kernel.
             | 
             | If the kernel is running out of memory, IMHO that's a bug.
             | The kernel is ultimately responsible for memory management
             | right? It needs to prioritize itself over everything else
             | or the system is in trouble.
        
               | esclerofilo wrote:
               | This sounds like a design choice that was made decades
               | ago and permeated everything else in Linux. I don't think
               | it's changeable at this point, if they wanted to.
        
             | jerven wrote:
             | Yes a lot. https://rust-for-
             | linux.github.io/docs/alloc/alloc/index.html
        
             | tialaramex wrote:
             | Out of the box Rust doesn't provide any way to allocate
             | heap memory, so, you can't run out if it.
             | 
             | Your ordinary userspace apps use std (the standard library)
             | which relies on the alloc crate, and _that_ provides heap
             | allocation which indeed panics if the allocation fails.
             | Because it correctly guesses that your  "clever" strategy
             | to handle allocation failure actually isn't and will just
             | triple fault anyway so it should cut to the chase.
             | 
             | The kernel obviously doesn't have std, and Rust for Linux
             | implements its own alloc crate.
             | 
             | Linus' requirement that you can fail memory allocation just
             | means all the calls in alloc that can actually allocate
             | memory now return Result to indicate whether the allocation
             | was successful. This isn't how you'd do things in
             | userspace, but, this isn't userspace so fine.
        
               | abendstolz wrote:
               | Can you elaborate as to why "his isn't how you'd do
               | things in userspace, but, this isn't userspace so fine"
               | holds?
               | 
               | Naive me - not a kernel dev at all - would argue that
               | returning Result<Memory, AllocationError> is always
               | better, even for userspace because it would allow me to
               | additionally log something or gracefully deal with this.
               | 
               | Even if I don't want to deal with it, I could just
               | `.unwrap()` or `.expect('my error message')` it.
               | 
               | Note: I am not trying to be snarky here, I genuinely
               | don't know and would like to.
               | 
               | If answering this is too complex, maybe you can point me
               | in the right direction so I can ask the right questions
               | to find answers myself? Thanks in any case!
        
               | cormacrelf wrote:
               | Tialaramex answered this in their post already, and you
               | almost answered the question yourself:
               | 
               | > _I could just .unwrap() or .expect( 'my error message')
               | it._
               | 
               | Panicking can allocate. Allocating can fail. Failing can
               | panic. Panicking can allocate. Allocating can fail. You
               | can bite yourself in the ass like a real Ourobouros.
               | 
               | IMO, a prerequisite to using fallible allocation APIs
               | should be attempting to write your own allocator,
               | handling the weird and wacky problem of initialising a
               | data structure (for the heap) in such a way that if it
               | fails, it fails without allocating but leaves some hint
               | as to what went wrong.
        
               | abendstolz wrote:
               | Oh, wow, I was under the impression that the error
               | message would be stack only, no heap involved, but as
               | Result is part of the std library and not of core, this
               | totally makes sense.
               | 
               | So for `Rust for Linux` they also need to implement a
               | `Result-like` type that is stack only based to solve this
               | issue, right?
               | 
               | If so, cool, thanks, you just made my day by tickling my
               | learning nerves! :)
        
               | bluejekyll wrote:
               | Result already is "stack" based, or sized. It also
               | already exists in core: https://doc.rust-
               | lang.org/core/result/index.html
               | 
               | The Error type currently isn't in core, but for other
               | reasons, that just got resolved: https://twitter.com/basc
               | ule/status/1452029363197784071?s=21
        
               | cormacrelf wrote:
               | It has nothing to do with Result, whatsoever. Result does
               | not allocate. If you used a Result that way, you could
               | certainly try to "gracefully" handle the allocation
               | failure, but if you think it would be easy, you would be
               | wrong. As Tialaramex said, you are probably just going to
               | make the problem worse because it is very difficult to
               | ensure you do not attempt to allocate during allocation-
               | failure-recovery. Rustc doesn't and can't really check
               | this for you.
               | 
               | It actually has to do with `panic!(...)`. When you use
               | `unwrap()`/`expect("...")`, you use the panic macro under
               | the hood; parts of the panicking infrastructure use a
               | boxed trait object which could contain a static string or
               | formatted String or anything else really. The box can
               | allocate if it is not a ZST. I believe the alloc crate's
               | default handler tries to avoid this kind of thing, so
               | that it can't fail to allocate AGAIN in the failure-
               | handling routine. It will likely do a better job than you
               | could.
               | 
               | This is a live issue at the moment, so to go into any
               | more detail I'd have to read a bunch of recent Rust
               | issues/PRs.
        
               | cormacrelf wrote:
               | An addendum to tie this back to the original discussion:
               | the reason kernel devs want these APIs more than userland
               | is that (a) in a kernel, panicking = crashing the
               | computer, which would be bad, and (b) they have a _much_
               | bigger toolbox for handling OOM.
               | 
               | They can kill entire misbehaving processes. What are you
               | going to do in your little program, clear a cache whose
               | objects are sprinkled evenly across 150 different pages?
               | You would need more control than you get from blindly
               | using malloc/free/rust_alloc globally. Something like
               | memcached would be able to use these APIs, because it
               | uses its own allocator, and knows enough about its layout
               | to predictably free entire pages at once.
        
               | bluejekyll wrote:
               | > panicking = crashing the computer
               | 
               | That isn't very accurate. In Rust when programming in
               | no_std, you can (must?) define your own panic handler:
               | 
               | https://doc.rust-lang.org/nomicon/panic-handler.html
               | 
               | Which you would define in the kernel. While I'm not going
               | to speculate on exactly what the implementation would
               | look like, you definitely do not need to "crash" the
               | computer. I haven't done any kernel programming, but I'm
               | guessing the kernel could do some things at that point
               | with shared memory space that is already allocated to
               | deal with this situation and try to recover in some way.
               | 
               | Edit: for example, I just found this in the kerla
               | project: https://github.com/nuta/kerla/blob/88fd40823852a
               | 63bd639e602b...
               | 
               | That halts now, but it probably doesn't need to, or could
               | do it conditionally based on the contents of PanicInfo.
        
               | abendstolz wrote:
               | Thanks for the thorough explanation!
        
               | tialaramex wrote:
               | > it would allow me to additionally log something
               | 
               | If you don't have any memory your allocations are _all_
               | failing. When you assemble the log message, the
               | allocation needed to do that fails. Bang, double fault.
               | 
               | Now, often people don't really mean they want allocations
               | to be able to fail generally, they're just thinking about
               | that code they wrote that reads an entire file into RAM.
               | If it was a 100GB file that would be a bad idea. But the
               | best answer is: Guard the allocation you're actually
               | worried about, don't ladle this into the fast path
               | everybody has to deal with on every allocation.
        
               | abendstolz wrote:
               | Mhm, thanks.
               | 
               | It never occurred to me (being in non-embedded land) that
               | returning an enum as the error or a &'static str instead
               | of a heap structure like String, could also fail.
               | 
               | Seeing that Result isn't part of core, but of std, this
               | makes sense.
               | 
               | Just to tickle my nerve though: theoretically speaking,
               | with your example, it would work, right?
               | 
               | I couldn't allocate 100GB (because OOM or not even enough
               | RAM to begin with) but it could be that the system can
               | allocate the needed memory for error message just fine.
               | 
               | Very interesting.
        
               | schuyler2d wrote:
               | Result is part of core [0]. Result data and/or errors can
               | be stack-only data. The parent was just saying that many
               | people that say they want to guard against out-of-memory
               | issues aren't cognizant of just how difficult that is.
               | 
               | Add to that that several operating systems will lie about
               | whether you're out of memory, so the 'error' or failure
               | will often not be on the Result() value but come in a
               | SIGKILL instead, it's just adding complexity.
               | 
               | People that are actually worried about it and no how to
               | deal with it, will be coding with a different style and
               | can use the alloc library where/when they need to. (at
               | least when it gets stabilized in Rust)
               | 
               | [0] https://doc.rust-lang.org/core/result/
        
               | abendstolz wrote:
               | Thanks for correcting my error.
               | 
               | I've never checked core before, so I did when checking up
               | for this discussion.
               | 
               | I somehow missed Result. Silly me didn't search on that
               | page, but ofc I found it on std
               | 
               | https://doc.rust-lang.org/std/result/index.html
               | 
               | Also thanks for clarifying that values of Result can be
               | stack-only!
        
               | rtpg wrote:
               | People say that "well if allocations fail all bets are
               | off" but can't you pre-allocate memory for error
               | handling?
               | 
               | Like sit down, figure out all the things you'll want to
               | do on an allocation failure, and once you have determined
               | that you slice a little chunk of memory when you start
               | your app (and maybe _that_ fails and you can't do
               | anything). and when you hit a failure you do your think,
               | then tear stuff down.
        
               | ensiferum wrote:
               | What if the failed allocation came from a different
               | allocator/heap than from where the allocations for string
               | logging came from?
               | 
               | In general Don't assume anything about your global
               | process state just because one allocator fails.
        
               | skitter wrote:
               | AFAIK one other thing to note is that in Linux userspace,
               | malloc (or fork) might succeed, but accessing the memory
               | later can fail because of memory overcommit.
        
         | gspr wrote:
         | A practical problem is that in order to gain the benefit of
         | linking Rust and C code, one has to give up Rust's wonderful
         | guarantees at the interface. So until islands of Rust meet up,
         | these interact through a C ABI, and have to expose C-like
         | behavior.
        
           | jll29 wrote:
           | The world really needs _two_ projects, a practical one that
           | is interoperable with the here and now, to get incremental
           | improvements in security from bits reimplemented in Rust.
           | 
           | And then, in the longer term, someone ought to write a book
           | about operating system implementation in Rust that ignores
           | Linux interoperability and focuses on readability,
           | maintainability by showing use of Rust's idiomatic style.
        
           | ls65536 wrote:
           | But you still get those guarantees and benefits in the Rust
           | implementation itself, even if not at the interface (to non-
           | Rust code). By a similar argument, a standalone Rust program
           | "gives up" its guarantees whenever the process makes a call
           | to libc or to the operating system (syscalls), but this isn't
           | really a practical problem.
        
             | gspr wrote:
             | Sure, I agree. The point is that if the total corpus of
             | Rust code is too isolated into too small islands, then the
             | boundary effects of each island may eat up a lot of the
             | benefit.
             | 
             | I still think the project is really cool and worthwhile.
        
         | phendrenad2 wrote:
         | The Linux kernel is massive, so writing any part of it in Rust
         | is an enormous undertaking. Even a few functions at a time,
         | we're talking decades.
        
           | krylon wrote:
           | Well, writing a kernel from scratch that supports as many CPU
           | architectures and devices as Linux would take about as long,
           | assuming one could attract a critical mass of developers.
           | 
           | Using an incremental approach at least gives us _some_
           | benefits in the near future.
           | 
           | I'm not even saying a fresh start would be a bad idea. But
           | incrementally replacing parts of Linux seems like a more
           | promising approach, IMHO.
        
         | pornel wrote:
         | Yes, gradual replacement is easier and more practical for large
         | projects. However, typical C APIs and idioms are different from
         | idiomatic Rust. Rust uses more type-system features, generics,
         | iterators, RAII, and prefers tree-like data structures and far
         | fewer pointers (no linked lists!).
         | 
         | C rewritten in Rust is still very C-like, and requires
         | refactorings that you can't do until it's all Rust.
         | 
         | BTW: There are two GCC-based Rust implementations in the works,
         | so compatibility with exotic platforms is going to be solved.
        
         | wongarsu wrote:
         | That's what may happen with the Linux kernel over time, now
         | that it allows parts written in Rust.
         | 
         | The limitation of that approach is that all internal interfaces
         | (which includes most data structures) have to be C-compatible.
         | You can get much of the safety benefits of Rust, but lose a lot
         | of Rust's ergonomics.
        
           | geewee wrote:
           | I don't think that the Linux kernel has decided (yet) to
           | allow rust code into the kernel proper, has it?
        
             | fork-bomber wrote:
             | It hasn't. But the attitude towards using Rust for device
             | driver development has now gotten very positive, especially
             | post the recent kernel summit.
        
             | pjmlp wrote:
             | Google has decided to go forward anyway.
             | 
             | Just like Android Linux compiles just fine with clang for
             | the last five years or so, it now makes use of Rust.
             | 
             | https://source.android.com/setup/build/rust/building-rust-
             | mo...
             | 
             | If upstream ever cares to support clang or Rust, that is
             | another matter.
        
               | jstarks wrote:
               | Unless I'm missing something, that link refers to
               | building Android user-mode components in Rust, not kernel
               | components in Rust.
        
               | steveklabnik wrote:
               | That's correct, but it came after them rebuilding the
               | bluetooth stack
               | https://news.ycombinator.com/item?id=26647981
        
             | ericbarrett wrote:
             | An LWN article about Rust in Linux that's very relevant to
             | this discussion: https://lwn.net/Articles/829858/
        
       ___________________________________________________________________
       (page generated 2021-10-25 23:00 UTC)