[HN Gopher] Mitigating Memory Safety Issues in Open Source Software
___________________________________________________________________
Mitigating Memory Safety Issues in Open Source Software
Author : littlestymaar
Score : 338 points
Date : 2021-02-18 12:28 UTC (10 hours ago)
(HTM) web link (security.googleblog.com)
(TXT) w3m dump (security.googleblog.com)
| WalterBright wrote:
| Most memory safety issues are buffer overflows, which is easily
| solved in D by using pointer/length pairs as a builtin type.
| Radim wrote:
| For what it's worth, I reached out to the Open Source Security
| Foundation (OpenSSF) after their press release on "Securing
| Critical Open Source Projects" [0], because my open source
| project is on their "critical list" (Python top #200) [1].
|
| After some clicking around, I was directed to the OpenSSF mailing
| list [2].
|
| ...where my request was left to rot, without any response.
|
| Made me wonder whether this is just a PR stunt / corporate power
| grab.
|
| [0] https://opensource.googleblog.com/2020/12/finding-
| critical-o...
|
| [1] https://news.ycombinator.com/item?id=25381397
|
| [2] https://groups.google.com/g/wg-securing-critical-projects
| giantandroids wrote:
| > ...where my request was left to rot, without any response.
|
| Hey, I am involved in the OpenSSF and happy to take at least a
| look. I maintain a popular python OSS project in the security
| area so we have that in common. Did your email actually go
| through, I just searched my list archives and there is nothing
| for gensim. EDIT: see your email now, have msg'ed the WG in
| slack.
|
| > Made me wonder whether this is just a PR stunt / corporate
| power grab
|
| I would not say it is, we have folks from OWASP on the
| technical advisory panel and many other non profit orgs /
| individual developers. One thing I would keep in mind, is that
| being a newly founded org, a lot of time has been spent working
| out and agreeing how we will operate, so the working groups are
| all still quite fresh.
| ZebusJesus wrote:
| I think it is, the maintainers of curl even talked about how C
| is a secure language and how curl is on almost every system
| possible which is why it has flaws, not because C is insecure.
| Debugging your code on almost every hardware out there is not
| possible which is why it is open source and why they intended
| you to adapt it not just run it
| kimsterv wrote:
| Unfortunately, the openssf members haven't come to consensus on
| the process for handling funding requests and the process.
| We'll get there (hopefully!) but it's looking like it's going
| to take more time. -Sincerely, Google's OpenSSF governing board
| rep
| twic wrote:
| I couldn't find "Gensim" in the Python top 200 list:
|
| https://www.googleapis.com/download/storage/v1/b/ossf-critic...
|
| In the list of all projects, it is 2909th.
|
| https://www.googleapis.com/download/storage/v1/b/ossf-critic...
|
| You posted in December that it was #119 for Python. That
| suggests to me that this list might be quite volatile, which
| calls into question its usefulness.
|
| Regardless of the actual position in the list, your email
| should have got a response!
| notpygame wrote:
| The security information about C and curl is a bit outdated in
| the post, and recent research shows Rust does not solve the
| memory safety issue.
|
| The "recent study" quoted in the article was published at the
| beginning of 2019, using older data.
|
| Current vulnerability data shows that curl has very much limited
| the risk of memory safety issues. How many reported security
| vulnerabilities in the last two releases of curl? Zero so far.
| You have to go back 9 months before you find one expired-pointer
| derefence issue resulting in potential data being sent to the
| wrong connection in rare circumstances and configurations. Which
| is a logic error that could happen in Rust too.
|
| To quote from a Oct 2020 study on Rust safety - "Our study
| results reveal that while Rust successfully limits the risks of
| memory-safety issues in the realm of unsafe code, it also
| introduces some side effects. In particular, most of the use-
| after-free and double-free bugs in our data set are related to
| the automatic drop scheme associated with the ownership-based
| memory management model." -- "Memory-Safety Challenge Considered
| Solved? An In-Depth Study with All Rust CVEs"
|
| They study 60 Rust memory safety vulnerabilities.
|
| As you can see not only does Rust not solve the memory safety
| problem, it has other issues. Additionally the old research
| quoted misleads people about the current status of reported
| memory safety issues in curl.
| duckerude wrote:
| Rust does not solve the memory safety issue. It does mitigate
| it, and the post is about mitigation. That study finds that
| Rust "successfully limits memory-safety risks to the realm of
| unsafe code".
|
| It also finds that Rust has novel patterns of unsafety in
| unsafe code. That's important! But it's fully compatible with
| the claim that Rust is much safer than C overall.
|
| I don't think it ever says that the sum of safety in safe code
| and novel unsafety in unsafe code adds up to as much unsafety
| as C. The paper's overarching claims aren't quantitative.
|
| I think you're overstating the claims of both the blog post and
| the study.
| notpygame wrote:
| The recent track record of curl shows it has zero reported
| memory safety issues recently. Reading the article and the
| old linked research you'd be mislead.
|
| It also states that Rust completely prevents them - it does
| not. The article talks about mitigation, but also says:
| "would have been completely prevented by using a memory-safe
| language". The "completely prevented" claim in the article is
| the one not supported by current research. If you only read
| this article, you'd be mislead about memory safety in Rust.
| duckerude wrote:
| > The recent track record of curl shows it has zero
| reported memory safety issues recently.
|
| Only if you look very recently. Earlier you said:
|
| > You have to go back 9 months before you find one expired-
| pointer derefence issue resulting in potential data being
| sent to the wrong connection in rare circumstances and
| configurations. Which is a logic error that could happen in
| Rust too.
|
| That bug is 6 months old and could not happen in safe Rust
| because references (pointers) cannot outlive their
| referents.
|
| > It also states that Rust completely prevents them - it
| does not. The article talks about mitigation, but also
| says: "would have been completely prevented by using a
| memory-safe language".
|
| It is literally true that they "would have been completely
| prevented by using a memory-safe language." The
| complication is that Rust is only memory-safe if you don't
| use unsafe.
|
| rustls (the new curl component in question) uses no unsafe
| itself. I find some unsafe in its dependencies, but most of
| it seems to be for FFI, which is inherently unsafe. I'm
| also not sure that those should count--do OpenSSL
| vulnerabilities count as libcurl vulnerabilities?
|
| > The "completely prevented" claim in the article is the
| one not supported by current research.
|
| I will grant you that it's slightly misleading because it's
| possible to write unsafe Rust code. But that's not news,
| "current research" has nothing to do with it.
| damnyou wrote:
| No memory safe language in existence can meet your
| standards, since they're all written on a bedrock of unsafe
| code.
| damnyou wrote:
| _Safe_ Rust solves the problem of memory safety (assuming that
| any unsafe code is correct). _Unsafe_ Rust continues to be
| potentially memory unsafe. That is equivalent to other memory
| safe languages like Java (which is memory safe under the
| assumption that the JVM has no memory safety bugs and any JNI
| code is correct), and a very large improvement over C and C++.
| tediousdemise wrote:
| Is there a reason why some developers prefer Rust over C++? I
| often hear that it's because of Cargo, but in my experience,
| universal build systems and package managers end up not really
| solving the problem they are designed to solve, and are a huge
| attack vector. Anyone who's tried to deploy their own complex
| Maven project knows that it's a true nightmare.
|
| C++, on the other hand, has a variety of choices for build
| systems. CMake is prevalent, you can use simple makefiles just
| like C, or you could use countless other custom solutions. I've
| found that languages which support a singular build system are
| really easy to build simple demo apps, but once you need to build
| something that the default build system doesn't support, you're
| in big trouble.
| rcxdude wrote:
| C++'s build systems are miserable. Almost any other language is
| preferable (yes, even a complex maven project). CMake and co
| are just as susceptable to issues when you want to do something
| off the beaten path, and have a few landmines on the beaten
| path just to spice it up. Cargo is a breath of fresh air in
| comparison (and as package managers go it's very good: they
| took some care to learn from the mistakes of others from the
| get-go). And rust isn't wedded to cargo: you can build your
| code using make if you wish. In fact rustc doesn't use cargo.
| ragnese wrote:
| Cargo is the least of reasons I prefer Rust. Rust APIs are much
| more ergonomic than C++'s, its type system is better (built-in
| ADTs and type classes, no null), it's obviously safer by
| default, it's expression-based which is just sooo ergonomic,
| its move semantics are enforced by the compiler, and it just
| doesn't have all the sharp edges of C++ (returning dangling
| references, implicit constructors, 57 different kinds of
| lvalue, rvalue, xvalue, lrxhvalue, whatever), etc.
| rational_indian wrote:
| >Anyone who's tried to deploy their own complex Maven project
| knows that it's a true nightmare.
|
| Maven != Cargo
|
| > C++, on the other hand, has a variety of choices for build
| systems.
|
| None of which even remotely approach the ease of use and
| maintainability of the Cargo build system.
|
| > once you need to build something that the default build
| system doesn't support, you're in big trouble.
|
| Cargo can build arbitrary dependencies using the build.rs build
| mechanism.
| pornel wrote:
| Variety of choices for C++ end up causing fragmentation, and
| all choices are flawed in some way. If you choose Cmake,
| someone will tell you how much it sucks, and you should have
| used Meson. You choose Meson, people will complain that it
| needs Python, or Bazel needs Java, or why bother with any of
| them if Makefile is good enough, and so on in circles forever.
|
| OTOH Cargo is the only option for Rust projects. There's
| nothing to bikeshed about. Everyone has it and knows how to use
| it.
|
| Apart from that the project is about memory safety. Companies
| like Google, Microsoft, and Mozilla already have huge C++
| codebases, top engineers with lots of C++ experience, spent
| tons of effort on making it safe, and still weren't happy with
| the results.
| Arch-TK wrote:
| At this point if you stick to a limited subset of C++ and don't
| go too deep you can get a perfectly safe language which is
| powerful and performant.
|
| If you want a safer language with in my opinion a far better
| type system than anything Rust could possibly achieve at this
| point you can use Ada.
|
| But why not those two options? Old, smelly and most
| importantly, difficult to deal with dependencies. You have to
| actually think about how your API will work in your library
| because if your library ever ends up used anywhere people are
| going to be running different versions and you don't want to
| ruin their day by having them change their code every time they
| update their machine.
|
| In comes cargo (and friends, lots of other languages have
| picked up the same idea) where everything is managed by the
| programming language's choice of build system and API stability
| and dealing with dependency hell no longer exists when you
| statically link everything.
|
| I think people don't care or want to care about why we got to
| where we are (distributions with dependencies and API stability
| and an aversion to micro-modules). A lot of people just want
| things to work when you run one command no matter how much
| complexity gets hidden under the carpet, how obscure the issues
| end up being and how windows-esque the debugging process ends
| up being when things do go wrong.
|
| All this pushed by lots of big companies who don't care about
| the origins of the open source model and how distributions do
| things and instead only care about lining their own pockets.
|
| Mark my words, distributions will disappear in a few year's
| time, people will cheer this on as some great technological
| advancement, things like snap will become the de-facto standard
| and debugging a complex problem will just never happen again.
| The internet will become filled with forum posts of how people
| "fixed" a problem by rebooting their machine or deleting a
| random file or re-installing.
|
| Anyway, maybe I'm just a crazy person thinking up crazy
| scenarios and being irrationally scared of "progress".
| c-cube wrote:
| Is Ada memory safe in the presence of malloc/free?
| lovasoa wrote:
| I like cargo because I don't like having to even think about
| build systems.
|
| I often renounce to contribute to C/C++ projects because I know
| I will spend more time figuring out how to install all of their
| dependencies than making my two-line fix.
|
| On the contrary, in the rust ecosystem, I know I can just make
| my two-line fix, run cargo test, and if there is no red line in
| the output, I'm done.
|
| > I often hear that it's because of Cargo
|
| I don't think the main reason is cargo. The main reason is that
| you don't have to worry that the very simple pointer
| dereference you just wrote may allow anyone on the internet to
| take full control of the machine of anyone that runs your
| software.
| hahahahe wrote:
| The more I look at Rust the more I realize it's going to become
| the next standard language.
| arcturus17 wrote:
| You mean standard for systems programming?
|
| I don't doubt that it will be a popular choice in that area,
| but even there it has competition from new languages.
|
| And above all, I think it might take longer than a century to
| phase out all the C/C++ legacy code!
| ragnese wrote:
| It'll be interesting to see how it plays out.
|
| I could absolutely see C++ devs jumping ship to Rust. I think
| const generics just stabilized or is about to. That was one
| of the first shortcomings I noticed when I learned Rust. I
| think custom allocators is pretty much the next big piece
| that would give C++ devs pause today.
|
| But I don't see Rust picking up a lot of C devs. Rust is
| significantly more complex than C. You certainly don't _have
| to_ use traits and Futures, etc, but you also don 't _have
| to_ use a bunch of features of C++ either and C devs still
| don 't jump to C++.
|
| Zig certainly gets a lot of buzz here, but I haven't
| personally looked into it. It does certainly _sound_ like it
| 's designed to appeal to the C crowd.
| cameronhowe wrote:
| I know I'm only one person, but I'm a C programmer at my
| day job and I do all my hobby projects in Rust.
|
| For me cargo, rust-analyzer, and lifetimes/borrow checker
| just makes programming so much more pleasant.
|
| edit: I dont go overboard on the more advanced features in
| the rust language, and probably write more C idiomatic code
| sometimes, but I still prefer using rust over C
| ragnese wrote:
| For sure. I didn't mean to suggest that nobody from C
| land would enjoy Rust. Just that I think the _trend_ will
| be that more C++ people move to Rust than C people.
|
| But my point of view is quite biased. I am still baffled
| that people would start new projects in C instead of just
| using a strict subset of C++, at least: classes, only
| trivial templates for generics, references, _maybe_ smart
| pointers depending on what you 're doing, but no STL, no
| exceptions, no inheritance, etc.
| oscargrouch wrote:
| > The ISRG's approach of working directly with maintainers to
| support REWRITING tools and libraries incrementally falls
| directly in line with our perspective here at Google.
|
| So, its not enough that anybody that don't want to rewrite
| millions of lines of code in Rust because its just pure insanity,
| need to be constantly harassed and forced to feel ashamed (or
| feel even feel guilt) because "how in the world you did not did
| this code in Rust"
|
| Now there's a whole INSTITUTION to funnel the scarce resources,
| not by fixing the security bugs but by forcing others to Rewrite
| Everything In Rust(r) or else they will have their popular
| projects hijacked by highly paid engineers from companies with
| unlimited financial resources to rewrite the whole thing in Rust
| leaving the maintainers of the original project eating dust if
| the don't conceive?
|
| If this is not going to support the source code maintainers to
| patch the bugs in whatever technology they are using, this is the
| Rust bullying that used to be just annoying social dynamics,
| being take into another level.
|
| Daniel Stenberg and libcurl will be exemplary nailed so others
| feel more inclined to follow or else see their projects being
| taken out of their hands if they do not consent.
|
| Imagine if you are a open source maintainer with scarce
| resources, being approached by an institution that promise you to
| give you some funds, but only if you show a plan to rewrite the
| whole thing in Rust?
|
| Edit: Just follow the Stenberg tweeter if you are curious
| https://twitter.com/bagder to see all the pressure this guy have
| being taken lately.
| trynton wrote:
| "bugs would have been completely prevented by using a memory-safe
| language"
|
| Relying on safe languages isn't going to work as the defect
| resides in the Memory Management Unit. You would think by now
| these 'integrated' 'innovators' could come up with a MMU that
| provides total process isolation.
| lovasoa wrote:
| The security issue with memory-unsafe languages is that
| programs can overwrite their own memory, not the memory of
| other processes.
| wtallis wrote:
| Have you confused the kind of memory safety that Rust, etc. are
| about with the Spectre and Meltdown vulnerabilities? Or do you
| think that each individual array that's potentially subject to
| a buffer overflow in C should have been allocated by an
| entirely separate _process_?
| bobcallme wrote:
| The more pressing issue is the fact that critical tools that we
| depend upon are not properly funded or the incentive to properly
| maintain them is simply not there. Many of these projects were
| started as pet projects in peoples spare time. Sure, Google gives
| a lot of money and contributes to some of these projects but
| their contributions do not match how much they depend upon them.
| This is all compounded by the fact that Google is actively
| working to undermine many of these same projects because they
| can't get a license change or projects don't want to bend to
| their will.
|
| Rewriting the world in Rust or other "memory safe" languages is
| not going to be the silver bullet that is being claimed. In the
| case of Rust, I am tired of people evangelizing it when it's
| plagued with issues that undermine its goals and rewrites will
| bring about more nasty bugs. "Unsafe" code will still need to be
| written and not being able to use or have stable toolchains to
| work with is a non-starter for a lot of projects. Until all of
| these languages have set standards and stable toolchains,
| hopefully the Rust Foundation will push, these languages are a
| non-starter.
| kbenson wrote:
| > In the case of Rust, I am tired of people evangelizing it
| when it's plagued with issues that undermine its goals
|
| Do you mind explaining what you mean here? You seem to be
| alluding to something different than an unstable toolchain
| which you mention after.
| [deleted]
| YesThatTom2 wrote:
| Rust? Not Go? But they finally decided to add Generics!
|
| I thought that would fix all the problems, cure cancer, and end
| world hunger!
| andagainagain wrote:
| From the title my first thought was "RUSTLANG!". Then I saw it
| was a google article and thought "well, that's not going to be
| rust-lang at all". I guess the answer was somewhere in between.
|
| It would be nice if there was more meat on the article though,
| like what types of things to do, or how each memory error
| happens, etc.
| kazinator wrote:
| > _Another analysis on security issues in the ubiquitous `curl`
| command line tool showed that 53 out of 95 bugs would have been
| completely prevented by using a memory-safe language._
|
| That's not just a command line tool but a library, libcurl, with
| a command line wrapper. That library is easily used from a myriad
| of memory-safe languages, which is possible in part because it
| doesn't foist highly opinionated memory representations onto the
| caller.
| divs1210 wrote:
| Nowhere does it call out Rust by name.
|
| Go / Zig / Pony / Haskell are all valid contenders.
| MaxBarraclough wrote:
| Pony isn't that well known, here's their home page, for the
| curious. [0]
|
| I remember a recent HN comment said that if you create a
| website for your new programming language, you should always
| ensure there's a meaningful example right there on the landing
| page. A pity that Pony's page fails to do this. Here's one of
| their example programs. [1]
|
| [0] https://www.ponylang.io/
|
| [1]
| https://github.com/ponylang/ponyc/blob/main/examples/timers/...
| nerdponx wrote:
| Also, to my recollection Pony is meant to be somewhat
| specialized for high-concurrency server applications, not for
| general-purpose tools/libraries.
|
| It seems like it's best grouped together with Go, Erlang,
| Elixir, and Crystal.
| nerdponx wrote:
| I suppose D would also fall into this category.
|
| Although I find it interesting that Common Lisp gets left off
| these lists. It's memory-safe (with GC of course), has great
| integration with existing C libraries via CFFI, has a formal
| spec, has a solid library ecosystem, supports a "mostly-
| functional-but-imperative-if-you-need-it" programming style,
| has optional/gradual compile-time type checking, and at least
| one open/libre implementation generates fast native code
| (SBCL).
|
| Might as well throw OCaml in the mix too, since it has all of
| the above minus the formal spec.
| divs1210 wrote:
| Oh yes, SBCL would also apply, I guess!
| nerdponx wrote:
| Edit because I can't edit my post anymore: Kotlin.
| KenanSulayman wrote:
| > The new Rust-based HTTP and TLS backends for curl and now
| this new TLS library for Apache httpd are an important starting
| point in this overall effort.
| simias wrote:
| IIRC Zig isn't memory safe though, is it? If not it seems like
| a bad choice if the explicit purpose is to mitigate memory
| safety issues...
| Zababa wrote:
| I think this is the issue tracking memory safety in Zig:
| https://github.com/ziglang/zig/issues/2301
| paavohtl wrote:
| Has any language successfully added memory safety as a
| "feature" after the initial release? To me memory safety
| seems like a fundamental aspect of a language that is very
| hard if not impossible to change afterwards, without so
| many breaking changes that you essentially in the end have
| created a new programming language.
| dom96 wrote:
| Depends what you consider an "initial release", is the
| point where it is available to the public or the 1.0
| release?
| paavohtl wrote:
| Let's say 1.0. Languages can change significantly between
| the first publicly released build and the first stable
| release.
| jedisct1 wrote:
| This.
| tptacek wrote:
| Not really, no:
|
| * Languages with significant runtimes are going to be
| rejected by project maintainers; nobody is going to rube-
| goldberg a GC into an existing project just to replace a
| risky component when the option not to do that exists.
|
| * Languages with insignificant adoption are going to be
| rejected by project maintainers because why take a flyer on
| an unknown quantity (not just technically but in terms of
| where the community will be 5 years from now) on a project
| you've been maintaining for decades? Say what you will about
| the Rust community and its long-term financial viability but
| the industry is pretty much already committed to it.
|
| I'm fine with Rust but, like, not it's biggest fan (strongly
| prefer Go) but Rust pretty clearly hits the sweet spot being
| targeted here. I think you can assume safely that most of
| this work will be done in Rust.
| benibela wrote:
| Or Ada or Wuffs
|
| But I have always used Pascal for that. It has memory safe
| strings and arrays with refcounting, so long as one avoids
| using other features like pointers, classes, inline assembly,
| it is perfect.
| nerdponx wrote:
| Wuffs? Never heard of that one before.
|
| Edit: here's the link, interesting idea.
| https://github.com/google/wuffs
| benibela wrote:
| And it is supposed to be safer than a language that is only
| memory-safe
|
| The memory-safe languages can still panic or have integer
| overflows, Wuffs should prevent even that
| the_duke wrote:
| Go, Pony and Haskell all have complex runtimes and a GC, which
| makes them unsuitable for embedding into other applications or
| exposing them as widely usable libraries.
|
| Rust, Zig, and to a certain extent D or Nim without GC, can
| easily expose a C API and have a minimal runtime.
|
| But the others do not provide the same guarantees as (safe)
| Rust with the borrow checker. All very cool languages in their
| own right, but memory safety in a no-GC environment is not one
| of their defining attributes.
|
| Note: a dependency on any of those compilers is still far from
| trivial for many projects.
| WalterBright wrote:
| D now has a prototype borrow checker implemented.
| nimmer wrote:
| Nim is not using GC anymore [1], is memory safe and support
| formal methods [2]
|
| It's also introducing isolation [3] to provide safe
| multithreading similar to Rust's sendable.
|
| [1] https://nim-lang.org/blog/2020/10/15/introduction-to-arc-
| orc...
|
| [2] https://nim-lang.org/docs/drnim.html
|
| [3] https://github.com/nim-lang/RFCs/issues/244
| floatboth wrote:
| From [1]:
|
| > ORC is Nim's all-new cycle collector based on ARC. It can
| be considered a full-blown GC since it includes a local
| tracing phase (contrary to most other tracing GCs which do
| global tracing). ORC is what you should use when working
| with Nim's async because it contains cycles that need to be
| dealt with.
|
| Sooooo unless you
|
| > disable the GC and do manual memory management, but then
| you lose access to most of the stdlib
|
| you still have to run a memory-managing runtime
| nimmer wrote:
| No. It's an inlined reference counter, not a runtime with
| dedicated thread and so on. Most people use "GC" to refer
| to the latter.
| whateveracct wrote:
| i just write all my projects in Haskell on any platform so
| the RTS is a given (or the RTS isn't present at execution
| time since the executable is generated from a Haskell eDSL)
|
| it's truly superior to any other programming language (5+
| years of pro experience talking), so why waste my time with
| the manual labor of imperative programming?
| jedisct1 wrote:
| Why not contribute to MesaLock Linux, which is already an entire
| memory-safe Linux distribution instead?
|
| Or encourage other memory safe-languages as well?
| saagarjha wrote:
| Someone probably should look at the kernel, too.
| fock wrote:
| Maybe I am missing something: public-facing systems generally
| don't build on a shell-like environment and I don't think
| memory-safety is a big problem anymore in 20 years-old core-
| utils people use individually on their CLI.
|
| So what's the point of reimplementing one-off-tooling with
| under the same name (like sh?!) in Rust, except adding
| confusion and new bugs? And in a container, you already can run
| only your rust/Go-program, which effectively means that the
| main memory-unsafe attack-surface is actually the kernel (which
| they don't swap out either).
| DrBazza wrote:
| It's buried in there, but it doesn't specifically say Rust:
|
| > That's why we're expanding our collaboration with the Internet
| Security Research Group to support the reimplementation of
| critical open-source software in memory-safe languages.
| KenanSulayman wrote:
| > The new Rust-based HTTP and TLS backends for curl and now
| this new TLS library for Apache httpd are an important starting
| point in this overall effort.
| DrBazza wrote:
| This item had a different title specifically mentioning rust
| which had been changed to remove it, before you made your
| comment
| lrossi wrote:
| What is left out of the title is that they do not provide funding
| to the current authors/maintainers of these open source projects.
| Instead, they fund an organization that will rewrite these tools.
|
| Correct me if I'm wrong, but that's my understanding of the
| rather terse article.
|
| I would rather have the original maintainers funded and still in
| control of the many pieces that form the basis of the major Linux
| distributions.
| gnfargbl wrote:
| The article says that Google is committing funding to the ISRG
| (https://abetterinternet.org). It isn't really clear whether
| that is one-off funding, or the start of an ongoing
| relationship.
|
| The ISRG seems to have two projects of this type mentioned on
| its website: an improvement to curl
| (https://www.abetterinternet.org/post/memory-safe-curl/), in
| which the ISRG funded the curl author directly; and an
| improvement to httpd
| (https://www.abetterinternet.org/post/memory-safe-tls-apache/),
| in which Google (via the ISRG) are funding an httpd committer.
| giantandroids wrote:
| lets encrypt are under the ISRG
| folex wrote:
| > The ISRG's approach of working directly with maintainers to
| support rewriting tools and libraries incrementally falls
| directly in line with our perspective here at Google.
|
| Incrementally - that means in-place. You can't fork and rewrite
| incrementally since there will be no adoption/users of the
| fork.
|
| Also they talk about maintainers of the projects. I think that
| largely means original authors and maintainers, not forks.
|
| I don't see how you come to that conclusion. Could you clarify?
| Thanks :)
| the_duke wrote:
| At least for curl the project leadership is closely involved,
| as evidenced by a related article on Stenbergs blog [1] and the
| ISRG annoucement [2], which mentions funding Stenberg directly
| and is actually linked from this anouncement...
|
| Please don't make statements like that without doing a minimal
| amount of effort to verify.
|
| [1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-
| hyp...
|
| [2] https://www.abetterinternet.org/post/memory-safe-curl/
| lrossi wrote:
| Maybe I got it backwards, but it's a PR post, and the burden
| is on the author to make the point clear. I've read it 3
| times and still can't find any sentence where they state
| clearly what the development strategy is.
|
| And from link #2 that you posted, I see they talk about the
| curl stuff, but also show a bunch of repos with tools
| rewritten from scratch, e.g. https://github.com/ctz/rustls
|
| So I really cannot draw a conclusion one way or the other.
|
| Normally I'm an optimist, but after the recent chromium
| misunderstandings, I am going to err on the cynical side.
| lrossi wrote:
| Replying to myself as I cannot edit anymore.
|
| As others pointed out, it's not clear who will do the rewrites.
| Based on the curl example, it might be the maintainers. It
| would have been great if they could state that concretely.
|
| I'm also wondering why there's a middle man instead of funding
| the projects directly, but that's another story.
| jaas wrote:
| I'm the head of ISRG, the "middle man" entity you're
| referring to.
|
| We have a memory safety initiative in which we plan and fund
| projects to move popular open source software to memory safe
| languages. For each plan we make we seek a funder - in a
| couple of cases now that has been Google, but it may be
| others in the future.
|
| We are in the "middle" of the financing in the sense that
| funding passes through us, but the memory safety initiative
| is coordinated primarily by us. We select projects and work
| with maintainers to make plans. Part of that coordination we
| perform is matching up projects with funders like Google, and
| then managing the contracts and overseeing the work to
| completion.
| lrossi wrote:
| Thank you for your reply.
|
| If I understand correctly, this is an initiative that might
| have multiple sponsors, so it makes sense to have a central
| entity organizing it, and following up with all the
| projects involved. This makes perfect sense to me now.
|
| Also, I was not aware that your org is behind Let's
| Encrypt. This puts things into a very positive light.
|
| It might make sense to post a sibling blog
| post/announcement to explain what this project is about and
| how it would work. I feel that Google's post left a lot of
| questions to be answered.
|
| If you don't mind me asking, is rust the only language that
| you have in mind? Would go work as well?
| flohofwoe wrote:
| Yeah, it sounds a lot like Google is going to "hijack" popular
| open source projects for the sake of "security".
|
| It'll be interesting to watch how this plays out. But I pity
| the projects where Google's gaze falls upon.
| mfer wrote:
| I'm not sure "hijack" is the right word. They are using money
| to entice projects to rewrite in memory safe languages.
| coldtea wrote:
| If they pay for the rewrite, they have control over it.
| flohofwoe wrote:
| If such changes had been planned in the project all along
| and the money injection can help speeding this process up
| that's fine. But I can't see Google security people getting
| involved being a good thing otherwise. They come across as
| prioritising security problems over anything else (e.g. "if
| all you have is a hammer...") and I'd be surprised if this
| sort of extremism will be healthy for a project in the long
| run. Memory safety is just one of many aspects to juggle in
| a software project.
| pyb wrote:
| -
| machello13 wrote:
| Wouldn't feel great, but what in the article gave you the
| idea they're doing that?
| chii wrote:
| Then be satisfied that your project is a good enough idea
| that someone else is willing to put up money to rewrite
| it using a safe language.
|
| But if you're not the one doing the rewrite, why would
| google pay you? And if you would be capable of such a
| rewrite, may be they would pay you to do so.
| sgift wrote:
| Since they don't do that (it's right there in the
| announcement, which no one in this thread seems to have
| bothered to read) that whole point is moot:
|
| > The ISRG's approach of working directly with
| maintainers to support rewriting tools and libraries
| incrementally falls directly in line with our perspective
| here at Google.
|
| And a link to:
| https://www.abetterinternet.org/post/memory-safe-curl/
| piokoch wrote:
| Could be something great, but could end up with some
| "entity" will rewrite, say, ls or cp and suddenly it will
| turn out, that ls is still free and open source, but sends
| all user listed files to that entity (obviously for the
| sake of the noble purpose of making ls even better and all
| those stats are anonymous).
|
| And there will be some pro version that does not grab that
| stats, but you need to purchase subscription.
| lullibrulli3 wrote:
| *to entice projects to rewrite other peoples projects
| Miner49er wrote:
| Your missing what the two parent comments are driving at.
| Google says that's what it is doing, but this money is
| going to "an organization" that will seemingly get it's
| money from Google, giving Google control over whatever open
| source projects they target.
| chii wrote:
| > control over whatever open source projects they target.
|
| but if they rewrite the project in rust, why _shouldn't_
| they control that project? If other people choose to
| switch, then it's not wrong for them to move, and not
| wrong for to google gain control of those users.
|
| The original project maintainers is a third party to this
| whole process, and i would argue, is uninvolved tbh. It
| is up to the users of the project to decide to trust
| google's rewrite (and their tendencies to abandon
| projects...), vs the original maintainer's version,
| balanced against the security aspect.
| cnfhsg wrote:
| If you translate an existing project into Rust, it is a
| derivative work and should retain the original license.
|
| If the rewritten project gets more successful than the
| original (perhaps due to corporate promotion), you have
| morally stolen the work of the original authors.
|
| If you write a project from scratch without looking, this
| of course does not apply. But I doubt that is how Rust
| rewrites actually happen, the temptation is too great.
| TheReverend403 wrote:
| > If you translate an existing project into Rust, it is a
| derivative work and should retain the original license.
|
| By this logic, WINE is a derivative of Windows and should
| retain their license.
| rcxdude wrote:
| WINE goes to some length to avoid any reverse engineering
| of windows components and certainly any glimpse of the
| source code (such as the leaks which have happened over
| the years: if you have seen any of them then WINE does
| not want your code). They do this to avoid any
| accusations of their code being a derivative work. If you
| re-implement an open-source project on the same terms
| then it does not qualify as a derivative work. However if
| you look at the source code and use that to develop your
| new implementation, then that qualifies.
| aaomidi wrote:
| > If the rewritten project gets more successful than the
| original (perhaps due to corporate promotion), you have
| morally stolen the work of the original authors.
|
| Ownership is by "moral" definition something completely
| arbitrary and made up.
|
| If you work in open-source you most likely have different
| views on ownership than others.
| adrianN wrote:
| Now you only need a lawyer army bigger than Google's to
| win your case.
| ardy42 wrote:
| >> control over whatever open source projects they
| target.
|
| > but if they rewrite the project in rust, why
| _shouldn't_ they control that project? If other people
| choose to switch, then it's not wrong for them to move,
| and not wrong for to google gain control of those users.
|
| I think the issue is the more critical projects Google
| controls, the more they can make decisions by fiat. Those
| decisions will favor Google's interests, and could
| sacrifice everyone else's interests in some way.
|
| It's sort of like web browsers. My understanding is that
| Chrome is so dominant that Google can basically dictate
| HTML standards at this point, or at least effectively
| push them in directions that are favorable to its ad-
| based business.
| tptacek wrote:
| You're scare-quoting "an organization", which suggests
| you don't know what the ISRG is. You might look them up.
| They're pretty well-liked.
| emteycz wrote:
| And? Google can't decide to fund organization that
| rewrites open source projects, or what? How is that
| different from forking the projects - apart from Google
| not taking anything at all?
| beckman466 wrote:
| The over-arching pattern is that capitalist firms are
| happy to take from the commons, yet do little to sustain
| it.
| tptacek wrote:
| This is like the archetypical lowbrow dismissal, since
| the ED of ISRG is literally on this thread saying that
| their M.O. is to fund project maintainers directly to get
| this work done. These workers control the means of
| production.
| emteycz wrote:
| They're literally funding an organization that will take
| software to the next level and share it with the world,
| what more do you want?
| bachmeier wrote:
| A rewrite is not taking it "to the next level". It
| provides a new version in a new language that does the
| same things, and if things go according to plan, will
| have fewer safety issues. There's nothing that binds them
| to maintain any of these projects. Given Google's
| history, it's irresponsible to not consider long-term
| maintenance and other issues.
| emteycz wrote:
| No, C and Rust definitely don't do the same things. If
| the project is open source, I don't see how any
| maintenance concerns are relevant - everybody can judge
| for themselves.
| gordian-mind wrote:
| It's not that smart to mistake the purpose or
| specification of a program, with its implementation.
|
| Anyway, if everybody can judge for themselves, then I
| guess it's cool to judge that giving Google an inch over
| many critical open source projects is a horrible idea.
| cycloptic wrote:
| You speak as if it's impossible for someone outside
| Google to maintain a project that isn't written in C++ or
| C...
| emteycz wrote:
| Sure, nobody says you have to use their software. But
| don't claim that they don't give back or that it's
| purposeless.
| nojito wrote:
| Most of those project authors refuse to rewrite or adopt
| memory safe languages in their projects.
| lullibrulli3 wrote:
| No reason no exclude them in the first place.
| nojito wrote:
| Sure, but the internet shouldn't be too upset when a fork
| happens with a version that has feature parity and is
| memory safe.
| mempko wrote:
| Any why shouldn't they?
| encom wrote:
| I don't know, reinventing the wheel with the latest
| buzzword compliant language sounds like a fun time. Think
| of all the exciting new bugs you get to fix!
| jk563 wrote:
| They specifically mention rust based backends for curl where
| the author of curl was funded to integrate as a starting point.
| I don't see any mention of funding being exclusively for
| organisations other than the author(s)?
| turminal wrote:
| Agreed 100%.
|
| Sadly everyone expressing concerns like this will be dismissed
| and everyone is going to once again hail google for being
| community friendly and supportive of FOSS.
| jaas wrote:
| (Executive Director of ISRG here)
|
| > What is left out of the title is that they do not provide
| funding to the current authors/maintainers of these open source
| projects. Instead, they fund an organization that will rewrite
| these tools.
|
| This's not what's happening. Here's what is:
|
| ISRG plans and coordinates investments in moving open source
| software to memory safe languages. We have a strong preference
| for working on these plans with maintainers and developers, and
| then funding them to execute the plan.
|
| Once we have a plan prepared, and, ideally,
| maintainers/developers on board, then we pitch it for funding.
| Once someone (e.g. Google) funds the project, then ISRG handles
| contracts with the project developers/maintainers to get the
| work done. In some cases maintainers may be on board but want
| us to find a contractor to actually do the work.
|
| In both of the first two projects, and in most future cases,
| the money that Google (or another company) provides will
| largely go to project maintainers / developers.
| Cullinet wrote:
| so potentially the result is a complete rewrite not involving
| the original authors in a language the original authors may
| not be current with, by a contractor code publisher /
| producer.
|
| given the potential for security theater and general fud and
| the potency that appeals to authorities such as sponsors like
| Google can create in the minds of the public, how has this
| been evaluated in terms of the possibility for widespread
| disintermediation of the open source software community at
| large?
| lrossi wrote:
| This sounds great, thanks for clarifying!
|
| Also, thanks for Let's Encrypt. I didn't make the connection
| between the two names. This initiative looks much better to
| me now.
|
| Is rust the only language that you are considering? Would go
| make sense as well?
| klelatti wrote:
| The example from the ISRG blog [1] directly contradicts the
| assumption you're making:
|
| > Memory safety vulnerabilities represent one of the biggest
| threats to Internet security. As such, we at ISRG are
| interested in finding ways to make the most heavily relied-upon
| software on the Internet memory safe. Today we're excited to
| announce that we're working with Daniel Stenberg, author of
| ubiquitous curl software, and WolfSSL, to make critical parts
| of the curl codebase memory safe.
|
| > ISRG is funding Daniel to work on adding support for Hyper as
| an HTTP back-end for curl. Hyper is a fast and safe HTTP
| implementation written in Rust.
|
| I too found the post terse and the lack of quantification of
| the financial commitment makes me suspicious but I think they
| deserve the benefit of the doubt for now.
|
| [1] https://www.abetterinternet.org/post/memory-safe-curl/
| adsharma wrote:
| Rewrite vs Transpile
|
| I see references to "fixing" C and rewriting. An alternative
| would be to transpile a memory safe, easy to program language
| such as python3 and generate code in a memory safe language.
|
| Since I don't know which one of these is the best, I have
| implemented basic (as in fibonacci and a simple class) in several
| languages.
|
| I hope more effort is spent on writing code in small statically
| typed languages and generating memory safe code.
|
| https://github.com/adsharma/py2many
| einpoklum wrote:
| Software written by a party with a proven record of surveillance
| of its users in the service of governments, and of selling user
| tracking data en masse to advertisers as its business model -
| cannot, in general, be trusted.
|
| Now, if the codebase is small enough to allow for scrutiny by
| 3rd-parties, that point may be moot, but even then, Google may
| have some ulterior motive which will not immediately become
| apparent.
|
| So, I'm kind of suspicious of such an initiative.
| rurban wrote:
| A worthwhile goal.
|
| But not in a memory unsafe language, please. There's hype and
| there are lies: https://github.com/rust-
| lang/rust/issues?q=is%3Aissue+is%3Ao...
| gpm wrote:
| This is flamebait, ignoring that "nearly perfect detection" and
| "no detection whatsoever" are very different things.
|
| But also, stack overflows aren't memory unsafe in modern rust.
| The issue search you want to find memory safety issues is:
| https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Ao...
| rurban wrote:
| No flame sit. It is either safe or not. Safer doesn't qualify
| as safe.
|
| Your query searches for type unsafeties. A third query would
| be for concurrency unsafety: deadlock. All three unsafeties
| are builtin by design. There's no way to deny it. But they
| still do repeat the same lies again and again.
|
| Stack overflows being memory safe you can tell your grandma
| or upper management. Problem was rust was once memory safe,
| but they ripped out the GC. Now it's just better than C++,
| but not better than Java or really safe languages. Overhype
| rarely reflects well on a community.
| saagarjha wrote:
| This is flamebait, and you know it. Stack overflows in Rust are
| perfectly safe because they hit a guard page.
| [deleted]
| enriquto wrote:
| semi-conspiracy theory: the main goal of this rewriting is to
| have less and less infrastructure depending on GPL and AGPL-
| licensed code
| jsty wrote:
| If all Google wanted was GPL- and AGPL-free code, they're quite
| capable of assigning their own engineers / contractors to work
| on it rather than the more tortuous route of going via a 3rd
| party
| bluGill wrote:
| Their accountants probably prefer this route though. They
| don't get a competitive advantage by have a proprietary
| license, so open source is a good choice, but the GPL has
| issues they want to avoid. By going this route they can use
| tax breaks that are not available for doing the work in
| house. Sure it is more complex, but the results matter, and
| the result they are hoping for is someone else does work that
| they don't pay for.
| konschubert wrote:
| Also it's probably cheaper to fund an OS developer for a
| year than paying for lunch alone for their own engineers.
| jononor wrote:
| What kinds of tax breaks are available for the case of
| using consultants instead of employees?
| bluGill wrote:
| It isn't consultants it is 501 charities doing the work.
| The charity hires the consultants.
|
| There are a lot of tricks needed to make the above work,
| and details that you need to consult a lawyer and
| accountant for.
| enriquto wrote:
| > they're quite capable of assigning their own engineers
|
| but if they develop it for internal usage only, then they
| don't get the added benefit of destroying the existing
| copylefted version!
| rcxdude wrote:
| It will take quite a lot more effort to rewrite something so
| that the (A)GPL doesn't stick. To do that you would need to
| rebuild it from its documentation and/or a clean-room reverse
| engineering, as opposed to a port of the source code.
| userbinator wrote:
| My semi-conspiracy theory: "safe" languages also make it harder
| for things like jailbreaks and other defeats of user-hostile
| "protections", allowing Big Tech to maintain control, so making
| them popular is very much in their interest. It's like how
| every utopian vision for society turns into a dystopian one.
|
| "Insecurity is freedom."
| kam wrote:
| Apache and Curl are not GPL or AGPL to start with.
| corebuffer wrote:
| Sure, but what if the person/team willing to rewrite wants to
| do so under GPL? Or what happens next when the next project
| to be rewritten is GPL? I think its fair to keep the original
| license, but if re-licensing is allowed it should be
| permissive.
| enriquto wrote:
| Of course any such endeavor must have a patina of plausible
| deniability. That's evildoing 101 :)
| corebuffer wrote:
| They removed licensing from their FAQ, but I was able to find
| this issue: https://github.com/ossf/tac/issues/26
|
| I'm also curious if they will accept GPL or not
| mythz wrote:
| Would be nice to see an approved list of "memory safe programming
| languages" as RTFA just mentions a Rust example.
|
| I'm assuming Go will also be an option, what are the others
| options for core OSS system tools?
| roel_v wrote:
| Can someone please fix the title to the original in the blog
| ('Mitigating Memory Safety Issues in Open Source Software'),
| because the current title is not at all representative of what
| the blog is actually saying.
| praptak wrote:
| Also "for rewriting popular systems in Rust" is just false. The
| blog is about improving memory safety and gives two pieces of
| software rewritten in Rust as an example.
|
| There's nothing about this effort being focused on rewriting
| things in Rust.
| wyldfire wrote:
| Are you sure?
|
| > That's why we're expanding our collaboration with the
| Internet Security Research Group to support the
| reimplementation of critical open-source software in memory-
| safe languages.
|
| Though the rest of the article doesn't go into detail about
| this effort beyond http/TLS libs for curl.
| smoyer wrote:
| The quote you provide above doesn't specify that these
| rewrites are to be in Rust and I don't think you've
| contradicted the parent in any way. The examples have been
| in Rust but the foundation doesn't seem to insist on
| rewrites being done in a specific language. As an aside,
| the foundation doesn't seem to be much more than a
| collection of papers at this point.
| jaas wrote:
| Rust is going to be used often, but we are open to other
| choices when they make sense.
|
| Since most unsafe code is C or C++, Rust usually makes the
| most sense for rewriting one component at a time. It
| integrates with C and C++ very cleanly because it has good
| FFI and no runtime.
|
| If we were rewriting more projects from scratch and
| integration with existing C and C++ code was not important,
| we might use other languages more often. I don't expect
| we'll attempt to rewrite many pieces of software from
| scratch.
|
| Source: I run the memory safety initiative at ISRG.
| praptak wrote:
| The "memory-safe languages" may mean Go, Java and a bunch
| of other less popular languages.
| oscargrouch wrote:
| In the end of the day only Rust will be chosen because we
| are talking mostly about system infrastructure.
| nick_ wrote:
| HN is like 15% pro-Rust propaganda these days.
| iwasanewt wrote:
| Have you considered the possibility that people might just
| like it?
| tptacek wrote:
| You're going to see a lot of Rust, because by and large the
| programs ISRG and Google are talking about remain in memory-
| unsafe languages because they have constituencies where
| dynamic-runtime memory-safe languages are problematic. That's
| why security people are so excited about Rust: it's hard to
| come up with modern settings where Rust is unworkable, unlike
| Go (a language I prefer to Rust), which, for instance, isn't
| a good fit for the Linux kernel or for the guts of Chromium.
| saagarjha wrote:
| There's a ton of platforms that Rust doesn't not target, or
| targets but without many guarantees of support. The can be
| problematic as its closest competitors (C++, but especially
| C) generally have much more widespread support.
| tptacek wrote:
| That's a problem, but it's one that's more likely to be
| worked around (most likely: by maintaining unsafe forks
| for those platforms) than "creating entire new
| communities of support for another language" or
| "redesigning whole projects to make GC tenable".
|
| I think you might see some full-system rewrites for
| serverside components where Go will work out. But I think
| the projects we're talking about, like I said, have ended
| up in this position mostly because they can't easily be
| written in Go, or they already would have been (if only
| be a competitor).
| saagarjha wrote:
| Oh, I'm not suggesting that they will use Go-they will
| use C and I'm just responding to you saying that it is
| difficult to find situations where Rust is unworkable.
| tptacek wrote:
| The premise of this story is that ISRG is going to secure
| funding to essentially bribe people (my words, and I
| think it's a very good project) to not keep critical
| software in C anymore.
|
| My guess is what will happen in the cases where Rust is
| unworkable is that there will be less-secure forks for
| the obscure platforms.
| theandrewbailey wrote:
| I'm a native english speaker, and I've never seen the word
| "fundings" in my life.
|
| Edit: the submission title is fixed.
| themarkers wrote:
| Might be reinventing the wheel ;D
| [deleted]
| stephen82 wrote:
| Would not be cheaper to fix those unsafe languages, such as C,
| and offer safe mechanisms that could be used upon need?
|
| Why spend thousands, if not millions of dollars, on rewriting
| existing codebases when they could help fixing the existing
| toolset and make it safer?
|
| I don't get it.
| K0nserv wrote:
| I don't know how well that would work. For example, my
| understanding of Rust's bounding checking for slice indexing is
| that often the compiler can prove it's not needed and the check
| is optimised out because of the type system. This woulnd't work
| in C because of C's lacking type system
| simias wrote:
| To me this idea sounds a lot like "why do we need IPV6? Why not
| keep IPV4 and add a few bytes?" except it's effectively almost
| the same amount of breakage. Either you're C-compatible or
| you're not.
|
| Making C memory-safe by adding extra annotations to allow for
| static zoning/borrow checking is possible, but the churn
| involved is probably not much different than rewriting in a
| different language. See Cyclone for instance if you want to see
| what such an approach looks like.
|
| On top of that modern languages have facilities that are
| seriously lacking from C, memory safe or not. C's error
| handling is atrocious, C doesn't have (standard) destructors, C
| doesn't have type inference, C doesn't have algebraic types, C
| doesn't have proper string/collection support etc...
|
| I like C quite a lot, it's probably the language I know best,
| but it's definitely very outdated. I think in the not so far
| future it'll be almost entirely replaced outside of legacy
| applications, hopefully with Rust. I think the watershed moment
| will be when the Linux kernel is rewritten in an other
| language, which definitely won't happen overnight but at this
| point I'm fairly certain that it _will_ happen within a couple
| of decades.
| stephen82 wrote:
| Your wish might become reality soon
| https://gustedt.wordpress.com/2021/01/12/improve-type-
| generi...
| wmanley wrote:
| > To me this idea sounds a lot like "why do we need IPV6? Why
| not keep IPV4 and add a few bytes?" except it's effectively
| almost the same amount of breakage.
|
| [DJB disagrees](https://cr.yp.to/djbdns/ipv6mess.html).
| Quote:
|
| >The IPv6 designers made a fundamental conceptual mistake:
| they designed the IPv6 address space as an alternative to the
| IPv4 address space, rather than an extension to the IPv4
| address space.
|
| > Either you're C-compatible or you're not.
|
| If we take this black and white approach then we must say
| that C++ is not C compatible just like Ada isn't C
| compatible.
|
| The reality is that C++ being broadly a superset of C was
| integral to its adoption, even though that compatibility
| isn't perfect. It enabled easy incremental porting of
| projects.
|
| This is really just a nit though. I do agree with your
| conclusions.
| simias wrote:
| With all due respect I disagree with DJB here. I can't see
| how his proposal would significantly improve the current
| status quo. Note step one of his self-described
| "straightforward" transition plan (emphasis mine):
|
| >0123456789abcdef0123456789abcdef to 192.5.6.30: The client
| sends a UDP packet to the .com DNS server asking for the
| address of www.google.com. _The client software,
| intermediate computers, and server software have all been
| upgraded to handle the client 's extended address_.
|
| A lot of work in this one "straightforward" step. I know
| that DJB is a great network engineer, so I'm willing to
| consider that I'm missing the point here, but at the very
| least I don't find this particular expose very convincing.
|
| >If we take this black and white approach then we must say
| that C++ is not C compatible just like Ada isn't C
| compatible.
|
| Agreed, in a parent comment I mention C++ being effectively
| mostly compatible with C. But I can't really imagine how
| you could do something like that with a memory-safe subset
| of C. The changes involved are pretty thorough (you
| basically either need a garbage collector or a borrow
| checker). Things like raw pointers (especially nullable
| ones) can't really exist in their current form in a memory
| safe language.
| raverbashing wrote:
| You can't fix what is fundamentally broken.
|
| In the same idea of "Any sufficiently complicated C program
| contains an ad hoc, bug-ridden, slow implementation of half of
| Common Lisp." let's paraphrase
|
| Any sufficiently safe C program contains an ad-hoc (etc.)
| implementation of the infrastructure needed to make it safe:
|
| - custom string implementation
|
| - custom object/memory management
|
| - custom memory slices and other data structures
|
| Not forgetting the compiler tricks added to try and catch when
| the program does something stupid
| carapace wrote:
| FWIW, with my contrarian hat firmly affixed, I suspect that C
| plus tools makes more economic sense than Rust.
|
| https://compcert.org/ > The CompCert project investigates the
| formal verification of realistic compilers usable for critical
| embedded software. Such verified compilers come with a
| mathematical, machine-checked proof that the generated
| executable code behaves exactly as prescribed by the semantics
| of the source program.
|
| https://frama-c.com/ > Frama-C gathers several static and
| dynamic analysis techniques in a single collaborative
| framework. The collaborative approach of Frama-C allows static
| analyzers to build upon the results already computed by other
| analyzers in the framework.
|
| https://www.cprover.org/cbmc/ > CBMC is a Bounded Model Checker
| for C and C++ programs. ... CBMC verifies memory safety (which
| includes array bounds checks and checks for the safe use of
| pointers), checks for exceptions, checks for various variants
| of undefined behavior, and user-specified assertions.
| stephen82 wrote:
| I based my comment on Jens Gustedt's article [1] about adding a
| "defer" mechanism in standard C.
|
| Clearly, the C committee works hard on improving the language
| and they are in the process to standardize important features,
| such as lambda functions, auto, typeof (like C++'s decltype
| more or less), and so forth.
|
| In my humble opinion, "defer" can mild the buffer / memory
| overflow, but of course not permanently remedied it 100%; to me
| though it's better to have such mechanism than have nothing at
| all to protect you.
|
| https://gustedt.wordpress.com/2020/12/14/a-defer-mechanism-f...
| floatboth wrote:
| grrr why are they introducing guard {} blocks? Would not want
| an extra level of {block} just for this.
|
| __attribute__((cleanup)) gcc/clang extension already works
| without extra blocks. GLib has been using it for a while
| https://blogs.gnome.org/desrt/2015/01/30/g_autoptr/
| simias wrote:
| Defer seems more like a solution to making cleanup easier and
| less error prone, preventing memory/fd/mutex leaks. It
| wouldn't significantly improve memory safety IMO. Adding a
| "slice" type with checked bounds (something basically every C
| codebase in existence ends up reimplementing somehow) would
| be a much greater improvement IMO.
|
| But overall even though I still use C almost daily, I'm a bit
| skeptical of the committee's direction with these changes.
| They added half assed generics through macros, now a defer
| mechanism and a bunch of other stuff that's, IMO, out of
| scope for C. People have already tried to make a "better C"
| with all these bells and whistles, it's called C++, and it's
| still mostly backward compatible with C.
|
| IMO C should be considered to be in "maintenance mode" these
| days. And again, my IRL job involves writing a lot of it, so
| it's not like I don't care about it.
| user22 wrote:
| Yup, there is a group of people who think that we should
| add brakes and lcd displays to horses because cars have
| these nifty things. For some reason, they don't understand
| it's ok that horses are not used for everyday travel
| anymore.
|
| I hate the mess that c++ is. It started out with a object
| based programming paradigm with c like syntax. Then
| generics/templates were added. Then we have modern c++.
| It's 3 different languages crammed into one. The c++
| committee has a serious case of nifty-itis.
| rcxdude wrote:
| Defer doesn't even replace destructors, let alone resolve
| memory safety in C.
| diegocg wrote:
| The people "in charge" of C are just not interested in radical
| changes - they have not even bothered to add better strings.
| There was a recent post here describing how the C standard
| bodies even oppose adding warnings, because it would increase
| the size of building logs.
|
| So the only chance for advancement are those who propose
| radical changes, like Rust, or Zig if you want something less
| complex and "spiritually" closer to C
| the_af wrote:
| > _Would not be cheaper to fix those unsafe languages, such as
| C_
|
| I don't think C can be "fixed" without turning it into a
| radically different language. At that point, why not simply
| start from scratch?
|
| This is about language design that makes some languages
| fundamentally unsafe, not about fixing some bugs or hardening
| the toolset.
| hypertele-Xii wrote:
| How do you "fix" C?
| jedisct1 wrote:
| Zig is a "fixed" version of C.
| darkwater wrote:
| No, it's another language.
| cy_hauser wrote:
| Rewrite it in Rust!
| wtallis wrote:
| The main strength of a memory-safe language is that the safety
| mechanisms are opt-out, not opt-in: you can confine the unsafe
| behavior to a small, manageable portion of the codebase. And
| trying to "fix" C to become a memory-safe language would
| probably just result in something like a Rust dialect that
| merely bears more superficial resemblance to C. In any case,
| thorough re-writing of existing codebases would be required.
| You might as well re-write in a language that currently exists,
| rather than add creating a new language to the top of the to-do
| list.
| bluGill wrote:
| I would call C++ a memory safe language that has more than a
| superficial resemblance to C. Rust makes is more obvious
| when/where you are intentionally opting out, but most C++
| memory bugs are where someone opted out of the modern C++ way
| to use the C way. (including new/delete as the C way).
| madmoose wrote:
| > I would call C++ a memory safe language
|
| C++ is not, in any sense, a memory safe language.
| bluGill wrote:
| Then you don't know anything about C++, only the C legacy
| it is built on. Modern C++ has good data structures and
| good memory management.
|
| To be fair actually using those features of C++ instead
| of the C legacy is still catching on. Most instructors
| aren't even teaching it.
|
| Using C in C++ is like using unsafe everywhere in rust -
| you can do it, but you lose all the advantages.
| Liquid_Fire wrote:
| Even if you stick to "modern C++" and never use C-style
| pointers, arrays, enums or unions, it is still trivial to
| have memory errors in C++.
| [deleted]
| rcxdude wrote:
| This is a common viewpoint but also wrong. There's plenty
| of ways to invoke memory unsafety in 'modern' C++:
| iterator invalidation is probably the biggest source.
| bluGill wrote:
| While you are technically correct, those who study C++
| security note that those are a tiny minority of all
| security errors. Still an issue, but not nearly as common
| an issue as the ones that come from the C roots of C++.
| rational_indian wrote:
| No true Scotsman huh?
|
| https://en.wikipedia.org/wiki/No_true_Scotsman
|
| Also can you link to an example of this purported memory
| safe C++ compiler? Please note for an apples to apples
| comparison it must reject all memory unsafe non-modern
| C++ constructs.
| floatboth wrote:
| Memory safety is not about what a language _has_. It 's
| about all the things a language _doesn 't_ have, doesn't
| allow.
| [deleted]
| ragnese wrote:
| If C++ compilers had a default mode of rejecting code that
| was unsafe (any code using raw pointers or unchecked array
| access, I guess?), then sure- you could call it that. But
| as it is, no way.
| rational_indian wrote:
| > would call C++ a memory safe language
|
| LOL.
| frumiousirc wrote:
| What if any requirements are there on the licensing of these
| reinvented wheels? Specifically would GPL or AGPL be acceptable?
| DesiLurker wrote:
| and abandon it right after they become technically uninteresting.
| seriously aside from not supporting current GPL licence on these
| open source projects I dont see a reason to not support them.
| choeger wrote:
| While rust certainly does a lot of things right, it is still a
| moving target with a relatively high velocity.
|
| There is a python library that recently switched to rust
| (partially) and now cannot be build on several platforms.
|
| This also means that very long-lived platforms will probably be
| forced to upgrade that library every half year or so in line with
| the development of rust.
|
| I'd say such an initiative needs a long-term supported version of
| rust, similar to C99 or so.
|
| And this does not even account for cargo. I know you don't _have_
| to use cargo, but is that practical?
|
| How are we going to maintain stable software for, say, ten years
| or more? Do we have anyone committed to providing a stable rust
| environment for these cases? Especially considering that rust has
| no ABI for dynamic libraries.
|
| Oh and by the way: We need a way to bootstrap rust as well. The
| current distribution downloads binaries of the last one to build
| itself. _And_ it ships with a vendored version of llvm.
|
| So in conclusion: I really like the rust language. But I am
| concerned that it is still way too immature and to "hip" for
| long-running, stable projects.
| pid_0 wrote:
| >There is a python library that recently switched to rust
| (partially) and now cannot be build on several platforms.
|
| Progress should not be held back for the rest of us because a
| few will get left behind. If you are running something so old
| that LLVM doesn't support it, upgrade. It's 2021.
| gpm wrote:
| > This also means that very long-lived platforms will probably
| be forced to upgrade that library every half year or so in line
| with the development of rust.
|
| As a security library... you really ought to be updating it
| every half year of so.
|
| This has nothing to do with rust though. Rust maintains full
| backwards compatibility, so even if you upgrade the compiler
| (which is unlikely on long lived platforms) and not the library
| nothing breaks.
|
| Nor is it necessarily the case that upgrading the library will
| require updating rust (though this isn't what you appear to be
| complaining about). Not only is it a build time dependency
| (that you don't even need to install if you are on common
| platforms which ave prebuilt binaries), but they are explicitly
| testing against a particular fixed old version of rust as well
| as recent ones:
| https://github.com/pyca/cryptography/blob/main/.github/workf...
|
| > I'd say such an initiative needs a long-term supported
| version of rust, similar to C99 or so.
|
| This describes _every_ version of rust since 1.0. Rust is fully
| backwards compatible, unlike new C standards.
|
| > Do we have anyone committed to providing a stable rust
| environment for these cases?
|
| Yes, the entire rust team...
|
| > Especially considering that rust has no ABI for dynamic
| libraries.
|
| This is... rather irrelevant? Technically it does have support
| for the C abi for dynamic libraries though.
| choeger wrote:
| I think you should reconsider the actual problem, if you do
| not understand it.
|
| The fact that rustc is backwards compatible is great, sure.
| But it does not solve the problem of distributors.
|
| Say you have libfoo-1.0 that uses rust and cargo 1.2 . A
| distributor that now releases their linux version and wants
| to ship libfoo-1.0 must consider the requirements for
| supporting it. This support usually outlives the upstream
| support.
|
| If libfoo-1.0 uses C, for instance, it is highly likely that
| a distributor can backport any security-relevant changes from
| libfoo-1.1, 1.2, and later into 1.0.
|
| In the case of rust there are two uncertainties creeping in:
| One is the highly dynamic nature of cargo dependencies.
| libfoo might depend on a myriad of crates transitively and
| rust simply is too young to provide any experience in how
| long they will be available or supported. And no, it is
| generally not a good idea to update to new feature releases
| for security updates. This is something that stable
| distributions explicitly try to avoid.
|
| So our distributor now must consider if changes in libfoo-1.2
| pulled in changes via cargo that cannot be easily backported
| because they are feature releases or have a dependency on a
| newer version of rust.
|
| The second problem is the compiler itself. Any change in
| libfoo-1.2 might require a newer version of rust. It is great
| that they want to be backwards compatible but now our
| distributor must consider upgrading a substantial piece of
| software during the support period (and testing the backwards
| compatibility, of course) just to backport a security fix.
|
| Please don't get me wrong, I really like rust and I think
| these problems _can_ be solved. But I also like stable
| software and all these languages with their package
| repositories seem to simply ignore everything that 's older
| than one or two years. If rust wants to become a new C, which
| I would very much like, it needs to demonstrate a certain
| stability.
| marshray wrote:
| > Any change in libfoo-1.2 might require a newer version of
| rust.
|
| In theory perhaps. But in theory this could happen with C
| or C++ too. In practice, security patches try to be
| minimal.
|
| Sure, if you're several versions behind you might have more
| work to do. But you knew you were gonna own that problem
| when you decided to lag behind.
| gpm wrote:
| I think you are missing the fact that the solutions are
| already in place for all of this.
|
| > One is the highly dynamic nature of cargo dependencies.
| libfoo might depend on a myriad of crates transitively and
| rust simply is too young to provide any experience in how
| long they will be available or supported. And no, it is
| generally not a good idea to update to new feature releases
| for security updates. This is something that stable
| distributions explicitly try to avoid.
|
| crates.io is designed so that dependencies put up on it
| remain available _forever_ , and that the only other cargo
| related dependencies also have to be hosted on crates.io -
| solving the availability problem.
|
| Of course if you're trying to backport a security fix to an
| old version of a transitive dependency that (like the non-
| transitive dependency) is no longer supported, you do have
| to make that backport. Once you make that backport it's
| easy to update the version of the transitive dependency
| with a [patch] section in the top level Cargo.toml.
|
| > The second problem is the compiler itself. Any change in
| libfoo-1.2 might require a newer version of rust.
|
| This is already mitigated in multiple ways. As discussed
| before, people do make sure to support old versions of
| compilers, you don't need to have a compiler on your target
| platform anyways. But moreover, the amount new language
| features make backporting difficult is dwarfed by the
| amount that internal library changes do.
| sammorrowdrums wrote:
| Was this an encryption library by any chance? Generating pgp
| keys to yubikey with an offline master involved having a system
| with rust toolchain in order to build a Python lib I required
| for a Yubikey utility.
|
| I could have got around it, but it took me by surprise that a
| Python dep required it. Not being negative or criticising the
| fact, just first time I had observed it.
| gpm wrote:
| It was, "cryptography".
| wizzwizz4 wrote:
| We _have_ a way to bootstrap Rust:
| https://github.com/thepowersgang/mrustc
| cp9 wrote:
| > There is a python library that recently switched to rust
| (partially) and now cannot be build on several platforms.
|
| there comes a point where we can't let dependencies on Amiga or
| DEC Alpha (companies/platforms that haven't existed in literal
| decades) hold the rest of us back. If someone wants to port
| rust/llvm to those platforms more power to them, but the rest
| of us don't need to suffer. The only platform that actually
| matters that was negatively effected by that change is
| musl/alpine because there aren't musl wheels available and you
| have to build the rust part from source.
| notacoward wrote:
| > dependencies on Amiga or DEC Alpha
|
| What about embedded platforms, which tend to remain relevant
| (i.e. present in large numbers out in the real world) much
| longer than desktop or server platforms? Don't they matter?
| Don't they need updates, including for security, as much as
| any other?
| bowyakka wrote:
| Is it not a bit of a paradox that said embedded platforms
| matter enormously right up to, but not including the point
| of having toolchains ported to them and maintained? And yes
| while gcc can be built to produce binaries for an alpha
| it's not producing any bugs right?
| saagarjha wrote:
| The interests of people writing new language toolchains
| sadly does not seem to extend to supporting older
| platforms. This doesn't mean that they are not important,
| it just means that that they are not important to the
| people writing the new language toolchains.
| wmf wrote:
| It's not a matter of interest; if an embedded platform
| matters they should be willing to pay someone to port
| Rust.
| saagarjha wrote:
| ...or they can just continue using C.
| tdimitrov wrote:
| Rust is a nice language for greenfield system software project.
| But everyone in the software industry knows that you can't just
| put a project on hold and rewrite completely in new language. I
| can't see anything good coming from this initiative.
| galgalesh wrote:
| Rust is designed to integrate into legacy codebases so security
| critical code can be rewritten in small steps. See the URL
| parser I Firefox. Moreover, the initiative this post is talking
| about uses rust to rewrite small parts of a C codebase.
|
| Honestly, I'm getting more and more disappointed by the quality
| of discussion here on HN.
| cesarb wrote:
| Rust is a nice language for gradual rewrites too, especially
| when the original language was C. Some examples are Mercurial,
| librsvg, and Mozilla Firefox.
| safercplusplus wrote:
| Some solutions aren't that well publicized. Here is an example
| of an open source png encoder/decoder written in C (mostly)
| being auto-translated to a memory-safe subset of C++:
|
| https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...
| tmotwu wrote:
| I wonder what implications this has for C/C++ in big tech
| backends. Startups, from what I've seen, have already moved away.
|
| No doubt, it will still be around in low latency or embedded.
| However, this is a significant cut of their market share,
| especially if Google, Microsoft, and Amazon retires its usage in
| its systems.
| mariusor wrote:
| A big chunk of the C++ development is done in game studios. I
| doubt they would be willing to move away from it just for
| memory safety's sake.
| Tobu wrote:
| Buggy AAA releases are currently the norm, but that takes a
| reputational and financial toll and doesn't make for happy
| developers either. Cyberpunk 2077 is the latest example.
| blt wrote:
| But aren't most user-visible bugs in games caused by logic
| errors, not memory errors?
| jellyfishfry wrote:
| hm
| wdb wrote:
| Do they also have a study what the cost is of reimplementing an
| existing project in a new project regarding feature parity
| existing bugs people might depend on?
| jaas wrote:
| Executive Director of ISRG here. To provides some more clarity on
| how this partnership works...
|
| ISRG plans and coordinates investments in moving open source
| software to memory safe languages. We have a strong preference
| for working on these plans with maintainers and developers, and
| then funding them to execute the plan.
|
| Once we have a plan prepared, and, ideally,
| maintainers/developers on board, then we pitch it for funding.
| Once someone (e.g. Google) funds the project, then ISRG handles
| contracts with the project developers/maintainers to get the work
| done. In some cases maintainers may be on board but want us to
| find a contractor to actually do the work.
|
| In both of the first two projects, and in most future cases, the
| money that Google (or another company) provides will largely go
| to project maintainers / developers.
___________________________________________________________________
(page generated 2021-02-18 23:00 UTC)