[HN Gopher] When Rustc Explodes
___________________________________________________________________
When Rustc Explodes
Author : milen
Score : 63 points
Date : 2022-07-13 18:39 UTC (4 hours ago)
(HTM) web link (fasterthanli.me)
(TXT) w3m dump (fasterthanli.me)
| PoignardAzur wrote:
| Every time I read one of these posts digging into logs and
| flsmegraphs I think "man, there really should be a simpler way to
| understand what a compiler is doing when you feed it code".
| fasterthanlime wrote:
| Author here: I honestly expected it to be much worse. The
| tracing integration in particular is fantastic - being able to
| just slap a `#[instrument(...)]` attribute on any function and
| then be able to filter which spans you want to see is kind of a
| superpower.
|
| That said I think much could be improved, still. I only breezed
| through the perf/nperf stuff because I've used them before, and
| through the self-profiling stuff because rustc devs helped me
| with it.
|
| I would've killed for a REPL while I was working on this (or a
| server architecture so I could write my _own_ rustc queries),
| and step through them, etc. I like Kate's approach to this[1] -
| I'd just like something a little more... interactive.
|
| What do _you_ think it should look like? I feel like a ton of
| good ideas come from folks who "simply didn't know it was
| impossible" (more accurately: haven't been trained to accept to
| work with subpar tools). I'm excited to try out Pernosco[2] for
| example, it seems like a much-needed rethink of debuggers.
|
| [1]:
| https://twitter.com/thingskatedid/status/1386077675211526148
|
| [2]: https://pernos.co/
| rowanG077 wrote:
| I recently started a Rust project coming from mostly writing
| Haskell in my job and oh boy was I surprised. I was expecting
| long compile times because of the complaints I read about
| everywhere. But in fact compilation is very speedy. Much, much
| faster then I'm used to with GHC in fact.
| zamalek wrote:
| It has significantly improved, and I mean _significantly,_ even
| in the past year alone.
| silverwind wrote:
| According to https://perf.rust-lang.org/dashboard.html
| compile times only have improved by like 5-10% since last
| year.
| rowanG077 wrote:
| That's pretty huge though.
| indiv0 wrote:
| 1. Don't use the `async` ecosystem.
|
| 2. Prefer dynamic dispatch to monomorphization (i.e., use fewer
| generics).
|
| 3. Don't use proc macros (i.e., don't depend on the `syn`
| crate, even transitively).
|
| Easy to say; hard to put into practice. But that's all there is
| to it.
| staticassertion wrote:
| Or don't, because all of those things are great. Compile
| times aren't that bad unless you're doing a clean uncached
| build, which is very unlikely.
| fasterthanlime wrote:
| > Don't use the `async` ecosystem.
|
| I want to make it /very/ clear that async isn't to blame at
| all for the pathological build times described here. It's a
| bug about traits and lifetimes, both very core concepts of
| Rust that you deal with even if you stay away from async
| code.
|
| async rust will certainly be more ergonomic once some more
| improvements land (hopefully later this year), but I don't
| feel like it deserves all the sighs it's been publicly
| getting these past few months. (And I /love/ to complain.
| I've written pieces named "Surviving Rust async interface",
| "Getting in and out of trouble with Rust futures", "Pin and
| suffering", etc.)
|
| > Prefer dynamic dispatch to monomorphization (i.e., use
| fewer generics).
|
| Unless you hit a pathological case as shown in the article,
| it tends to not be _that_ bad, especially if you enable `-Z
| share-generics=y` (unstable still, yet enabled by default for
| debug builds if I remember correctly).
|
| Overall still solid advice - although "use fewer generics"
| sometimes turns out to be "just turn a big generic type into
| `Box<dyn Trait>`" (it's not _just_ boxing, that would be
| `Box<T>`). That's what axum[1] does with all services, and
| it's never had the compile times issues warp[2] had, for
| example.
|
| > Don't use proc macros (i.e., don't depend on the `syn`
| crate, even transitively).
|
| Good news there, I hear there's some progress on the proc-
| macro bridge (which improves macro expansion performance) AND
| "wasm proc-macros". I hope this piece of advice will be
| completely irrelevant in a year (but for now, it's spot-on.
| using pin-project-lite instead of pin-project is worth it,
| for example).
|
| [1] https://lib.rs/crates/axum
|
| [2] https://lib.rs/crates/warp
| vgel wrote:
| My understanding of point #2 is that LLVM may still try to
| devirtualize the call, which would reduce the performance
| impact -- is that true for Rust? I know it happens
| sometimes in C++.
|
| Also for proc macros, rust-analyzer seems to struggle with
| them sometimes as well, so I try to avoid them (outside
| Serde, which is worth any price) for that reason.
| aaaaaaaaaaab wrote:
| Async seems to be the first big "footgun" of Rust. It's
| widespread enough that you can't really avoid interacting
| with it, yet it's bad enough that it makes people resent the
| language.
| rowanG077 wrote:
| Why is it a footgun? I have been using it and I didn't
| notice anything bad with it yet.
| vgel wrote:
| It's really not as bad as it's made out to be. You can
| paint yourself into a corner with it, but a lot of that is
| that async is _fundamentally more complicated than sync /
| threaded code_, and there's only so much any language can
| do to paper that over. Rust exposes a lot of details, so it
| can be complicated to get to grips with how they combine
| with async in certain corner cases, but the happy path is
| quite happy even now.
|
| A lot of the async Rust code I work with already looks like
| `async fn foo() -> ... { do_request().await?.blah().await
| }`, plus the occasional gathering of futures into a `Vec`
| to join on. That sort of thing, not much different from
| Javascript, but with a lot more control of the low-level
| details.
|
| A good deal of corner cases should get better once async
| traits are stabilized, which will mean much less need for
| manually writing out Future types. But honestly, even now
| it's not _that_ bad. I have a codebase that uses async to
| read hundreds of thousands of files[1], streaming gunzip
| them, pass them to another future which streaming parses
| records from them, and then pushes those parsed records
| into a `FnMut` closure for further non-async processing. It
| took a bit of thinking and design to get everything moving
| together nicely, but that corner of the codebase now is
| only ~200 lines of pretty straightforward code -- there 's
| like 1 instance of `Unpin`. It's not _that_ bad.
|
| [1]: I know async isn't necessarily faster for reading
| files, but it started life doing network requests and it
| can still saturate a 200-core machine so I haven't felt the
| need to port it over to threads.
| inferiorhuman wrote:
| FWIW I really don't like async in rust. It's improved
| significantly over the past couple years and it's nowhere
| near as bad as callback hell in Javascript but things
| still feel opaque. I've been toying around with a little
| monitoring agent (think Nagios or Sensu) to keep an eye
| on my defective LG fridge. So far I've managed to crash
| rustc twice. Trying to wrap my head around one library
| (that I was using incorrectly) I managed to "fork bomb"
| the damn thing and realize that I've little to no insight
| into the runtime. Try to find the current number of
| running tasks being managed by tokio...
|
| The beauty of the rust async stuff is that you can move
| to a multi-threaded runtime as you desire with minimal
| effort.
| fasterthanlime wrote:
| > Try to find the current number of running tasks being
| managed by tokio...
|
| As a heavy user of async Rust in production (at a couple
| places), resource leaks / lack of visibility into that
| has been a top issue.
|
| In this area, tokio-console[1] is an exciting
| development. I have high hopes for it and adjacent tools
| in the future. (Instrumenting your app with
| tracing+opentelemetry stuff can help a lot, too).
|
| Until those become featureful/mainstream enough, Go has
| the upper hand in terms of "figuring out what's going on
| in an async program at any given time".
|
| [1]: https://lib.rs/crates/tokio-console
| fasterthanlime wrote:
| Quick aside: if you're willing to live the nightly life
| (unstable rustc), the `type_alias_impl_trait` feature
| gets you most of the way to "async trait methods". You
| still have to have a `Future` associated type, but in
| impl blocks, it just becomes `type Future = impl
| Future<Output = Blah>`, and then the compiler infers what
| the concrete (and probably unnameable, if you use async
| blocks) type is - no need to mess with `Pin<Box<T>>`.
|
| The most egregious code comes when implementing one of
| the `AsyncRead`/`AsyncWrite` traits or similar, and that
| can come up a bunch in backend services, for example if
| you want to record metrics on how/when/where data flows,
| apply some limits etc. I'm curious how the ecosystem will
| adapt once async trait methods land for real.
| svnpenn wrote:
| > The crux of the problem is "gee grandma, what big types you
| have there", because... essentially, tower is a couple of
| important traits.
|
| > And the basic idea is "oh shit we don't have async trait
| methods" (hence the Future associated type) but also
| "backpressure is love, backpressure is life", by way of the
| poll_ready method.
|
| I really don't like this guys style of writing. It seems like the
| bones of the article are interesting, but then they slap like
| three layers of meme talk, irony and sarcasm, and it just ruins
| the read for me. It reminds me of TARS humor setting. It doesn't
| need to be zero, but its currently at like 98, can we turn it
| down some?
| fasterthanlime wrote:
| > but its currently at like 98
|
| Come on now. There's two chasms between "an academic paper", my
| blog, and "a blog with minion/the office reaction GIFs".
|
| The amount of humor is finely-tuned so that it's not /too/
| distracting from the actual technical topic at hand (of which
| there's always one, I'm not a sit-down comic), but it also
| filters out folks who'd rather focus on the form than the
| content / take themselves too seriously.
|
| Looks like the net caught you! Hi!
| staticassertion wrote:
| I would really like to see more minion gifs in your posts
| svnpenn wrote:
| No, I just don't want to read the TikTok equivalent of a blog
| post, but nice try.
| sophacles wrote:
| Isn't the TikTok equivalent of a blog post just a Tweet? I
| mean, back when the first came out there was a whole notion
| of "microblogging" that they were tapping into.
|
| This post has paragraphs, sections, coherent layout. It's
| certainly not a TikTok analogue.
| biorach wrote:
| Seriously! Some of Amos' posts about Rust as in-depth as
| anything on the internet. TikTok it is not.
|
| With the amount of effort he puts in he's earned the right
| to make a few dumb jokes
| aaaaaaaaaaab wrote:
| It's the kind of humor that usually not even the author finds
| funny. They just learned that this is how "funny" people talk.
| vgel wrote:
| It's fair to not appreciate the humor oneself, but this is a
| really unfair judgement / assumption of the author's intent.
| game-of-throws wrote:
| The emperor has no sense of humor?
___________________________________________________________________
(page generated 2022-07-13 23:00 UTC)