[HN Gopher] RJIT, a new JIT for Ruby
___________________________________________________________________
RJIT, a new JIT for Ruby
Author : pmarin
Score : 243 points
Date : 2023-03-07 11:29 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| sambostock wrote:
| Several points discussed in these comments are addressed by the
| author in the linked https://bugs.ruby-lang.org/issues/19420
| yxhuvud wrote:
| Are they adding a new jit each version now?
| pmarin wrote:
| It is replacing MjIT by the same author.
| beders wrote:
| Is jRuby still a thing? I didn't see it in the perf comparisons.
| Lio wrote:
| Yes it's definitely still a thing.
|
| They actually had a new release at the beginning of March.
|
| https://www.jruby.org/2023/03/08/jruby-9-4-2-0.html
| titzer wrote:
| Honest question, I do not know Ruby's semantics well. But, as
| someone who has worked on many JITs in the past, how is it in
| these results, _three different_ JITs failed at getting more than
| a 2x performance improvement? Normally, a JIT is a 10-20x
| improvement in performance, just from the simple fact of removing
| the interpreter dispatch loop. What am I missing?
| seunosewa wrote:
| Dynamic language, which allows anything to be changed at any
| time.
| titzer wrote:
| It's clearly more complicated than that. JavaScript is also
| highly dynamic and JITs there often give 10-100x speedup.
| nicoburns wrote:
| My impression is that Ruby is _more_ dynamic than pretty
| much everything else. I think this is true in terms of
| language features, but also in terms of the style that code
| is written in practice.
| tenderlove wrote:
| I'll take a stab at this.
|
| YARV (Ruby's VM) is already direct threaded (using computed
| gotos), so there's no dispatch loop to eliminate. YARV is a
| stack based virtual machine, and the machine code that YJIT
| generates writes temporary values to the VM stack. In other
| words, it always spills temporaries to memory. We're actively
| working on keeping things in registers rather than spilling.
|
| Ruby programs tend to be extremely polymorphic. It's not
| uncommon to see call sites with hundreds of different classes
| (and now that we've implemented object shapes, hundreds of
| object shapes). YJIT is not currently splitting or inlining, so
| we unfortunately encounter megamorphic sites more frequently
| than we'd like.
|
| I'm sure there's more stuff but I hope this helps!
| CyberDildonics wrote:
| These comparisons seem to be to other ruby implementations. How
| does this compare to LuaJIT ?
| jhatemyjob wrote:
| It probably doesn't. LuaJIT is still state-of-the-art, despite
| being in maintenance mode for almost a decade...
| brokencode wrote:
| That is kind of irrelevant if you are running a Ruby
| application. And I think that if a developer is looking to
| start working on a new web server where performance is a
| significant concern, they are more likely to look at Go, Rust,
| or even JavaScript rather than either Lua or Ruby.
| ecshafer wrote:
| Performance is a weird metric for a web application. You can
| say Go or Rust will be more performant than Ruby or Lua,
| sure. But with web applications so often your performance has
| nothing to do with the language or hiccup. But you aren't
| just processing N requests and spitting out a response, you
| are communicating with other services and databases. IO is
| almost always a bigger source of latency in response than
| language speed, until you have a large enough service where
| you can start to worry about those small issues. Before the
| Developer performance matters more.
| CyberDildonics wrote:
| LuaJIT should be much faster than javascript
|
| Also how slow is still ok? People can talk about things that
| aren't 'performance sensitive' but at some point it's going
| to matter. If a program is serving up web pages, that's an
| interactive application and people are waiting on the
| program.
| brokencode wrote:
| JavaScript is so much more widely used for so many
| applications that it's hard to justify LuaJIT, even if it
| is faster. They're both much faster than most scripting
| languages like Python and Ruby.
|
| If JavaScript really isn't fast enough, then I think you
| should be thinking about something like Go or Rust instead.
|
| LuaJIT is an incredible piece of technology, but in this
| performance tier JavaScript has won out due to sheer
| ubiquity and the size of its ecosystem.
|
| Edit: I never said slow was okay. I'm not advocating for
| slow, I'm just saying that the target audience for RJIT is
| developers who are already using Ruby. For significantly
| better speed, you probably want to look elsewhere.
| winrid wrote:
| Even a slow framework is still fast for humans. My Django
| site renders the homepage in 10ms, and django is kind of in
| the realm of Rails performance wise.
|
| It's all about cost, really. But you can just tell Nginx to
| cache pages and then it's not a problem for the vast
| majority of use cases.
| JohnBooty wrote:
| Also how slow is still ok? People can talk about things
| that aren't 'performance sensitive' but at some point
| it's going to matter
|
| Done a fair amount of Rails perf tuning over the years. One
| of my favorite things to work on.
|
| "Fast enough" for me, is when your web framework is
| _nowhere near_ your bottleneck. On your average web app
| endpoint you 're probably spending 95-99% of your time on
| external calls to Redis/Postgres/etc and Postgres is
| probably your specific bottleneck.
|
| For _these_ apps, Rails is most definitely fast enough. You
| could rewrite your app layer in well-tuned C or assembly
| and guess what, it 's getting maybe 1-5% faster if you're
| lucky. Maybe you get from 100ms down to 95ms and all you
| had to do was rewrite 100,000 lines of Ruby in 200,000
| lines of C.
|
| For other cases, obviously, maybe Rails is your bottleneck.
| Maybe you're providing a read-only API and everything is
| cachable in RAM. Rails will be fast, maybe 10ms per
| request, but a faster framework can spew out responses in
| 2ms and now you have 5x the capacity and your P95s during
| peak hours are really smoothed out.
| nirvdrum wrote:
| Do you have a particular task in mind? The languages have
| different semantics and that has an effect on performance.
| E.g., in Ruby nearly everything is a method call. Ideally a JIT
| would eliminate that overhead, but you'll likely see varying
| degrees of performance depending on what your task is. And
| that's before you get to core library methods, which often are
| written in C and not handled by the JIT.
| foxandmouse wrote:
| Is there any use of ruby in the deep learning space? It's my
| language of choice, but Python seems to be ubiquitous.
| cdiamand wrote:
| It looks like there was a little movement in the space not too
| far back:
|
| https://ankane.org/new-ml-gems
|
| But I don't know how often this is used in production. I've
| ended up training the models in python and then loading them in
| Ruby.
| rco8786 wrote:
| Why would one use this over YJIT?
| maxfurman wrote:
| To add to sibling comments, YJIT needs the Rust compiler, so if
| you have to build your own Ruby binary, and you have to build
| it on a system where you can't get a Rust compiler, then RJIT
| will make your life easier. Not sure how common this is in the
| real world though.
| sparker72678 wrote:
| Right now maybe you wouldn't, very much (though you should
| profile your code to see which performs better). But having a
| JIT written in Ruby potentially makes further development on it
| more accessible to the community. We will see!
| weatherlight wrote:
| seems strange, since most rubyist aren't compiler engineers.
| I feel like you'd still want to keep writing your compiler in
| Rust, and try to eek out your performance there.
|
| I'm still scratching my head, other than accessibility, Why
| Ruby over Rust.
|
| Note: I'm a Ruby dev, I don't know Rust. I've written few toy
| interpreters in Elixir and OCaml. This is my very limited
| understanding of compiler design, etc.
| sudhirj wrote:
| Using a JIT seems different from creating a JIT, though,
| and doesn't seem to require compiler engineers. From what I
| understand this converts Ruby code directly into C object
| code? Or does it transpile to C and then compile it? Either
| way, transpiling doesn't seem as complicated as compiling.
| sparker72678 wrote:
| I think you could say yjit fills that role; in any case,
| having multiple active jit projects leaves open a lot of
| room for experimentation. In the end, you might be right
| and rjit will fade away -- we will see!
| simlevesque wrote:
| > most rubyist aren't compiler engineers.
|
| You could say that about any language.
| weatherlight wrote:
| Can you say that about Standard ML? theres traditionally
| certain langs with particular ergonomics that lend them
| self to this kinda work. Like having (parser, lexer libs
| as apart of the lang's stdlib)
|
| Rust's ADT seem particularly useful in this context. it
| really makes refactoring a breeze. (OCaml has a similar
| type system.)
|
| Your point is generally true though.
| ColonelPhantom wrote:
| What if you turn it around? "Most compiler engineers
| aren't Rubyists."
|
| I don't know if that's true or not, but I imagine most
| compiler engineers tend to be more engrossed in languages
| like *ML, Rust, or Haskell. Or languages that are common
| in general, like C++ or Python. Ruby isn't that popular
| (outside of the Rails niche at least?), and it doesn't
| fit very well in a compiler niche either, I think.
| greenpeas wrote:
| But why would a compiler engineer that's not familiar
| with Ruby work on Ruby? They have so many other languages
| to choose from. I don't necessarily think that writing a
| JIT in Ruby is a good strategy to attract people to work
| on it, but if you are going to attract compiler people,
| you most likely want those who are also Rubyists.
| JonChesterfield wrote:
| Speaking mostly for myself, at least some compiler
| engineers like difficult source or target languages. The
| very dynamic ones are difficult to compile efficiently
| and thus more interesting than some alternatives. Plus
| Ruby hasn't had the attention paid to it that JavaScript
| has so the design space is closer to greenfield. I can
| see the attraction.
| riffraff wrote:
| there is no concrete reason to use it right now, and it's
| marked as experimental, but being pure ruby would allow for
| exploration and experimentation more easily.
| nerpderp82 wrote:
| Being pure Ruby, it should make it much easier for Ruby
| programmers to hack on. I am really impressed with the latency
| numbers. This will be some enjoyable code to read. It also
| sounds like through this and other's work, that Ruby will have
| a defacto JIT interface to the VM. This could open up the door
| for domain or framework specific jits, AI powered jit, jits
| that reload their past state, etc.
|
| I am a huge Rust fan, but going up stack and writing your jit
| as first party is pretty cool. Maybe after RJIT is well
| factored, the internals could done in Rust again.
|
| Really happy for Ruby!
| mabbo wrote:
| Does RJIT get JIT compiled... by itself? That would be lovely in
| the sense that as RJIT finds more optimizations to speed up code,
| it would become itself faster.
| extrememacaroni wrote:
| How many levels of JIT would be too many I wonder
| ignoramous wrote:
| AOT + JIT + PGO (profile guided optimization) is where things
| are at.
| aardvark179 wrote:
| I mean, that's what happens with JITs like Graal. It can
| present a warmup issue which is part of the reason Graal did so
| much work to enable AOT compilation.
| sparker72678 wrote:
| I love all the attention Ruby performance is getting lately!
| Qem wrote:
| Congrats to the Ruby developers, now they are on the way to
| have more than one production-grade JIT available in the
| reference implementation. I hope Python catches up soon, and
| the proposal to merge CPython and Pyston goes forward.
| jhoechtl wrote:
| Boy I lost track of all the Ruby Jit attempts.
|
| According to the computer language shootout all micro-
| optimizations
| ezekg wrote:
| YJIT sped up my Rails app by about 30%. It has a memory
| overhead, but it's worth it.
| maxime_cb wrote:
| For anyone curious, we've been working to reduce the memory
| overhead and have added some stats to keep track of memory
| usage over time. On this graph, you can see a comparison with
| the CRuby interpreter:
|
| https://speed.yjit.org/memory_timeline#railsbench
| greenpeas wrote:
| What happened on jun 14? (the dramatic drop in memory
| usage)
|
| Edit: I guess this (https://github.com/ruby/ruby/pull/5944)
| PR was merged.
| maxime_cb wrote:
| Yes. Prior to that point we used to allocate a large
| chunk of executable memory upfront. We switched to
| mapping that memory on demand, and that alone was a huge
| improvement.
| nerpderp82 wrote:
| That is huge, did it also reduce(~~bring in~~) tail latency?
|
| *edit, fix confusing vernacular
| ezekg wrote:
| Nope -- P99 also decreased. I've heard similar things for
| other Rails apps as well.
| nerpderp82 wrote:
| Sorry, in my usage "bring in" means move tail latency
| P99/P100 more to the left not as introduce, I'll be more
| clear next time.
|
| So yes! That is great news.
| nirvdrum wrote:
| https://ruby-compilers.com/ is a comprehensive list of the
| various Ruby compilers. There's a table summarizing them along
| with detailed descriptions for some.
| barrenko wrote:
| Is there any alternative to RubyMine as an IDE for Ruby newbies?
| nirvdrum wrote:
| For IDEs, Shopify provides the Ruby Extension Pack for VS Code
| [1]. Closely related to RubyMine is the Ruby plugin for
| IntelliJ IDEA. Those seem to be the biggest set of IDEs for
| several languages. There's Ruby support for editors like Vim
| and emacs, but that's a different experience from an IDE.
|
| [1] -- https://github.com/Shopify/vscode-shopify-ruby
| Alifatisk wrote:
| Vscode + solargraph + ruby extension
| zac23or wrote:
| I work everyday with Rails.
|
| In my experience, Ruby is not super slow.
|
| In my machine, I can create 1M of empty hashs on 0.17sec.
| Benchmark.measure{1000000.times{Hash.new}} @total =
| 0.1706789999999998s
|
| It's very good for a dynamic language.
|
| But ActiveRecord (and Rails) are incredibly slow.
|
| In my machine in 0.17sec only 2000 Models can be created.
| Benchmark.measure{2000.times{User.new}} @total =
| @total=0.17733399999999833.
|
| Some SQL+Network runs in less than 10ms, in these cases Active
| Models creation is slower than that.
|
| Yes, Rails can be slower than database access.
| Alifatisk wrote:
| I think the idea of Ruby being slow was back in 1.9, this was
| way before Matz announced the goal of 3x3 with Ruby 3.
| jeltz wrote:
| No, it is actually from even before that, from Ruby 1.8 which
| did not have a proper VM.
| Someone wrote:
| I would guess _Hash.new_ does little more than one or two
| allocations (one for the object, possibly one for an empty hash
| table that can later be resized), and if it did two
| allocations, linked one to the other.
|
| If so, that probably is more a benchmark of your memory
| allocator, which probably is written in C than of ruby.
|
| I also guess you ran the benchmark from a new ruby instance.
| That means memory wasn't fragmented. That certainly doesn't
| make the allocator's job more difficult.
| Alifatisk wrote:
| > ...many methods are direct translations of the Rust code into
| Ruby.
|
| Impressive
| l_theanine wrote:
| I'd definitely be more apt to have this as part of production
| system instead of the Rust one.
|
| Rust has got to be the ugliest, most unfriendly programming
| language I've ever laid my eyes on. And I wrote Perl for 10+
| years, so that's really quite a feat of aesthetics failure.
|
| Anyhow, I'm pretty impressed with the performance thus far, I
| like the idea of having multiple JITs available for a single-
| language ecosystem, regardless of how disgusting the language
| used to implement them. I think having competition means that
| there will be a race to the bottom and towards the "center" of
| general work. It's already really cool to see how the different
| approaches have clear preferences of the tasks they excel at and
| where they fall short.
|
| This is hugely valuable because it pushes Ruby forward for
| everybody, and will hopefully result in not only a faster Ruby
| for X, but a faster Ruby for everything, which is just an
| objectively good thing.
|
| Python is in a weird spot in this arena, because it is very
| clearly and very strongly orienting itself to continue to
| dominate practical data science work, and that means the need for
| JITs to handle regular jobs like text munging and whatnot fall by
| the side in order for the latest NumPy and Jax stuff, whatever is
| the current hot shit in the AIverse. Ruby doesn't suffer from
| that because it's pretty solidly lodged in the web development
| sphere, while also having a capable presence in netsec tools,
| application scripting, and probably a few more areas that I'm not
| aware of.
|
| If you're interested in some of cutting edge Python stuff, I'd
| recommend taking a look at exaloop/Codon. Codon will soon be able
| to output Python extensions that are compatible with Python's
| setuptools, so it will soon be possible to just include some
| .codon files with your project, use setup.py, and have decorators
| that can (literally) 100x your hot loops.
| bluedays wrote:
| Be honest, you mostly write this post so you can say you hate
| Rust, didn't you?
| l_theanine wrote:
| I guess you'll have to find an adult to read the whole thing
| to you to find out. I can see you got your feelings hurt
| after the first two sentences. :(
| pawelduda wrote:
| Quite interesting. My takeaway is that it can be on par with YJIT
| or even outperform it despite being in early development.
|
| Btw one project I work on switched to YJIT in production and
| there are no problems so far (but no noticeable perf gains
| either)
| stanislavb wrote:
| Yeah, I have the same experience with SaaSHub. I moved to YJIY
| a week ago - no issues, but no noticeable perf gains either
| (unfortunately).
| weatherlight wrote:
| What about in memory usage?
| maxime_cb wrote:
| You're right that the peak performance could be on par (or even
| better), but, and I acknowledge that I'm biased since I'm tech
| lead of the YJIT team, my takeaway is:
|
| 1. Kokubun, who works with us on the YJIT team, is leveraging
| insights he's learned while working on YJIT to build this. He
| has said so in his tweets, some of the code in RJIT is a direct
| port of the YJIT's code to Ruby. This is his side-project.
|
| 2. One of the challenges we face in YJIT is memory overhead.
| It's something we've been working hard to minimize. Programming
| RJIT in Ruby is going to make this problems worse. Not only
| will it be hard to be more memory-efficient, you're going to
| increase the workload of the Ruby GC (which is also working to
| GC your app's data).
|
| 3. Warm-up time will also be worse due to Ruby's performance
| not being on par with Rust. This doesn't matter much for
| smaller benchmarks, but for anything resembling a production
| deployment, it will.
|
| On your second point, if you're not seeing perf gains with
| YJIT, we'd be curious to see a profile of your application, and
| the output of running with `--yjit-stats`. You can file an
| issue here to give us your feedback, and help us make YJIT
| better: https://github.com/Shopify/yjit/issues
___________________________________________________________________
(page generated 2023-03-08 23:00 UTC)