[HN Gopher] I rewrote a Clojure tool in Rust
___________________________________________________________________
I rewrote a Clojure tool in Rust
Author : praveenperera
Score : 116 points
Date : 2020-12-21 17:23 UTC (5 hours ago)
(HTM) web link (timofreiberg.github.io)
(TXT) w3m dump (timofreiberg.github.io)
| Narishma wrote:
| The code snippets in the article are unreadable due to low
| contrast.
| kimi wrote:
| Not sure about Rust, but the on the Clojure side, code does not
| seem pretty.
| gcb0 wrote:
| ah, and the real world meet language flame war.
|
| both of these codes are awful imo. but real world code always is,
| so that's not a valid critic.
|
| usually when code pass over errors they are just ignoring design
| holes and offloading the work to fix the design to the end user,
| but that's also how real world programs are architected every
| day.
|
| in the end: great, and bave article, to show how the languages du
| jour elegant syntax sugars added with formalism and other
| academic dreams in mind will be bent over backwards to just-get-
| shit-done in a needlessly complex method that tries to do
| everything with whatever data structure the underlying library
| (excell in this case) decided to presented the data instead of
| the better one for the method logic.
| TurboHaskal wrote:
| I loved the "mutability here! it's fine though" comment, it reads
| a bit like "I'm not racist but...".
|
| I guess this is what happens when you spend too much time in the
| functional programming cult.
| omn1 wrote:
| Great post! I like articles where the author has practical
| experience in two languages and compares them. Helps me make
| better decisions for future projects.
| [deleted]
| SomeHacker44 wrote:
| I personally was quite astonished (not in a good way) by the use
| of #_"comments" instead of ;[;[;[;]]] comments.
| saleheen wrote:
| Great Post!
| chrisulloa wrote:
| While I definitely agree Rust is a much faster language than
| Clojure, I would be interested to see benchmarks on your code
| that show just how much faster your Rust code was on the same
| data.
|
| I also noticed that you mentioned avoiding lazy sequences is not
| idiomatic in Clojure. I disagree with this since using
| transducers is still idiomatic. I wonder if you could've noticed
| some speed improvements moving your filters/maps to transducers.
| Though I doubt this would get you to Rust speeds anyway, it might
| just be fast enough.
| j-pb wrote:
| I just moved a medium sized codebase from clojure transducers
| to JS, and after having used clojure for 7+ years, and done so
| professionally, I don't wanna go back, ever. The JS solution is
| shorter, faster, and easier to understand. I'm thankfull for
| the insights into reality and programming clojure has provided,
| but highly optimised clojure is neither idiomatic nor pretty,
| you end up with eductions everywhere. Combine that with
| reaaaallllyy bad debuggability with all those nested inside out
| transducer calls (the stack traces have also gotten worse over
| the years, I don't know why, and a splintered ecosystem (lein,
| boot, clj-tools)) I'd pick rust and deno/js any day for a
| greenfield project over clojure. sadly.
| kostadin wrote:
| I have limited exposure to Clojure transducers but I spend
| most of my time writing JS/TS and I've found
| thi.ng/transducers[0] a pleasure to work with and super
| elegant for constructing data processing workflows.
|
| [0] https://github.com/thi-
| ng/umbrella/tree/develop/packages/tra...
| nbardy wrote:
| Debugging performance problems is the reason I stopped using
| `cljs`. Those stack traces are so painful.
| bjoli wrote:
| I doubt it will bring much. If properly implemented, there is
| nothing that makes generator-like laziness slower than
| transducers, and since it is pretty central to clojure I doubt
| you will see much speed gain by using transducers.
|
| In scheme, the srfi-158 based generators are slower than my own
| transducer SRFI (srfi-171) only in schemes where set! incurs a
| boxing penalty and where the eagerness of transducers means
| mutable state can be avoided.
|
| Now, I know very little clojure, but I doubt they would leave
| such a relatively trivial optimization on the table. A step in
| a transducer is just a procedure call, which is the same for
| trivial generator-based laziness.
| lilactown wrote:
| When I was reading the article, I thought the author of the
| post was probably pointing more in the direction of Clojure's
| immutable data being slower, rather than laziness
| specifically.
|
| IME (admittedly in a different context, doing UI development)
| Clojure's seqs and other immutable data can be a huge
| performance drag due to additional allocations needed. If
| you're in a hot loop where you're creating and realizing a
| sequence immediately, it's probably much faster to bang on a
| transient vector. Same with creating a bunch of immutable
| hash maps that you then throw away; better to create a
| simpler data structure (e.g. a POJO or Map) which doesn't
| handle complicated structural sharing patterns if it's just
| going to be thrown away.
|
| Transducer's would help in the author's first case to take
| the map/filter piped through the `->>`, which is going to do
| two separate passes and realize two seqs, and combine it into
| one.
| bjoli wrote:
| I stand by what I said (even though I am partial to
| transducers): there is no reason for lazy sequences
| overhead to be much more than a procedure call, which is
| exactly what a step in the reduction in the case of
| transducers is. At least when implemented as in clojure or
| srfi-171.
|
| I understand that there might be some overhead to creating
| new sequence objects, but removing intermediate sequence
| object creation should be simple, at least for built in
| map-filter-take-et-al.
|
| Edit: I seem to be wrong. People are recommending
| transducers over clojure's lazy map-filter-take-et-al
| because of overhead, which to me seems off, but there might
| be something about clojure's lazy sequences that I haven't
| grooked.
| adamnemecek wrote:
| I mean Clojure is still running on JVM so there will be at
| least that difference. AFAIK Clojure is slower than Java so
| there's that also.
| dgb23 wrote:
| Transducers are definitely idiomatic. They are more general
| over "similar things to transform in steps" (including
| sequences, messages and so on), so you can apply them to
| collections ("I have the whole data in advance") or channels
| ("I get the data piece by piece") and so on.
|
| Another idiomatic way to improve performance are transients[0].
| From the outside your function is still a function, but on the
| inside it's cheating by updating in place instead of using
| persistent data structures. See the frequencies function for a
| simple example[1].
|
| Clojure and Rust are both very expressive languages and even
| though they both can be considered niche, they have _massive_
| reach: Clojure taps into JVM and the JS ecosystems, Rust can
| also compile to WASM or be integrated with the JVM via JNI.
|
| The big difference between the two, and why I think they
| complement each other nicely, is that Clojure is optimized for
| development, and does its best at runtime, but Rust is
| optimized for runtime, and tries its best at development. (A
| similar take in the article). In other words: they both achieve
| their secondary goal well, but resolve trade-offs by adhering
| to their primary in the vast majority of cases.
|
| [0] https://clojure.org/reference/transients
|
| [1]
| https://github.com/clojure/clojure/blob/clojure-1.10.1/src/c...
| burnthrow wrote:
| > After switching to Rust, I had to implement more complicated
| logic that resolved dependencies between the rules I was diffing.
| This became complicated enough that even with a static type
| system I could only barely understand it. I doubt I would have
| been able to finish it in Clojure.
|
| I have no idea what the author's trying to say here.
| mattrepl wrote:
| I believe they are saying the static type system and type
| checker provided by Rust helped them write complicated code.
| burnthrow wrote:
| Yes but it's not clear how. The post is light on details in
| general.
| mistersys wrote:
| Sure, but having used static vs. dynamic languages
| extensively on gnarly problems, it's pretty clear what
| they're saying. There comes a point where dynamic languages
| can be quite hard to debug compared to language with well
| deigned types, especially when working on tricky problems
| with many constructs and elements. Less rules does not
| equal greater productivity.
| kristoft wrote:
| Nice post, very well written! Just not sure why compare non-gc
| low-level statically-typed language with vm-based gc dynamic
| language.
| nindalf wrote:
| It's a fair comparison because many developers, including the
| author, would have to make a choice between languages, some of
| which would be GC and while others are non GC.
| didibus wrote:
| A bit sad there was no profiling done, or at least the article
| doesn't mention it. Maybe optimizing Clojure wouldn't have been
| that hard, could have been only a few places needed tweeking. In
| any case, Rust is obviously targeting high performance in a way
| Clojure isn't. Rust is faster than Java, and Clojure can only
| ever match Java in performance, not exceed it. But still, it's
| not clear if the author tried to optimize the Clojure version or
| not?
| dgb23 wrote:
| > Rust is faster than ...
|
| Not if you use the wrong constructs, copy stuff around in the
| heap, use ref counting everywhere in longer running processes.
|
| I'm not nitpicking here, in Rust you can get really fast, but
| its on you to make that happen.
|
| For example persistent data-structures (used in Clojure) are
| really fast and for some operations and cases even close to
| optimal.
|
| Performance is hard, and I very much agree with your question
| here. What has been measured and what are the results.
| didibus wrote:
| I agree with you, but I think the assumption here is that
| we're comparing two code bases that are both trying to be
| performant. The Rust defaults will probably start off more
| performant, and the ceiling will be higher as well.
|
| A lot of the performance comes from the different paradigms
| though, so it's not always an apple to apple comparison. But
| I also think that's an assumption being made when talking
| about a Clojure Vs Rust implementation. In the latter, you're
| most likely implying using mutable collections, fixed size
| structs, primitive types, and a tighter memory allocation
| surface. And not surprisingly, those are the same changes
| you'd make to your Clojure code base to speed it up (most
| likely).
| burnthrow wrote:
| > Clojure can only ever match Java in performance
|
| You're technically correct, but the typical Java program making
| heavy use of threads has inefficiencies (and incorrectness)
| that would be avoided with Clojure's higher level async APIs.
| As it's easier to write idiomatic, performant C than the
| "faster" ASM.
| systems wrote:
| i like how in the end the system was replaced by a database
| solution
|
| every language need easy access to a query-ble database many
| problems are a lot simple when solved declaratively as a query-
| ble database
|
| the relational model, is functional, and is a very good solution
| to a wide range of problems
|
| i think the Sqlite engine should be integrated in the standard
| library of every language, and either use sql, or the language
| can provide a native sql alternative in the original language
| itself, or we can create a new standard language (because yes,
| sql can be improved upon)
|
| I think Chris Date D language can be a place to start to
| investigate SQL alternative , or as a language that can be more
| easily emulated in other languages
| secondcoming wrote:
| 'queryable' is a word!
| [deleted]
| wwweston wrote:
| What would you say the advantages of Date's D over SQL are?
| systems wrote:
| This is from memory, and i saw few examples but what i recall
| the syntax was less dsl-ish SQL is a DSL, it is not a general
| purpose language
|
| Chris Dates' D ( not to be confused with Walter Bright D)
| looks more like a regular programing language
|
| And was functional
|
| So I think an language can add a library or an extension that
| act like D, or simulate D
|
| Check this to get an idea
| file:///C:/Users/ammselshishini/Downloads/Rel-and-Tutorial-D-
| Quickstart.pdf
|
| Also, as I recall, the Notion of RelVars made it sound
| functional
|
| Each Relational Operator, returned a RelVar So you were
| passing RelVars between operators until you get the result
| you want, which was a RelVar
|
| Anyway this is from memory, so I maybe be wrong on many
| things .. but again from memory D or tutorial D sounded a lot
| less DSL-ish and lot more functional than SQL , which was an
| improvement in my mind
| ithrow wrote:
| The articles makes a good a case of how Rust can look as clean as
| other high-level programming languages when writing high-level
| business logic.
| dgb23 wrote:
| I agree that Rust has some really great concepts. The example
| code makes heavy use of traits for example, which are very
| ergonomic and provide a dynamic feel.
|
| The context is a rewrite AKA runtime optimization. So the
| result is already understood. A great use-case for Rust is top-
| down implementation.
|
| Also the code doesn't show any of the more painful cases. From
| the article:
|
| > There are some inefficiencies visible here, and they're
| probably the most important spots for performance improvements.
| But they're still there as fixing them was too hard and/or
| time-consuming for me.
|
| Resolving these "inefficiencies" is where Rust really shines.
| Because it _can_ resolve them and does it internally consistent
| on top of it. But at the same time, this is where you really
| _slow down_ in development and need to think about the more
| complex and intricate concepts such as lifetimes and borrow
| semantics.
___________________________________________________________________
(page generated 2020-12-21 23:00 UTC)