[HN Gopher] A quick introduction to data parallelism in Julia
___________________________________________________________________
A quick introduction to data parallelism in Julia
Author : amkkma
Score : 101 points
Date : 2020-10-06 17:44 UTC (5 hours ago)
(HTM) web link (juliafolds.github.io)
(TXT) w3m dump (juliafolds.github.io)
| tkf wrote:
| Hi, the author here.
|
| One of the hidden messages of the introduction is: watch Guy
| Steele's talks [1][2][3] if you are interested in data
| parallelism! These talks are not Julia-specific and the idea is
| applicable and very useful in any languages. My libraries are
| heavily inspired by these talks.
|
| Of course, if you haven't used Julia yet, it'd be great if (data)
| parallelism gives you an excuse to try it out! It has a fantastic
| foundation for composable multi-threaded parallelism [4].
|
| [1] How to Think about Parallel Programming: Not!
| https://www.infoq.com/presentations/Thinking-Parallel-Progra...
|
| [2] Four Solutions to a Trivial Problem
| https://www.youtube.com/watch?v=ftcIcn8AmSY
|
| [3] Organizing Functional Code for Parallel Execution; or, foldl
| and foldr Considered Slightly Harmful https://vimeo.com/6624203
|
| [4] Announcing composable multi-threaded parallelism in Julia
| https://julialang.org/blog/2019/07/multithreading/
| nextos wrote:
| I love Julia, and in many areas both the language and standard
| libraries are quite polished.
|
| However, simple parallelism based on threads (e.g. a parallel
| map) surprisingly requires a third party library. ThreadsX, the
| one you posted about, is also my favorite option.
|
| Besides, the standard pipe operator (|>) doesn't support
| partial application. This has lead to several third-party
| reimplementations via macros, transducers, etc. Since this is a
| basic feature, fragmentation is a bit worrying.
| tkf wrote:
| Yeah, I get the point that it's nice to have basic things in
| the core language and stdlib. I'm rather on the "minimal
| language core" side in this discussion and I think it'd be
| better to not start adding more things in Julia Base and
| stdlib. At least not right now, to avoid accumulating "dead
| batteries." There are some non-trivial things to figure out
| for a composable data-parallel interface. For example, what
| the data collection interface should be to support folds over
| them? I'm using a minimalistic interface defined in
| SplittablesBase.jl for this but it's not wildly used outside
| my packages. So I'm not super confident that it's enough yet.
|
| [1] https://github.com/JuliaFolds/SplittablesBase.jl
| eigenspace wrote:
| Just to echo Tkf here, I think it'd be great to have
| something like ThreadsX.map in base someday, but when the
| multi-threading features in Julia first came out, it was
| unclear what the best interface to provide would be.
|
| Since Julia is committed to semantic versioning, this is a
| kinda scary prospect because it means that any high level,
| exported interface we decide on, we'll be stuck with until at
| _least_ 2.0 and probably forever.
|
| So with things like this, it's important to be conservative
| about the interfaces we expose and instead, we've have people
| explore different approaches out in the package ecosystem.
| One day, if someone can make a strong enough argument, I'm
| sure we'll see one of these package solutions end up in Base.
| melling wrote:
| There's a current MIT course call Introduction to Computational
| Thinking that's using Julia. I've watch a handful of videos so
| far. Good introduction to Julia.
|
| https://computationalthinking.mit.edu/Fall20/
| Sukera wrote:
| One of the lecturers working on this is Grant Sanderson, of
| 3Blue1Brown fame. Great video series so far!
| eigenspace wrote:
| Takafumi has really built up some amazing infrastructure in the
| package ecosystem. His Transducers.jl [1] is really interesting
| and powerful and lately he's done a lot of work with things like
| FLoops.jl [2] and ThreadsX.jl [3] to try and bring the benefits
| of transducers to more 'regular' familiar representations so more
| people can enjoy the benefits. The basic idea behind all of it is
| that he has an efficient and modular way of describing various
| 'looping' constructs that can be stuck together, optimized and
| parallelized automatically.
|
| It'd be quite interesting to see this stuff extended to GPUs.
|
| [1] https://github.com/JuliaFolds/Transducers.jl
|
| [2] https://github.com/JuliaFolds/FLoops.jl
|
| [3] https://github.com/tkf/ThreadsX.jl
| cmcaine wrote:
| I really like Baselet.jl[1] which provides Tuple-specialised
| implementations for a bunch of Base APIs that the compiler is
| very good at unrolling :)
|
| [1]: https://github.com/tkf/Baselet.jl
| cbkeller wrote:
| Tkf has so many cool packages! Less to do with transducers or
| parallelism, but in terms of just cool and useful stuff, Maybe
| [1] and BangBang [2] definitely come to mind as well.
|
| [1] https://github.com/tkf/Maybe.jl
|
| [2] https://github.com/JuliaFolds/BangBang.jl
| eigenspace wrote:
| One that I really thought was cool recently was
| UnderscoreOh.jl [1]. He has a real knack for finding crazy
| fun things you can do with julia syntax.
|
| [1] https://github.com/tkf/UnderscoreOh.jl
| amkkma wrote:
| Another one that's forward looking:
| https://github.com/tkf/Mutabilities.jl
___________________________________________________________________
(page generated 2020-10-06 23:01 UTC)