[HN Gopher] OCaml 5.0 Multicore is out
___________________________________________________________________
OCaml 5.0 Multicore is out
Author : sadiq
Score : 217 points
Date : 2022-12-16 12:21 UTC (10 hours ago)
(HTM) web link (discuss.ocaml.org)
(TXT) w3m dump (discuss.ocaml.org)
| systems wrote:
| Still no native windows build To install on windows, I guess your
| best bet is via WSL (Windows Subsystem for Linux)
|
| I now run mainly on windows and this is an issue for me to try
| OCaml
| Tomte wrote:
| Do I misremember? I thought that was a goal for 5.0.
| octachron wrote:
| Not for 5.0, the aim of 5.0 was to focus on getting multicore
| support out of the door. Thus only mingw64 is supported for
| Windows. Support for MSVC will come with later versions. At
| the same time, improving opam ecosystem support for Windows
| is one of the major goal of opam 2.2 . Thus hopefully the
| support for native Windows will improve in the future (next
| year?).
| Tomte wrote:
| Thank you, that's good.
| runevault wrote:
| I saw some kind of other installer that might come with either
| MSYS2 or Cygwin I forget which, but it also said that installer
| takes 2 hours to run. They seem to strongly recommend WSL over
| it based on how I read things.
| dra27 wrote:
| opam 2.2's release cycle has fallen a bit behind the compiler's
| (actually because of the Windows support). It's an experimental
| branch, but this works with opam-repository-mingw to get a
| vanilla mingw-w64 build of OCaml 5.0.0:
|
| opam switch create 5.0
| --repos=dra27=git+https://github.com/dra27/opam-
| repository#windows-compilers --packages=ocaml.5.0.0,ocaml-
| option-mingw
| no_wizard wrote:
| Jane Street must be so happy today! I think this opens up a whole
| new host of fast applications on ocaml.
|
| I wonder how this will affect ReScript
| Alifatisk wrote:
| What an amazing ending of 2022!
|
| Just Hotwire Strada left.
| [deleted]
| runevault wrote:
| Been dabbling in f# lately, but this has me wanting to give OCaml
| a try for comparison, very interested in the new effects stuff.
| rigoleto wrote:
| I wish OCaml had something like F#'s lightweight syntax:
| https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref...
| bitbckt wrote:
| It's been tried
| https://people.csail.mit.edu/mikelin/ocaml+twt/
| runevault wrote:
| I'd never seen the verbose syntax for f#, I thought you had
| to write it the whitespace dependent way. Huh.
| cies wrote:
| ReasonML or ReScript not your cup of tea? They are different
| syntactical front-ends for OCaml.
| devmunchies wrote:
| Yes, I started in OCaml but switched to F# for the last 3 years
| and will likely move back to OCaml when effects are mature
| (when lwt and async are snuffed out).
|
| Being able to use C# libs is very nice but also it sucks that I
| don't have true null safety.
|
| Who knows, maybe F# is always more performant than OCaml and it
| doesn't make sense to switch back.
| keepquestioning wrote:
| Does it support Apple Silicon Macs?
| phplovesong wrote:
| Yes.
| logicchains wrote:
| I remember around a decade ago when I first got into programming,
| I was super excited that OCaml would soon get multicore,
| regularly checking the progress. Although it took a lot longer
| than I imagined it would, nevertheless it feels amazing to see it
| finally here, almost like a dream when you've waited so long for
| something and you can't believe it's finally happening.
| InitEnabler wrote:
| What OSS is out there that uses OCaml?
| ziotom78 wrote:
| FFTW is written in C, but it used a FFT compiler written in
| OCaml to generate snippets of C code. [1]
|
| [1] https://www.fftw.org/fftw3_doc/Generating-your-own-
| code.html
| giraffe_lady wrote:
| A lot of projects also _started_ on ocaml and then later moved
| off of it once they had succeeded by showing the concept works
| and got some momentum going. People like it for exploratory
| compiler dev, then switch off it when their language can self
| host. IIRC both rust and elm started like this, certainly
| others as well.
| Skinney wrote:
| nitpick: Elm was and is written in Haskell.
| giraffe_lady wrote:
| Oh you're right. Now that I think about it elm is more
| stylistically similar to haskell too. Last time I used elm
| I had never played with haskell so I probably just assumed
| it was related to ocaml which I did know.
| phplovesong wrote:
| Yup. Rust was originally written in OCaml
|
| https://github.com/rust-
| lang/rust/tree/ef75860a0a72f79f97216...
| abathologist wrote:
| here are a few
|
| - https://github.com/coq/coq
|
| - https://github.com/mirage/mirage
|
| - https://github.com/returntocorp/semgrep
|
| - https://github.com/bcpierce00/unison
|
| See https://v2.ocaml.org/learn/companies.html for some more
| leads, as lots of those companies maintain useful OSS software.
| actionfromafar wrote:
| The Haxe language: https://haxe.org/
|
| https://frama-c.com/
|
| https://fbinfer.com/
|
| https://mirage.io/
|
| https://coq.inria.fr/
|
| https://github.com/ygrek/mldonkey (Stale project but large
| codebase.)
|
| https://akabe.github.io/ocaml-jupyter/
|
| https://reasonml.github.io/
|
| https://github.com/moby/vpnkit (Used by Docker)
| dunham wrote:
| Also https://flow.org/
| badrabbit wrote:
| I started learning ocaml and i stopped when I heard about the
| lack of multicore support (felt like it wasn't ready for general
| purpose use) this is an awesome developemt. From all the
| functional lanuages I've sampled so far Ocaml is the most
| intuitive. Hoping to finally get into it in 2023.
|
| Great work Ocaml devs!
| hnra wrote:
| Did you compare it to F#? I'll probably try to learn an ML
| language next year and without doing much research F# looks
| really nice since it gets all the .Net benefits.
| ecshafer wrote:
| I like F# and I think its a great language. But I also think
| it will always be second class for Microsoft.
| badrabbit wrote:
| I haven't tried F# because I am already having to work with
| powershell and C# and that whole .net/visualstudio ecosystem.
| The last thing I will look at on my personal time is a
| language in that ecosystem but since you recommended it I
| will have a look.
| mbac32768 wrote:
| F# is nicer than the alternatives if you need to be (or want
| to be) on the Microsoft ecosystem but there are some things I
| miss when I use it from OCaml, like modules and functors. It
| also doesn't support named arguments, which are a fairly
| trivial thing that drastically cuts down the bugspace and
| increase usability for me.
|
| I also don't _think_ F# is as fast as OCaml?
|
| Fairly relevant piece of culture on why a company switched
| from OCaml to F# https://blog.darklang.com/new-backend-
| fsharp/
| jjtheblunt wrote:
| https://learn.microsoft.com/en-us/dotnet/fsharp/language-
| ref...
|
| might be now in there.
| vkazanov wrote:
| Amazing! An incredible amount of effort! This was holding back
| ocaml almost for as long as I remember - and now it's just gone.
|
| Congratulations to the team!
|
| What are the next plans for the project? Spread newly available
| features..?
| sadiq wrote:
| There's definitely some work to build atop the new
| functionality available in 5.0 and make sure there's plenty of
| good learning material.
|
| In terms of the compiler and runtime development, the OCaml and
| ML Workshops at ICFP in October have videos that cover some of
| the experimental work happening: https://watch.ocaml.org/video-
| channels/ocaml2022/videos and
| https://www.youtube.com/playlist?list=PLyrlk8Xaylp7f8T7L5SFF...
|
| There's also a compiler development newsletter that's posted on
| the discuss at regular intervals which details some of the
| other work happening: https://discuss.ocaml.org/t/ocaml-
| compiler-development-newsl...
| sadiq wrote:
| There are a few OCaml contributors lurking and happy to answer
| questions if you have them.
| jon_smark wrote:
| Are the plans for typed algebraic effects solidifying, or are
| they still nebulous? Concretely, are you willing to take a
| guess as to when we are expected to see OCaml 6? ;-)
| kcsrk wrote:
| Rather than a full effect system, we're very likely to have
| lexically scoped "checked" effects with the help of modal
| types. I briefly talked about it at the end of my ICFP
| keynote: https://icfp22.sigplan.org/details/icfp-2022-papers/
| 48/Retro...
|
| There are other cool stuff that is being worked on, which I
| am very excited about: https://discuss.ocaml.org/t/jane-
| street-compiler-development.... Hopefully, we will see many
| of these make it into OCaml 6.
| jon_smark wrote:
| Thanks for the reply. I hope that the array and list
| comprehensions land soon in upstream; it's a useful and
| hopefully not-too-controversial feature.
|
| I'm more ambivalent regarding the local allocations and the
| unboxed types. I totally understand why they'd be useful
| when you are trying to squeeze every last drop of
| performance, but they do require a not-so-trivial
| complexification of the language.
| kcsrk wrote:
| The local types are less invasive than the full support
| for typed effects. In particular, they are opt-in and
| associated complexity is pay-as-you-go. In my initial
| experiments, they seemed pretty nice to program with.
| octachron wrote:
| The type system for algebraic effects is still in the
| research and design phase at this point.
|
| Right now, I am not even taking a guess of what will be the
| defining new major features of OCaml 6 (effect system +
| modular implicits maybe? Maybe not?).
| jon_smark wrote:
| Thanks for the reply. I'm hoping that modular macros land
| soon. I'm very ambivalent about the PPX mechanism, and I
| hope that modular macros reduces the need of PPX.
| pdimitar wrote:
| As an Elixir (which steps on Erlang) and Rust dev I'm curious
| if you think OCaml 5.0 / Eio will give the Erlang's BEAM VM and
| Rust's tokio a run for their money in the parallel runtime
| space.
|
| I'm super curious about OCaml, picked it up and left it several
| times in the last 3 years. Now that multicore is here I'll
| absolutely be picking it up again and try to use it for
| parallel scripting and for some of my work. Great job!
| sadiq wrote:
| We certainly hope so. You may find Thomas Leonard's talk from
| last year's workshop interesting: https://watch.ocaml.org/vid
| eos/watch/74ece0a8-380f-4e2a-bef5...
|
| The paper we wrote on retrofitting effect handlers:
| https://arxiv.org/abs/2104.00250 also has some http
| benchmarks
| mbac32768 wrote:
| People who aren't PL theorists understand the multicore benefits
| even if the retrofitting paper might be over their heads. On the
| other hand, the bounding data races in time and space
| paper/presentation is also a fairly significant change that I
| don't think the average dev has an easy to understand
| relationship with.
|
| Anil covers it in a bit more plain English in this Signals and
| Threads episode. Here's a bit of the transcript, starting at
| about 50 minutes in: https://signalsandthreads.com/what-is-an-
| operating-system/
|
| > Ron: Do you have a pithy example of a pitfall in multicore Java
| that doesn't exist in multicore OCaml?
|
| > Anil: There's something called a data race. And when you have a
| data race, this means that two threads of parallel execution are
| accessing the same memory at the same time. At this point, the
| program has to decide what the semantics are. In C++, for
| example, when you have a data race, it results in undefined
| behavior for the rest of the program, the program can do
| anything. Conventionally, daemons could fly out of your nose is
| an example of just what the compiler can do.
|
| > In Java, you can have data races that are bounded in time so
| the fact that you change a value can mean later on in execution,
| because of the workings of the JVM, you can then have some kind
| of undefined behavior. It's very hard to debug because it is
| happening temporally across executions of multiple threads.
|
| > In OCaml, we guarantee that the program is consistent and
| sequentially consistent between data races. It's hard to explain
| any more without showing you fragments of code. But conceptually,
| if there's a data race in OCaml code, it will not spread in
| either space or time. In C++, if there's a data race, it'll
| spread to the rest of the codebase. In Java, if there's a data
| race, it'll spread through potentially multiple executions of
| that bit of code in the future.
|
| > In OCaml, none of those things happen. The data race happens,
| some consequence exists in that particular part of the code but
| it doesn't spread through the program. So if you're debugging it,
| you can spot your data race because it happens in a very
| constrained part of the application and that modularity is
| obviously essential for any kind of semantic reasoning about the
| program because you can't be looking in your logging library for
| undefined behavior when you're working on a trading strategy or
| something else. It's got to be in your face, at the point.
|
| (and so on)
| poulpy123 wrote:
| any easy way to try it on windows (not wsl) ?
| dra27 wrote:
| It's from an experimental branch, so not very easy, but this
| works with opam-repository-mingw to get a vanilla mingw-w64
| build of OCaml 5.0.0:
|
| opam switch create 5.0
| --repos=dra27=git+https://github.com/dra27/opam-
| repository#windows-compilers --packages=ocaml.5.0.0,ocaml-
| option-mingw
| lilactown wrote:
| Is there a guide on how to use the new constructs? The link to
| the 5.0 docs was broken on the site, and after manually fixing
| the URL all I found was some type annotations in the `Effects`
| module.
| resoluteteeth wrote:
| IIRC, while they added the underlying language support for
| effects to this version to get the multicore support working,
| the standard library support for effects isn't really ready
| yet, so you may want to hold off another year or so.
|
| I think eventually it will hopefully turn into something like
| what languages like purescript have which would be really cool.
|
| (I've only used ocaml a tiny bit and use f# a lot more but I
| keep periodically checking the status of this because it's
| something that would make ocaml a lot more interesting to me.)
| c-cube wrote:
| The Effects module is kind of low level right now, as it
| understand it. You should like at Eio for a library that gives
| you nice fibers and non blocking IOs on top of effects! It's a
| neat library.
| cies wrote:
| > like at Eio
|
| Look at Eio here: https://github.com/ocaml-multicore/eio
| anentropic wrote:
| https://github.com/kayceesrk/ocaml5-tutorial
| octachron wrote:
| The link to the Effect section of the manual at
| https://v2.ocaml.org/releases/5.0/manual/effects.html works for
| me and should contain an higher level description of what are
| effect handlers. Which link was broken for you?
| lilactown wrote:
| The "Manual" link on this page https://ocaml.org/releases is
| broken
| octachron wrote:
| This is fixed now, thanks!
| lilactown wrote:
| I also didn't think to look in the "language extensions"
| section of the manual, instead going to the API docs and
| clicking on the "Effect" module.
|
| This looks much more interesting, on a skim. Thanks!
| maattdd wrote:
| Amazing work from the team! I wonder if this is actually the
| first mainstream language which has managed to remove its "global
| lock" without breaking changes ?
| gadmm wrote:
| To clear up any misconception, out of the box OCaml will behave
| like OCaml 4 with a single domain and a "domain lock". Programs
| currently using multiple threads for concurrency will remain
| single-core for the time being, as they will need to opt-in to
| parallelism features. In this sense, adding parallelism to
| OCaml does not break existing programs, but they still might
| have to be audited for thread-safety depending on how they want
| to use parallelism. There is no magic.
| sadiq wrote:
| Just to add to the sibling comment. To maintain backwards
| compatibility, OCaml 5 has both threads and domains.
|
| Threads belong to a domain and only one thread can hold the
| runtime lock for the domain. This is the same behaviour as in
| OCaml 4.
|
| With OCaml 5 you can have as many domains as you want though
| (we recommend no more than you have cores though).
| UncleOxidant wrote:
| $ opam update $ opam switch create 5.0.0
| --repositories=default $ eval $(opam env) $ ocaml
| OCaml version 5.0.0 Enter #help;; for help.
| rg111 wrote:
| What is the most fun and best source to learn OCaml for a
| programmer?
|
| I know this one [0] so far. Also the famous Coursera PL course
| covers ML.
|
| [0]: https://cs3110.github.io/textbook/cover.html
| laylomo2 wrote:
| Congrats! Super excited to start playing around with the effects
| system.
| [deleted]
| eatonphil wrote:
| If you want to see an example of it:
| https://v2.ocaml.org/releases/5.0/manual/parallelism.html.
| nih0 wrote:
| this was being talked about when i was still in high school and
| last year i did my masters
| octachron wrote:
| The road to multicore OCaml was indeed longer and harder than
| expected. At the end of the day, the constraint of trying to
| preserve the behavior of almost all existing programs ended up
| driving a majority of design choices. Typically, this required
| at least one major rewrite of the multicore runtime along the
| way.
| rg111 wrote:
| What is the most fun and best source to learn OCaml for a
| programmer?
|
| I know this one [0] so far. Also the famous Coursera PL course
| [1] covers ML.
|
| [0]: https://cs3110.github.io/textbook/cover.html
|
| [1]: https://coursera.org/learn/programming-languages
| nequo wrote:
| Check out Real World OCaml too:
|
| https://dev.realworldocaml.org/
| melling wrote:
| OCaml has a reputation for being fast.
|
| https://sixthhappiness.github.io/articles/python-scheme-and-...
|
| Does anyone have any multicore benchmarks that illustrate
| performance increases in 5.0?
| samuell wrote:
| Great, so, can someone send me a PR with an OCaml
| implementation of "my"/our GC-content benchmark (a simple
| string proccessing benchmark counting the fraction of G:s and
| C:s in DNA sequences, compared to all the A:s, C:s, G:s and
| T:s)?
|
| https://github.com/samuell/gccontent-benchmark
|
| :D
| kcsrk wrote:
| Here are some parallel benchmarks from the Sandmark continuous
| benchmarking service:
| https://sandmark.tarides.com/?app=Parallel+Benchmarks¶ll...
| [deleted]
| jon_smark wrote:
| Congratulations and a big thank you to the OCaml team! I hope
| that multicore support finally ticks all the requirement boxes
| that had prevented many from taking a serious look at OCaml. The
| language certainly deserves it: it hits that sweet spot between
| expressiveness, performance, and pragmatism like no other.
| pjmlp wrote:
| Great news! Kudos to everyone that helped make this happen.
| tempodox wrote:
| Christmas came slightly early this year.
| octachron wrote:
| The Christmas presents will be all the issues and bugs that
| will be discovered between now and Christmas eve in the shiny
| new multicore runtime.
| aylmao wrote:
| No way, it's finally out! This is fantastic news.
| pjmlp wrote:
| Great news! Kudos to everyone that helped make this happen.
| ghostwriter wrote:
| Multicore is there, STM instead of mutable imperative logic in
| all the libraries developed for the past two decades: not so much
| though.
| c-cube wrote:
| I haven't heard anyone talk about STM for OCaml, funny. People
| talk about, or work on, lightweight fibers, lockfree data
| structures, io_uring, etc. but not STM. Is it falling out of
| fashion? Even in clojure I hear that few people actually use
| it.
| 2wrist wrote:
| Oh congratulations! Well done to all involved!
| mnming wrote:
| Is it still possible to use Reason as frontend for Ocaml 5?
|
| I felt an ergonomic and modern syntax is the only missing piece
| in Ocaml.
| nequo wrote:
| Signals and Threads had an interview with Anil Madhavapeddy last
| year:
|
| https://signalsandthreads.com/what-is-an-operating-system/
|
| He talked about the work to put a multicore-ready memory model[1]
| and GC[2] under OCaml.
|
| [1] https://anil.recoil.org/papers/2018-pldi-memorymodel.pdf
|
| [2] https://arxiv.org/abs/2004.11663
| inbx0 wrote:
| Offtopic, but Signals and Threads is my absolute favourite
| programming podcast, maybe even favourite podcast overall. They
| go into interesting topics deeply, instead of the way too
| common "interviewer read the summary of the wikipedia page
| about the subject and is now interviewing someone who actually
| read the entire page."
|
| Hoping they'll get more content out soon.
| mirekrusin wrote:
| Yaron Minsky is such a charismatic speaker, watched all his
| recordings you can find on internet more times that I'd like
| to admit, s&t is great.
| dimitropoulos wrote:
| thanks for mentioning this! I've been looking for a new one
| that's good. Even if they're behind schedule, it looks like
| I've got some catching up to do to keep me busy in the
| meantime.
| Decabytes wrote:
| I'm curious about what design decisions lead to OCaml not having
| Multi-threading when version 1.0 came out. Majorly impressive
| work getting something as complex as that added on afterwards.
| Kudos to everyone involved!
| dljsjr wrote:
| Caml 1.0 was released in '85 and OCaml (the O is for Object
| Orientation) was 1996. Multithreading wasn't a high priority
| for anybody back then. The JVM didn't even have threads until
| 1997 and those threads were green threads, OS threads came to
| Java later.
| pasc1878 wrote:
| However this is all Unix and academia based. If you wrote
| code for Windows and OS/2 in the commercial world you were
| using threads sine '89 and thus did not want to use the
| languages that did not use threads e.g. python, OCaml
| cmrdporcupine wrote:
| In the late 80s and early 90s we wrote things with threads
| but they were primarily a kind of convenience to get
| multitasking behaviour and not any kind of performance
| boost.
|
| Multicore / multiprocessor systems were not a mainstream
| thing in consumer hardware until the 21st century.
| Mikeb85 wrote:
| > I'm curious about what design decisions lead to OCaml not
| having Multi-threading when version 1.0 came out
|
| You do know that C (edit - didn't have multithreading in the
| language spec until C11, right?)
|
| It was common on languages of that era. Also Ocaml has had
| libraries for multithreading for many years, just like C has
| POSIX threads and things...
| c-cube wrote:
| I thought C11 had threads and atomic and concurrency
| primitives?
| Mikeb85 wrote:
| Ah yeah my bad. Still like 30+ years after it's creation
| though. And the vast majority of C software out there
| doesn't use C11 anyway.
| toolslive wrote:
| Halleluja! I think they were at it for more than a decade.
| ecshafer wrote:
| OCaml is one of those languages that is a real joy to use and
| makes me wonder why its not used more often. I feel like if I
| jump into a Lisp or ML language, I am so productive and can write
| some really complicated software with relative ease. But they are
| relatively rare in industry, and I have never really crossed the
| t on why, despite various reasonings about it.
|
| This is a great achievement for OCaml, but does anyone have an
| explanation on why it was so difficult to implement for them?
| c-cube wrote:
| It's just very hard to write a solid concurrent GC. There are
| not many in existence today. Here the challenge was doubled by
| the obligation to preserve the single core performance of
| existing programs, with the very fast allocation path on the
| minor heap. It's always harder to add these features to an
| existing language that already has significant programs written
| in it.
___________________________________________________________________
(page generated 2022-12-16 23:00 UTC)