[HN Gopher] Elixir and Phoenix after two years
___________________________________________________________________
Elixir and Phoenix after two years
Author : bfm
Score : 187 points
Date : 2021-04-05 17:54 UTC (5 hours ago)
(HTM) web link (nts.strzibny.name)
(TXT) w3m dump (nts.strzibny.name)
| nichochar wrote:
| I wrote elixir for a couple years in a high scale environment,
| and I agree with OP on most of what he described. My favorite
| aspects, ranked: 1) immutable data / actor model paradigm 2) mix
| (super modern build tool that does it all) 3) pattern matching
| sbaildon wrote:
| Big fan of Elixir. First and third party libraries are typically
| very high quality; community support is great; and documentation
| is best in class.
|
| Right now, I'm trying to find a way to speed up builds in CI
| because they're the biggest bottle neck to deploying. Building an
| umbrella with 5 apps, 3 of which are phoenix, leveraging
| parallelised docker buildkit, will still take 8~ minutes.
| emerongi wrote:
| Interesting. Elixir builds are the fastest builds in my CI
| pipelines. I make sure to cache the builds/ folder; usually
| only a small amount of files need to be re-compiled, which is
| very fast. This was actually improved even further in Elixir
| 1.11 [0]. Compared to TypeScript + React and Java, Elixir build
| times are significantly better.
|
| [0] https://github.com/elixir-
| lang/elixir/blob/v1.11/CHANGELOG.m...
| joelbluminator wrote:
| Oh no same crap of mixing hash symbol and string keys as in ruby?
| You didnt have to borrow that Elixir! Does Elixir also have
| HashWithIndifferentAccess?
| praveenperera wrote:
| Elixir does not have HashWithIndifferentAccess
| bglusman wrote:
| Not built in, and not _remotely_ suggesting anyone use this,
| but as an experiment I did build this a few years ago as a
| toy /to play with a few ways it might look if we ever wanted
| something similar....
| https://hex.pm/packages/indifferent_access
| https://github.com/bglusman/indifferent_access (see also
| https://github.com/vic/indifferent )
| regulation_d wrote:
| Typically string key maps should exist only at the edges of
| your system, but they are still often necessary when you're
| interacting with the outside world. Params in Phoenix come in
| as a string key map because the atom table does not get garbage
| collected, so where you don't know what keys may be passed in,
| string keys are necessary to avoid the risk of running OOM.
| That said, once I know the shape of the data I'm dealing with,
| I pretty much always convert to an atom key map as quickly as
| possible. If I see a string key outside of the context of a
| controller or worker, I am immediately suspicious.
| ch4s3 wrote:
| You really should never mix them, String keys are basically
| only there for accepting untrusted input, and should get
| sanitized. Any internal map should only have atom keys. String
| keyed maps exist so that you can accept inputs from untrusted
| sources without worrying about a malicious or malformed input
| from overflowing the atom table and crashing your VM.
| ironmagma wrote:
| Honest question, why is this a problem?
| joelbluminator wrote:
| Just google HashWithIndifferentAccess...
| JediPig wrote:
| last hype train i joined, was scala... this language reminds me
| of scala... a HORRIBLE developer experience, yet we zerglings are
| happy to be lemmings to someone's "best idea evah"
|
| This brings up the pain of scala... only this time im not going
| to join the fan club.
| davidw wrote:
| What are people finding as the real sweet spots for Phoenix?
|
| I have used Erlang very successfully in a semi-embedded context,
| but that's quite different from a web server that can usually be
| scaled horizontally pretty easily.
|
| One obvious one is if you have to hold open a lot of concurrent
| connections like web sockets. It'd be great for that. Others?
| bluesnowmonkey wrote:
| We used Elixir to implement a columnar database and Phoenix for
| the web front end, which was about 20% of the code. It's very
| convenient having all the tests run together, including end to
| end integration tests. Elixir (with NIFs) has the necessary
| performance for the database layer, and Phoenix has the
| necessary productivity for the web layer, and we don't have to
| switch languages to work on both.
| jfim wrote:
| Pretty curious about this, are there more details that are
| publicly available? The only thing I can find about this is
| some meetup from 2018 with a speaker from Pinterest.
| dnautics wrote:
| > I have used Erlang very successfully in a semi-embedded
| context
|
| Elixir is really quite good in the semi-embedded space with
| several successful companies having their IOT bread and butter
| in Elixir Nerves platform. The deployment story is getting
| really mature in Elixir.
|
| The article has some really salient points: Testing and
| Documentation and really amazing in Elixir. Concurrent tests
| are amazing. So for example normally a "database" would be a
| global resource and running concurrent tests against the
| database is really tricky. It's basically turnkey in Elixir
| (need to set up two settings).
|
| With a bit of work, it's not terribly hard to write concurrent
| tests that exit the VM and come back into the vm and use a
| test-shard of a global resource. So, for example, you write a
| test that issues an HTTP request to itself, and the request is
| instrumented with parameters to connect it back to the test
| process and use the correct "temporary shard" of the database.
| (In the case of the database it's a transaction, I use the
| phrase "temporary shard" because the same mechanism can
| extended to other concepts too, like module mocks, ets tables,
| process registries, etc, which all use the same mechanism, you
| only have to set it up once).
| davidw wrote:
| > Elixir is really quite good in the semi-embedded space
|
| Yeah, I don't need any convincing there. Erlang was a perfect
| fit for the device I worked on. High level enough to get
| things done quickly, but with a really solid, predictable
| runtime.
| lostcolony wrote:
| >> but that's quite different from a web server that can
| usually be scaled horizontally pretty easily
|
| So a lot of the other responses are comparing it against Rails
| and the like, but it sounds like you may be asking specifically
| around scaling.
|
| Erlang (and by extension, Elixir), is nice even when scaling
| because the actor model will scale to I/O or CPU bound workers
| simply, without having to tweak threadpools or worry about
| thread starvation or etc. Other languages that provide n:m
| concurrency here have the same benefit though (i.e., Golang).
|
| Where it shines in comparison to those is in its fault
| tolerance and memory model. Immutable non-shared data +
| supervisor hierarchy gives you tools to tackle state that are
| less error prone than most other languages.
|
| And the fact it has a Rails like environment in Phoenix (and
| Plug, and Ecto, and the whole ecosystem) means you get the same
| quick-to-build app functionality, without giving up the
| performance and state management.
|
| It's basically having all of these that make it desirable on
| this front; you can write code as quickly and simply as in
| Rails, getting the braindead simple scaling behavior of any
| actor/CSP based model, with the state management of an
| immutable language, and the fault tolerance of Erlang.
| davidw wrote:
| Not that concerned about scaling - I think it's going to beat
| Rails there, but for a larger web app growing quickly, the
| difference between going to N web servers from 1 might not be
| _that_ many months.
| lostcolony wrote:
| Oh, sure, Erlang isn't going to save you from going multi-
| node...in fact, it shouldn't; just basic resiliency should
| mean you're starting multi-node.
|
| But the actual number of concurrents per instance, and the
| effect on latency, is quite another thing.
|
| And it also provides you a better story around shared state
| within the instance, and better controls around its access.
| ch4s3 wrote:
| It's pretty great anywhere that you might use Rails or Django,
| but if you expect spike in traffic that are hard to predict you
| get nice stable worst case latency.
|
| I think it's also really good if you need to hold state server
| side for any reason.
| davidw wrote:
| Rails has a _ton_ of high quality code available for it. It
| looks to me like Phoenix is certainly 'good enough' for a
| lot of tasks, but it just hasn't been around as long.
|
| I'm looking for those use cases where someone picked Phoenix
| and it was just clearly a better tool than, say, Rails
| because of X, Y, and Z, despite maybe being inferior for one
| or two other things.
| cancan wrote:
| I can add a bit of my experience. I used Rails here and
| there since 2.3 and have followed Elixir since its
| inception.
|
| - Channels in Phoenix are just a joy to use compared to
| ActionCable. This is partly due to the language (pattern
| matching, especially) but not having to deal with a Redis
| instance (and/or AnyCable) is also appealing. It just works
| out of the box and is ridiculously performant.
|
| - I find the Repository pattern much easier to wrap my head
| around than ActiveRecord. I feel like with Rails, I always
| have to know the state of an object whereas w/ Ecto, things
| are more explicit. I know some people prefer AR here, but I
| prefer not making an accidental query.
|
| - I feel like I am lost every time I'm in a Rails project
| with so many abstractions now. Maybe this is me being a
| curmudgeon, but I remember being able to trace request all
| the way from where it hits the machine (say, Nginx) to the
| HTML rendering even with Rack. Now, if I had to do
| something similar, there are so many layers to peel. This
| is again partly due to the language, and partly to Rails'
| age.
| ch4s3 wrote:
| Having done both, I think Rails has a ton of baggage around
| ActiveSupport and ActiveRecord that are full of gotchas.
| Ecto prevents N+1 queries by default, which I think is
| clearly better. I also think that the lack of lifecycle
| hooks in Ecto is a better decision than the pile of foot
| guns in ActiveRecord hooks.
|
| I would also argue that Plug is a large improvement over
| Rack, and the idea of explicitly passing a single context
| map all the way through the request is just a better way to
| build http responses. I also think that the Fallback
| controller is obviously a better way to handle common
| errors.
|
| Anything related to web sockets will be leagues better in
| Elixir, because the BEAM is built to do a thing like that.
|
| SSR html as a compiled linked list is a better idea than
| runtime string interpolation.
|
| Sure, Rails has libraries for everything, but some of the
| core parts just aren't a nice. So if you don't need all of
| that breadth of ecosystem then Phoenix is a better choice
| IMHO having worked professionally with both for a number of
| years.
| news_to_me wrote:
| In terms of language, Phoenix is a complete replacement for
| Rails for me. I feel more comfortable growing a functional
| codebase, and the BEAM means I don't have to worry about
| scaling as soon as I would with Rails. I think it's a good fit
| for when you want to do more with a small, experienced team.
|
| I would reach for Rails when I'm concerned about finding
| developers (Elixir devs are fewer and more expensive,
| generally), or if there are Ruby libraries I want that aren't
| available in Elixir.
| toast0 wrote:
| I worked at a well known company running Erlang. Out of maybe
| 25 people hired to work on software in Erlang, I think two
| had used it before, and they were hired several years after
| Erlang became our key enabling technology. I was ahead of
| many because I remembered seeing the slashdot post when it
| was open sourced.
|
| If you hire smart and flexible people, and give them a bit of
| time to learn the syntax, Erlang and I assume Elixir will
| bend their mind to a new shape, and they'll be fine.
|
| Figuring out what you want the computer to do, and what steps
| will be needed are much more important than the specific
| syntax you use.
| blunte wrote:
| I fear it would be difficult to hire Elixir people as well,
| but now I realize it's actually also hard to hire decent
| Ruby/Rails people. Honestly I think we should just accept
| anyone who can demonstrate thinking and programming skills of
| any language and then plan for 2-3 months of ramp-up time to
| get them into our language of choice.
| danjac wrote:
| I'd be happy (if I were job-hunting) to be able to try out
| a different language/platform every so often. Now and again
| I'll play with, say, Ruby or Elixir or whatever. But I know
| that I'll never get a job working with these languages
| because I don't have the _n_ years experience with them.
| The tech hiring process is gruelling enough with languages
| and frameworks we are familiar with, your resume won 't
| even get a glance if you apply for jobs where you don't
| have that experience. But I think that narrow thinking is
| to everyone's detriment.
| news_to_me wrote:
| I agree, and to be clear I've never really been in a
| position to worry about hiring anyway. But I would imagine
| building a medium/large team with varying experience levels
| (including juniors) would be easier with a more established
| framework like Rails.
| toolz wrote:
| I find elixir and supporting libraries to be the best general
| web-dev experience of any language. Optional typing, first
| class documentation, ecto as a library for validations is far
| more successful at encouraging separation of concerns than I've
| seen in other CRUD webdev ecosystems.
|
| The performance for typical stateless webapps is great, but
| honestly the thing I love is the amazing tooling and libraries.
| Elixir libraries are often very high quality.
| news_to_me wrote:
| +1 for Ecto, really a game-changing library.
| blunte wrote:
| Looking purely at webapps, Liveview is the killer feature of
| Phoenix.
|
| Other than that, I would say that any place with a complex
| service oriented environment where you could leverage the
| Erlang VM would be an obvious place to use Phoenix for doing
| your webapps.
| te_chris wrote:
| Fast templates are a superpower for our CMS. Combine it with
| LiveView to eliminate react and we deliver a really good site
| quickly.
| nesarkvechnep wrote:
| I wish more Node.js people shake off their Stockholm syndrome and
| check out Elixir and Phoenix.
| heipei wrote:
| Let me tell you the single reason I haven't switched to Elixir
| yet: I develop backend and frontend (SPA) so I'd rather just
| stick with a single language and library catalog. It helps that
| there's plenty of libraries in JS land too. I would love to
| just write elixir code but at the end of the day I feel
| sticking with node+browser js is the more pragmatic choice
| right now.
| void_mint wrote:
| Isn't phoenix meant to remove the JS dependency on FE dev?
| (Not arguing with any of your points btw, JS is matter-of-
| factly more popular and pragmatic).
| datavirtue wrote:
| "Elixir is not an object-oriented language. We practically only
| write modules and functions. This helps tremendously in
| understanding code..."
|
| Sign me up. I hold hope that Elixer is the thing that starts
| pushing the knife into OOP. Microsoft has added a ton of features
| to make functional style a thing in C#, and nearly everyone hates
| Java...so maybe the stars are aligning.
| blunte wrote:
| To add to the author's experience, I spent 18 months of
| production time with Elixir and Phoenix.
|
| As he says, the templates are compiled and are blindingly fast
| compared to Rails.
|
| Pattern matching is really really nice when used in the right
| places (and you'll miss it if you go back to Ruby); but it can be
| overused. There's a faction of Elixir folks who attempt to avoid
| all conditionals and instead seem to prefer multiple
| dispatch/multi-methods to handle different cases. That's nice and
| very concise, because then you can simply call a function and let
| the pattern matching resolve which of the various implementations
| you've defined handle it. The big downside here is, as a reader
| of the code, you have to basically mentally imagine what all
| cases are covered and what they mean. Sometimes simply reading a
| switch statement or if/else/then is much clearer.
|
| The super special magic is in the Erlang VM. If you put more
| energy into learning it and its capabilities, and using it where
| appropriate, it can shape the structure of your greater system
| beyond just one webapp; and it can provide a lot of features
| without you having to cobble together many other (good but
| independent) solutions.
|
| Lastly, single thread performance is basically a dog. In my
| anecdotal experience, the same external service written with
| Elixir+Ecto was 25-50% as performant as a Python+SQLAlchemy
| program. So the lesson there is, find ways to parallelize or
| otherwise scale your process if it is batch oriented and handling
| a large volume of data.
|
| If you asked me today if I would prefer to use Elixir (and
| Phoenix) over Ruby and Rails, I would say yes... but honestly
| mostly just because it's a new fascination with different
| tradeoffs and a better functional story. Function is the past and
| the future, and it makes your life easier and simpler. Elixir as
| a language... borrowed too much from Ruby and has too much
| syntax. It is noisy in a Perl-like way, and perhaps there could
| be a more concise enhancement of Erlang which would get the job
| done and not have you spending time visually parsing code.
| ch4s3 wrote:
| > In my anecdotal experience, the same external service written
| with Elixir+Ecto was 25-50% as performant as a
| Python+SQLAlchemy program
|
| I'd be interested to see an example here. Using Plug+Ecto has
| been roughly on par with most real world examples I've seen
| with SQLALchemy and Flask for single request performance.
| Python might be a bit faster for some workloads but once you
| start saturating the CPU, Elixir/BEAM really shines.
| blunte wrote:
| I should have been clearer in my final note. My Elixir vs
| Python example was for standalone batch (script) processing.
| It was not in the context of a webapp.
|
| The case was a recurring calculation system which would take
| a few hundred thousand records and do about a dozen different
| calculations against those (some calculations requiring
| additional lookups).
|
| The original script was in Python, and then after the new
| webapp was built in Elixir, I rewrote it also in Elixir. I
| discovered that even tuning the batch sizes, the Elixir/Ecto
| version was much slower than the Python version. Then I
| looking into per-thread Elixir performance and found out that
| that is not its strong suit :). So the answer there of course
| is to leverage the strength of the tools and parallelize it;
| but I had no need to do that since I already had a working
| Python version. Could Elixir have been faster if multi-
| threaded? Probably. Single? No, I doubt it.
| ch4s3 wrote:
| Ahh, ok. You've sort of hit a weird spot for Elixir. Python
| will just start up faster and that will be noticeable. Math
| is much better optimized in Python, though this is changing
| with NX.
|
| The real gain here would probably be streaming chunks of
| rows out of ecto and maybe firing those off to tasks per
| CPU core. Which is easier than it sounds.
|
| That said, you had working code so no real reason to
| rewrite it except as a learning exercise.
| dnautics wrote:
| Hm, if you have a chance to revisit it, you may want to
| consider Flow, which will likely take a lot of the pain of
| parallelization away.
| ksec wrote:
| >Lastly, single thread performance is basically a dog.
|
| Is that still the case? I thought BeamVM recently added JIT? I
| don't expect it to be LuaJIT or JS V8, but Ruby and Python
| Single Thread performance should be reasonable expectation?
|
| I also wish some of these experience has more context in terms
| of code base size and team size. A Production environment of a
| small project with a team of 2 is very different to production
| environment of a project that is large and team of dozens if
| not hundreds.
| toast0 wrote:
| I've seen several reports of pretty good performance
| improvements from the new JIT, but keep in mind a couple
| things:
|
| It's included in the not yet finalized release that is
| currently only a release candidate. It's a little early to
| expect people to have experience with it. Some organizations
| might update quickly, but others will take quite some time.
| Also, the JIT isn't on all supported platforms; i think it's
| amd64 and aarch64 only at the moment, which probably covers
| most performance oriented servers, but maybe not everyone.
|
| The other thing is it's not an optimizing JIT like Hotspot or
| v8; it 'only' turns the beam opcodes into native code as it's
| loaded. This eliminates interpretation overhead, but there's
| potential to optimize the native code in the future.
| blunte wrote:
| In my case it was a team of 1 working on OTP 19 and 20. Since
| that is now 3+ years ago, it is quite possible things have
| improved in the single-thread performance area.
| onlyrealcuzzo wrote:
| Elixir single thread performance /is/ much faster than Ruby
| and Python [1].
|
| Are you sure you weren't just doing something sub-optimal
| with Ecto?
|
| [1] https://amp.reddit.com/r/elixir/comments/46v1l1/is_elix
| ir_fa...
| astrowilson wrote:
| The 3+ years ago is a good reference, thanks for sharing.
| The issue could also be Ecto related. Ecto 3, which might
| not have been out at the time, had a bunch of improvements
| on this front.
| astrowilson wrote:
| > and perhaps there could be a more concise enhancement of
| Erlang which would get the job done
|
| Erlang already is a more concise language than Elixir but also
| noisier as there are more punctuation characters. I would love
| if Erlang would drop some of its punctuation, as some of it is
| frankly unnecessary.
|
| Elixir syntax is definitely simpler than Ruby's. Probably in
| the same ballpark as Python complexity wise: Elixir has less
| keywords and less rules thanks to the macro system but on the
| other hand more affordances, such as optional parenthesis.
| dmitriid wrote:
| > Erlang already is a more concise language than Elixir but
| also noisier as there are more punctuation characters.
|
| What?
|
| Here's more-or-less the entirety of Erlang's "noisier syntax
| with more punctuation characters" -spec f(A
| :: any(), B :: some_type(), C :: list()) -> any(). f(A,
| {B}, [C | _]) -> A1 = fun() -> io:format(a, []) end,
| A1(), case C of <<1, _/binary>> -> 1;
| _ -> #person{ok = ok, field = C} end.
|
| Edit: %% plus map syntax #{"tuple"
| => {1,2}} M#{"key" := "new_value"} %% plus
| list comprehensions [X || X <- List, X /= 10].
|
| Elixir has _all that_ plus more. The equivalent in Elixir is
| something along the lines of @spec f(a ::
| any(), b :: some_type(), c :: list()) :: any() def f(a,
| {b}, [c | _]) -> a1 = fun() -> IO.inspect(a) end,
| a1.(), case C do <<1, _ :: binary>> -> 1
| _ -> %Person{ok: :ok, field: c} end end
| ## and don't forget the capture and pipe syntax
| x |> (&f(y, {&1}, list)).() ## and default
| function parameters def f(a, b, c \\ []), do:
| something() ## and sigils
| String.replace(str, ~r".*", "") ## and Ecto adds
| its own characters id = 1 (from p in
| Person, where: p.id == ^id, select: p) |> Repo.all()
| ## and dropping down to Erlang is a function call on an atom
| :io.format(x, y)
|
| And I'm definitely forgetting more...
|
| Edit: ## and string interpolation
| a = 1 s = "The value is: #{a}" ## and
| two different map syntaxes %{a: :map, with:
| "various", keys: "as", atoms: 0} %{"another" =>
| "map", "but" => "keys", "are" => "strings"} ## and
| different map access map[key] map.key
| ## and map update %{ map | key: "new_value" }
| ## and list comprehensions for x <- list, x != 10
| do end
| ch4s3 wrote:
| > Sometimes simply reading a switch statement or if/else/then
| is much clearer
|
| Case statements are pretty common, and for code I read the with
| statement[1] is even more common. This allows you to code a
| happy path broken into small steps and then collect your edge
| cases. It doesn't solve every case for if/else or cases, but
| it's a nice tool.
|
| [1]https://hexdocs.pm/elixir/Kernel.SpecialForms.html#with/1
| strzibny wrote:
| I love with, completely forgot to mention it.
| conradfr wrote:
| > attempt to avoid all conditionals
|
| I think that's a symptom of the lack of a return keyword (and
| therefore early returns).
| toolz wrote:
| sqlalchemy is an ORM - I'm not sure it can be compared against
| a web framework, but my experience with phoenix vs python web
| frameworks is that phoenix is easily faster even for single-
| threaded web requests (which of course will utilize many
| threads for things like DB thread pooling etc.)
| waynesonfire wrote:
| i'm not a web developer but enjoy paying attention to this space
| from the sidelines.
|
| elixir and phoenix are wonderful. the phoenix liveview is a
| fantastic piece of technology; it moves the processing to the
| backend and allows the web application to be developed in the
| same language (mostly).
|
| i just recently discovered microsoft blazor and it seems like
| it's an even better improvement. compared to liveview, the
| computation is moved back to the client while the development
| experience is still a single language. there is no more
| javascript (at least that's the claim) and the platform takes
| advantage of webassembly to deliver a high performance UX. really
| compelling stack. I hope it continues to drive the innovation in
| web development. I'm so excited to see the javascript eat dust.
| Maybe light at the end of the tunnel to a horrible 10+ year
| period of web development.
| devoutsalsa wrote:
| I'm an Elixir developer, and I love the language, but I'm not
| sold on Phoenix LiveView. It sounds really cool, but it seems
| like there are too many edge cases. How do you scale it
| horizontally? What happens to user state when a connection is
| dropped? What do you do when you get a business requirement
| that hits one of LiveView's pain points? What if you have to
| swap out your backend for business reasons, and now you have to
| rewrite your frontend, too? I'm not trying to talk anyone out
| of LiveView; these are just the thoughts that prevent me from
| getting excited about it.
| mstipetic wrote:
| I have a dream that one day most of our software will be built
| with something like elixir and most of the saas products we're
| using now (octa, zapier, newrelic...) will be just installable
| modules with a simple api
___________________________________________________________________
(page generated 2021-04-05 23:00 UTC)