[HN Gopher] Switching from pyenv, rbenv, goenv and nvm to asdf
___________________________________________________________________
Switching from pyenv, rbenv, goenv and nvm to asdf
Author : tosh
Score : 420 points
Date : 2022-04-05 10:42 UTC (12 hours ago)
(HTM) web link (jinyuz.dev)
(TXT) w3m dump (jinyuz.dev)
| js2 wrote:
| > Everything was fine until I kinda felt my config.fish file got
| bloated and somehow nvm was kinda slowing down my shell startup.
|
| I'm quite happy with direnv to avoid this problem.
|
| https://direnv.net/
|
| Funny enough, it was submitted to HN most recently just
| yesterday:
|
| https://news.ycombinator.com/item?id=30908321
|
| direnv is complementary to the various language-version tools, so
| it obviates needing asdf:
|
| https://github.com/direnv/direnv/wiki#project-layouts
|
| But it seems some folks combine the two:
|
| https://blog.mikecordell.com/2021/12/18/better-project-envir...
| wyufro wrote:
| Indeed, direnv and asdf integrates, such that PATH is set to
| the values exposed by your .tool-config. This avoids the asdf
| shim and makes executing the tools slightly faster. Good when
| you have to run them over and over in a script.
| kot-behemoth wrote:
| I use direnv myself, and it's great. However, for anyone trying
| it out I'd caution that it doesn't play well with conda - the
| Python package management system (see this open issue
| https://github.com/direnv/direnv/issues/326). It can be a
| dealbreaker for some.
| sph wrote:
| direnv is a killer tool in my arsenal. Writing 12 factor apps
| without it (especially managing configuration) is an exercise
| in pain and frustration.
| qbasic_forever wrote:
| direnv is nice but it really makes me uneasy if it catches on
| for modern collaborative development.
|
| You are giving people a _ton_ of trust when cloning a repo or
| pull request that has a .envrc which will be automatically
| executed by direnv. This file could do anything from steal your
| SSH keys to delete your entire home directory or worse. And the
| only thing preventing that from happening is the suggestion of
| "oh be sure to eyeball the .envrc file and make sure it looks
| ok, then run this command to enable it". People will just
| google for the first "how to I unblock .envrc file" and blindly
| run the command, then oops. Or a determined attacker will just
| obfuscate what they're doing so the .envrc looks fine but you
| failed to realize some esoteric bash-isms were actually
| invoking an obfuscated script and... oops. It's just begging
| for a supply chain disaster on a similar scale as npm's
| infamous issues.
|
| VS code and some vim extensions have similar issues with
| automatic project-specific configurations that allow arbitrary
| code execution too, and neither have a good solution beyond
| 'just read all the code and understand it to make sure it's not
| going to hurt you'. It's not really a unique thing to direnv.
|
| We really need a better model for collaboration and consistent
| dev environments. I think containers and docker/chroot/etc.
| environments with explicit opt in of your local resources,
| configs, files, etc. are a much more sane and security
| conscious path towards it. A script executing to setup my
| python environment for a project should never, ever have access
| or even know about my personal password store database for
| example (unless I explicitly decide it should).
| Nextgrid wrote:
| Direnv maintains a local cache of the scripts it's allowed to
| execute. It will not execute a brand new script and instead
| ask you to manually run "direnv allow".
| qbasic_forever wrote:
| Yes, that's why I explicitly mentioned this:
|
| > And the only thing preventing that from happening is the
| suggestion of "oh be sure to eyeball the .envrc file and
| make sure it looks ok, then run this command to enable it".
| People will just google for the first "how to I unblock
| .envrc file" and blindly run the command, then oops. Or a
| determined attacker will just obfuscate what they're doing
| so the .envrc looks fine but you failed to realize some
| esoteric bash-isms were actually invoking an obfuscated
| script and... oops.
|
| You need to be absolutely sure you've read and understand
| every single line in the .envrc file, and every single line
| in any script it may be invoking, before you run direnv
| allow. Otherwise you are giving code from the internet a
| blank check to do anything on your machine with your user
| account.
|
| I personally do not think bash/shell is a sensible language
| to trust that you can quickly or easily determine from a
| visual check alone if a script might be doing nefarious
| things.
| Nextgrid wrote:
| I use direnv to source environment variables - the only
| scripts I use it for consist of "export" statements.
| Anything beyond that and I'll be questioning it.
|
| Sure, you can trick idiots into running malicious code,
| but I don't think you need direnv for that - any pip/npm
| install can run anything and so does curl|bash.
| hardwaresofton wrote:
| I don't include the env file in the source, but rather
| include a sample in the README.
|
| If I wanted to include it, I use something like git-crypt and
| encrypt the contents
| DandyDev wrote:
| direnv actually won't run anything until you explicitly allow
| it to. And if the .env file changes, you have to re-allow
| direnv to read and execute it.
|
| Seems safe to me
| qbasic_forever wrote:
| Unfortunately like the other comment you didn't read what I
| said and my concern about this approach. Please see my
| response: https://news.ycombinator.com/item?id=30922199
| smokey_circles wrote:
| what in the actual horrifying toolset is this.
|
| pyenv I can understand. dunno the others.
|
| but goenv?? you deserve whatever hell this is. it's a single
| binary configurable with an envvar for gopher sake. just RTFM.
| trey-jones wrote:
| I see comments about Nix and I've heard rumblings about it
| before, so I might look into that. But Docker covers my needs for
| version management completely. To me it seems much more explicit,
| more portable, less error prone, less complex than any of these
| virtual environment managers. Never mind that I don't even want
| node.js on my system if I can help it at all (I can't,
| generally).
| hushpuppy wrote:
| Docker doesn't allow for LSP servers in my text editor. Asdf
| "just works".
| trey-jones wrote:
| This is a reasonable point, and something that is being
| worked on. I definitely think Docker _can_ provide LSP
| servers as necessary, we 're just not quite there. This is
| something that I'm interested in working on, though I'm not
| the only one: https://github.com/emacs-lsp/lsp-docker
|
| For the most part, latest version linting is good enough for
| me, currently.
| dlisboa wrote:
| It's obviously doable, but it's _very_ heavy for languages.
| Like someone else said, you immediately run into basic issues
| like readline, LSP compatibility, volume persistence, port
| forwarding, etc.
|
| Docker is better for version managing services like databases,
| caches (Postgres, Redis, etc), things that have little need for
| different configuration or exploration and persist through
| multiple sessions. Once they're up, they're done.
|
| Asdf is the next best thing for local development, where you
| need that kind of exploration.
| rkangel wrote:
| The main issue with docker is that you have to start from
| scratch with your environment. With Nix you can layer it over
| your existing Linux environment (or you can have it from-
| scratch by running `nix-shell --pure`).
|
| There are advantages to the docker approach, and Nix can be
| painful to get going, but a significant portion of the
| community (including me) find the pain to be worth it.
| samgranieri wrote:
| I've used ASDF since 2016 and it's been working beautifully. The
| only hiccups I've had with it aren't related to ASDF, rather
| stuff like Erlang 24.0 not compiling on macOS (which was
| subsequently fixed). Every engineer at my company uses ASDF
| vlunkr wrote:
| Same. We switched when we started an elixir project and pretty
| quickly switched all our codebases over. On top of languages
| we've used it for database systems in some cases.
| freedomben wrote:
| asdf is indeed the best solution in the elixir/erlang world.
| I use it for node, ruby, and python now too and it's been
| great to have a consistent experience across languages.
| vmception wrote:
| asdf for solution to package manager hell? missed opportunity for
| naming it fgsfds
| blunte wrote:
| Happy asdf user for several years now. Unfortunately, there are
| still cases where it either can't be used or it would take a
| great deal of hacky stuff to make it work. One such case is
| getting PyCharm to understand it.
|
| I found it works well for Ruby, Elixir/Erlang, and Node.
| pmontra wrote:
| And PostgreSQL too. I tend to use the PostgreSQL plugin for
| projects in which I use asdf. It creates the data directory
| somewhere and I can use that exact version of the database
| without docker. Then it's pg_ctl start / stop from within the
| project directory. The version coming with the OS is never the
| right one.
| burke wrote:
| At Shopify, we do a similar thing using a tool we wrote called
| Shadowenv[1], which evaluates little programs to manipulate the
| environment on directory entry/exit. It's a lot like direnv, but
| it can't do anything that makes it slow, and is guaranteed to run
| in at most 100ms (though it typically only takes a few)
|
| When I set `ruby: 2.6.8` in my `dev.yml` file, for example, I get
| this Shadowenv program written out to
| `.shadowenv.d/550_dev_ruby.lisp`: (provide
| "ruby" "2.6.8") (when-let ((ruby-root (env/get
| "RUBY_ROOT"))) (env/remove-from-pathlist "PATH" (path-
| concat ruby-root "bin")) (when-let ((gem-root (env/get
| "GEM_ROOT"))) (env/remove-from-pathlist "PATH" (path-
| concat gem-root "bin"))) (when-let ((gem-home (env/get
| "GEM_HOME"))) (env/remove-from-pathlist "PATH" (path-
| concat gem-home "bin")))) (env/set "BUNDLE_PATH" ())
| (env/set "GEM_PATH" ()) (env/set "GEM_HOME" ())
| (env/set "RUBYOPT" ()) (env/set "RUBYLIB" ())
| (env/set "RUBY_ROOT" "/opt/rubies/2.6.8") (env/prepend-
| to-pathlist "PATH" "/opt/rubies/2.6.8/bin") (env/set
| "RUBY_ENGINE" "ruby") (env/set "RUBY_VERSION" "2.6.8")
| (env/set "GEM_ROOT" "/opt/rubies/2.6.8/lib/ruby/gems/2.6.0")
| (when-let ((gem-root (env/get "GEM_ROOT")))
| (env/prepend-to-pathlist "GEM_PATH" gem-root)
| (env/prepend-to-pathlist "PATH" (path-concat gem-root "bin")))
| (let ((gem-home (path-concat (env/get "HOME")
| ".gem" (env/get "RUBY_ENGINE") (env/get "RUBY_VERSION"))))
| (do (env/set "GEM_HOME" gem-home)
| (env/prepend-to-pathlist "GEM_PATH" gem-home)
| (env/prepend-to-pathlist "PATH" (path-concat gem-home "bin"))))
|
| I really like this strategy. It's really nicely generalizable to
| any of these language runtimes that need to change on a per-
| project basis.
|
| [1]: https://shopify.github.io/shadowenv/
| ravi-delia wrote:
| Oh I like the look of this. Can't wait to try it out
| this_is_not_you wrote:
| I've wasted so much time trying to get pyenv to work properly and
| failed. Switched to asdf and things just worked.
| rsanheim wrote:
| I switched to using asdf away from nvm/rbenv for a few projects
| in about a couple hours of time, and that included the
| install/build time for various versions of things and the time to
| re-run bootstrap and verify things worked from a fresh clone.
|
| It was really impressive how smooth and quick the process was.
| Great tool!
| twblalock wrote:
| It seems like a lot of these virtual environment tools are
| designed to work around the assumption that packages are
| installed globally on a system.
|
| I don't see why this is so hard to avoid with per-project version
| pinning. I've never had this problem with Java.
|
| This seems like a lot of effort and complexity to work around
| global package installation. Just stop assuming that packages are
| global and this would all go away. How do we know that? Because
| other languages don't require this kind of workaround!
| nicoburns wrote:
| It's less about package versions (which are indeed versioned on
| a per-project basis in sane ecosystems - python being a notable
| exception) and more about versions of the language toolchain
| itself. e.g. you might want one project on Java 17 and another
| on Java 8. These tools allow you to seamlessly switch between
| those.
| twblalock wrote:
| Java tooling already provided that and it's trivial to
| install multiple JDK versions on a system. I probably have 4.
| Most IDEs have a dropdown to pick the one you want, there are
| CLI tools for it too, and overall it's about as complex as a
| $PATH variable.
|
| It's also possible to install multiple versions of Python on
| a system. The dependency management is the real problem --
| having more than one version of a runtime is not the hard
| part.
| nicoburns wrote:
| If Java provides good first-party support for this then it
| may not make sense to use asdf for Java. Rust is similar in
| that rustup is first-party the recommended tool to install
| Rust with. I don't bother with asdf for Rust. Other
| ecosystems like python/ruby/javascript don't provide this
| as a first party tool meaning there are lot's of 3rd party
| alternatives. For these ecosystems asdf is nice in that it
| allows you to use one tool across all the ecosystems.
|
| > The dependency management is the real problem
|
| I'm not sure how this works in Java, but in many ecosystems
| like JavaScript and Rust dependency management isn't an
| issue at all. Installing a package defaults to a local
| install and even things like dependencies transitively
| depending on different versions of the same package are
| supported out the box and just work.
| smoe wrote:
| What do you refer to by saying Python is an exception? I
| haven't used anything but different language and package
| versions on a per-project basis since around 2007. I had some
| project with deployments on client servers where their
| sysadmin insisted on having everything installed as os
| packages, but this at least from my perspective hasn't been
| the norm for quite a while now.
|
| Not to say that I like the Python package management story.
| nicoburns wrote:
| Commonly used tools like virtualenv allow you to do this,
| but it's not the default in the python ecosystem: `pip
| install foo` will install foo globally. In most other
| ecosystems the equivalent command (e.g. npm install foo)
| defaults to a local install with no effort on your part.
| tootie wrote:
| Java (well Maven) I believe does this the most clever and
| simple way. It does have a global folder of dependencies but
| will gladly stash multiple versions of the same library. Local
| projects just link to the right version. That saves downloads
| and allows for simple version pinning. And JVMs are backwards
| compatible so having a new JVM can still run bytecode for past
| versions with just a flag. Features like this are so much more
| valuable than anything that goes into the actual language
| syntax.
| npteljes wrote:
| I come from the Java world and it's mindblowing to see that
| mature, widespread languages like Ruby, Python or JavaScript
| require all kinds of hackery to run properly. Dependency
| management with Maven is a breeze in comparison.
| __MatrixMan__ wrote:
| I only spent two years in contact with Java. I was
| initially very pleased with how easy dependency management
| was but it soured over time.
|
| I'd respond to production disasters with hacky bash scripts
| and then we'd spend the next sprint incorporating that
| functionality into the server so the hacks weren't
| necessary. We kept catching ourselves working really hard
| to optimize what we thought at first needed to be custom
| code but what ended up being easily achievable by leaning
| on well-known non-java tools like `diff`. There's something
| about the way Javs encapsulates everything into a JAR that
| discourages Java devs from using anything on the outside.
|
| If you can live in a world where everything you need is in
| the JAR, life is good, but that hasn't frequently been my
| world.
| tootie wrote:
| I spent many years working with Java and never used bash
| for deployment. Maven covers all the bases. And when you
| have something really custom, you write a maven plugin.
| Which can also be managed with Maven.
| __MatrixMan__ wrote:
| Oh the bash scripts had nothing to do with deployment,
| they just fixed the data that the buggy deployment
| created.
|
| It's only relevant because the high level of
| encapsulation that made deps easy to manage also
| represented a barrier. Using diff and awk was fast
| enough, the reimplementation in java wasn't (happened
| just a few times). Language ecosystems with a more
| permeable encapsulation style make it easier to reference
| tools outside of that ecosystem at the cost of making it
| harder to get your deps straight.
| robertlagrant wrote:
| Bear in mind basically one guy maintains the Python package
| installer and default packages repo, or at least he did for
| many years. It's open source.
| Tainnor wrote:
| What do you mean by Ruby "hackery"?
|
| I'll agree that how to do things properly with Ruby isn't
| obvious when you're new to the language - just as it isn't
| obvious how to do things in Java (or other JVM languages)
| when you're coming from Ruby or another language.[0]
|
| But Bundler doesn't work all that differently from Maven,
| in the end. Instead of running your commands through
| `./mwnw`, you run them through `bundle exec`[1]. Dependency
| management is straightforward and mostly hassle-free (the
| problems arise "only" when you use C extensions which, to
| be fair, is not that rare, but I don't really see a way
| around this in a language where you need C for certain fast
| things). In particular, Bundler by default[2] _also_ stores
| dependencies centrally like maven to avoid duplication and
| then selects the appropriate version of a given dependency
| (unlike e.g. NPM which stores everything locally).
|
| Rbenv (or alternative tools such as chruby or asdf) solves
| a problem completely distinct from Bundler or Maven, it's
| about selecting _the right Ruby version_ for your project,
| a problem that also needs to be solved with Java. IME, many
| developers just let the IDE deal with that, but Ruby
| developers are more used to running things from CLI, so
| that 's why rbenv exists. In any case, the most popular
| analogue for Java would be sdkman which works, mostly, the
| same way (and which, IME, is even a bit hackier than
| rbenv).
|
| JS and Python are different. Node is a mess IMHO, and
| Python isn't able to agree on which package manager to use
| (I would hope people would migrate to Poetry, which seems
| to be doing things right).
|
| [0]: For example, one thing that I find very annoying over
| in Java-land is that it's a real hassle to make sure things
| behave the same way when called from the CLI as opposed to
| from the IDE, such as setting the active spring profile.
| This is because, mostly, the IDE circumvents maven/gradle
| (you _can_ run things through the build system, but
| typically the output, e.g. test output, is better when
| using the IDE-internal system, at least with IntelliJ).
|
| [1]: There are "hackeries" to get rid of having to run
| everything through "bundle exec", such as binstubs and
| direnv, although I would mostly call them "shell script
| tricks", and anyway, if you instead just alias "bundle
| exec" to something shorter or create a wrapper script, it's
| not really worse than `./mwnw`.
|
| [2]: This can be overriden and is actually a good idea for
| production builds, especially if you use a multi-stage
| docker build since then it's easy to copy over all your
| dependencies.
| npteljes wrote:
| By Ruby, Python and JavaScript hackery I was meaning rvm,
| venv / pip, and npm. I have the most experience with rvm,
| and the hackery part comes from the experience when I
| created a web interface that could instruct the server to
| do ruby tasks. Turned out that rvm (and also bundler)
| does a bunch of stuff to the environment of the user,
| that's sometimes inaccessible, even if you invoke a
| "login shell" programatically. One of these "hacks" is
| doing a bunch of stuff when the user navigates to a Ruby
| project directory with cd, which is when rvm creates or
| uses the existing virtual env for the project - named is
| a specific way based on the directory name. I solved this
| of course but the discovery was quite annoying.
|
| Regarding Python, I don't have much experience, other
| than the general feeling that it's fragile and that
| specific projects work with specific dependency
| managegers / virtual environments, and they are not
| really interchangeable or anything. I either could
| replicate the env or I couldn't. I definitely don't have
| much of a Python experience.
|
| Now regarding your Java IDE annoyance, I share the same
| experience. Also the case with Tomcat. IDEs do so much
| hackery so that user experience is convenient, that you
| would be hard pressed to replicate the results, and often
| need to test both because it just doesn't work the same
| way, due to the different dependencies used, deployment
| method etc.
|
| Now that I think about it, every single one of the
| programming ecosystems I used were quite annoying.
| Tainnor wrote:
| > By Ruby, Python and JavaScript hackery I was meaning
| rvm, venv / pip, and npm. I have the most experience with
| rvm, and the hackery part comes from the experience when
| I created a web interface that could instruct the server
| to do ruby tasks. Turned out that rvm (and also bundler)
| does a bunch of stuff to the environment of the user,
| that's sometimes inaccessible, even if you invoke a
| "login shell" programatically. One of these "hacks" is
| doing a bunch of stuff when the user navigates to a Ruby
| project directory with cd, which is when rvm creates or
| uses the existing virtual env for the project - named is
| a specific way based on the directory name. I solved this
| of course but the discovery was quite annoying.
|
| Keeping the "right context" when you do things in
| different directories with the same shell config is,
| unfortunately, a problem that I don't think is easy to
| solve. My preferred approach is to use direnv
| (https://direnv.net/), but that's not perfect either.
|
| As for rvm, I think that a lot of people at some point
| discovered that rvm was maybe doing a bit too much (in
| particular, it would also manage gem sets, something that
| seems superfluous with Bundler). I think that's a reason
| why rbenv and chruby emerged as lighter-weight
| alternatives which are _a bit less_ magic. I haven 't
| seen many people use rvm anymore in recent years. By
| contrast, Bundler doesn't really do anything to your
| shell.
|
| > Now that I think about it, every single one of the
| programming ecosystems I used were quite annoying.
|
| That's something I agree with. As a more recent convert
| to the JVM world (and mostly just for lack of better
| alternatives), I'm still sometimes annoyed at the IDE-
| centrism (and UI-centrism, see also sonarqube) in this
| space. Luckily, things are already better than they were
| when I learned Java in university and "it runs in my IDE"
| was the default state of affairs...
| npteljes wrote:
| Thanks for the suggestions. I'll look into direnv, as I
| only heard about it, but never really investigated.
|
| If people don't use rvm, do you know what they use to
| install a specific Ruby version? We basically use it so
| that we can use a Ruby that's not bundled with the
| distros anymore (large legacy project).
| tootie wrote:
| I think you're missing that Maven does actually solve the
| rvm/nvm/pyenv problem. You can specify the bytecode
| version in the config and compile/run any backward
| compatible version. Your actual JDK need only be new
| enough, not an exact match and it doesn't need to change
| between projects. I have never had a single compiler or
| runtime version compatibility issue even with native
| dependencies ever with Java. The IDE situation can have
| some subtle edge cases, but no other language (save C#)
| actually gains so much utility from an IDE. Stack traces,
| break points, runtime inspection and profiling are all
| nearly flawless in IntelliJ (or Visual Studio) in ways
| that are monstrously difficult with any dynamic language.
| Tainnor wrote:
| Yes, but now you're not comparing maven vs. Bundler,
| you're comparing statically vs. dynamically typed
| languages. It's almost impossible to get the strong
| guarantees you get in a statically typed language in a
| dynamically typed one.
|
| If that's the discussion you want to have, well, I
| personally also prefer statically typed languages, but
| that's beyond the point.
|
| That said, I've _had_ issues, e.g. on CI, that didn 't
| manifest locally because of a mismatch in Java versions
| (something with JAXB, IIRC), and Java is maybe sometimes
| a bit too cavalier about the issue.
| oftenwrong wrote:
| The Maven ecosystem partly solves the "the _right_ java "
| problem, although it's not necessarily well known and not
| particularly great.
|
| 1. For selecting the correct JDK to use with standard
| plugins, including the compiler plugin, there's support
| for toolchains:
|
| https://maven.apache.org/guides/mini/guide-using-
| toolchains....
|
| 2. For ensuring Maven itself is run with the correct JDK,
| it is a commom recommendation to use the Enforcer plugin
| to specify the required JDK version and/or vendor, as
| well as the Maven version:
|
| https://maven.apache.org/enforcer/enforcer-
| rules/index.html
|
| 3. It is increasingly common to use Maven Wrapper to
| supply Maven, which you seem to be aware of.
|
| 4. Maven will run with the JDK pointed to by JAVA_HOME,
| of course.
| twic wrote:
| So is asdf good then? I have been using SDKMAN! to install JDKs,
| but i noticed that Gradle will automatically detect JDKs
| installed using SDKMAN!, asdf, and Jabba [1], which signals to me
| that the other two are worth a look. Although now i dig into it,
| asdf does not have a first-party Java plugin, so the support is
| through a third-party plugin maintained by three people and a
| robot [2].
|
| [1]
| https://docs.gradle.org/current/userguide/toolchains.html#se...
|
| [2] https://github.com/halcyon/asdf-java/graphs/contributors
| e12e wrote:
| TBH I've had a bit mixed experiences with asdf and java (mostly
| big enterprise apps like Oracle SQL Developer needing some
| hand-holding to work properly, and issues with http kit
| intercepting network traffic used by SoapUI).
|
| Not sure the error lies at the feet of asdf, though.
|
| For ruby/node it's a relief to be able to easily "activate" a
| certain version of ruby _and_ a certain version of nodejs - at
| the same time, via simply "asdf local ruby ... and asdf local
| node ...".
| SingAlong wrote:
| I wouldn't go by whether a plugin is first-party or third-
| party.
|
| The only plugins under the asdf-vm github org are for Ruby,
| Node.js, Elixir and Erlang, because those are the only
| languages I needed.
| vips7L wrote:
| Jabba never worked on windows for me. So now I just install
| jdks globally via scoop and have a powershell plugin that
| automatically detects a .env file in each directory.
| kbd wrote:
| Tried asdf but can't stand the global shims. I only want it to
| work on a per-directory basis with direnv. I tried to follow the
| asdf-direnv instructions but IIRC still wound up with asdf's shim
| directory in my global PATH, and it needed to install its own
| copy of direnv. Disappointing that asdf expects to take over my
| system like it does.
| vorticalbox wrote:
| the nodejs plugin fails on version asdf v0.7.8 use v0.8.0 or
| higher to avoid the unbound variable error
| oauea wrote:
| The link to adsf-vm just points to Google. Is this a broken link,
| or just wasting the readers time by telling them to "just google
| it"?
| tosh wrote:
| https://asdf-vm.com
| rubyist5eva wrote:
| asdf is wonderful, been using it for 2 years and get everyone I
| see using one of those single-language tools to switch and they
| also love it.
|
| if asdf could have a plugin that hooks into their shims to also
| handle direnv I would be in heaven
|
| edit: just did a quick google search and it exists, i'm elated
| orsenthil wrote:
| I like this quote in asdf help.
|
| "Late but latest" -- Rajinikanth
|
| https://github.com/asdf-vm/asdf/blob/master/lib/commands/com...
|
| Thalaiva! :)
| whalesalad wrote:
| I think of myself as a pretty experienced hacker but asdf has
| never really worked for me on any platform. It's always a kludge.
| Love the idea, but I'm sticking with pyenv for now.
| this_is_not_you wrote:
| Funny, had the exact opposite experience. I could pyenv not get
| to work with my Linux VM and VSCode terminal (zsh).
| CraigJPerry wrote:
| I just raided the project git repo and there is definitely no
| common lisp in there. Unfortunate name clash!
|
| https://asdf.common-lisp.dev/
| chobytes wrote:
| My first thought as well! From the title I was intrigued to see
| how CL was being used for this.
| omaranto wrote:
| Renaming because of this name clash has been suggested but the
| suggestion wasn't taken up:
|
| - https://github.com/asdf-vm/asdf/issues/383 -
| https://github.com/asdf-vm/asdf/issues/547
|
| They did do something to ameliate this though: now they refer
| to the proect as asdf-vm as much as possible, to distinguish it
| from Common Lisp asdf.
| peterhil wrote:
| My first reaction was to see if the project is made with
| Commnon Lisp's ASDF, and cool now it has plugins for different
| languages. Common Lisp ASDF is very well thought out, but not
| the easiest package management system.
| ravi-delia wrote:
| Yeah I was really wondering who found it easier to hack ASDF
| to work with other languages than just use a few separate
| package management solutions. It's an incredible library, but
| man I would not want to push its limits or it would go from
| odd to unusable pretty quick.
| ClumsyPilot wrote:
| I am sorry for off-topic, but it i show this title to a normal
| person, they will conclude that our industry is a joke.
| tptacek wrote:
| I get pyenv and rbenv. I do not get why anyone would use goenv.
| I'm sure there's some corner-casey reason, but it's not a
| mainstream thing. I can't imagine working in Ruby or Python
| without wrappers, but Go, just keep the most recent version
| installed and that's pretty much it. If you're new to Go and
| installing something like goenv because you had to do that for
| all the other languages... reconsider!
| EscargotCult wrote:
| +1 the only thing that's "surprised" me between Go releases has
| been their changes to build flags in 1.18, and gofmt takes care
| of that anyway. Now that I think of it, gofmt changes if you
| have autoformatting enabled is probably the only "breaking"
| change that might happen between minor releases, but I haven't
| seen any issues with this from 1.15 through 1.18. In any case,
| if gofmt changes are disruptive enough to block upgrading, then
| a team has some deeper-seated issues in how they ship.
| ryanschneider wrote:
| I'm not sure if this is a feature or bug but `asdf` requires
| you to recompile every tool installed via `go install` (nee `go
| get`) when you change go versions.
|
| IIRC the main reason our team started tracking go versions was
| we were already using `asdf` for tracking other tools and it's
| a nice way to ensure that everyone/everything was using the
| same to version (e.g. for unit tests which usually aren't run
| in a container) but it is another spot to keep in sync with the
| Dockerfile.
| wyuenho wrote:
| asdf is not a replacement for pyenv, rbenv, goenv and nvm, it's
| yet another abstraction on top of them. Nvm is a fork of rbenv
| for node, and pyenv is a fork of rbenv for python, the core of
| all of these asdf plugins are exactly ruby-build, python-build
| and node-build, which came from rbenv, pyenv and nodeenv. For go,
| you don't need any kind of environment, you can switch between
| versions with just a change of an environment variable. Also, a
| much simpler solution is to simply install these language runtime
| versions with your favorite package manager like Homebrew or
| Macports, and use direnv and dotenv to switch between them as you
| cd into different directories. All of these languages have
| convenient environment variables you can control to put the right
| things in your paths.
| jitl wrote:
| Nit: nvm is fork of rvm. nodenv is fork of rbenv. The rbenv
| model works much better than nvm model because it uses PATH
| executables instead of heavy shell functions.
|
| Regarding the suggestion to "do things manually" -- what is the
| value of doing things manually? For me, it is very far away
| from how I want to spend my energy, which is doing computer
| things that only I can do. I am happy to outsource environment
| variable fiddle faddling to someone else.
| wyuenho wrote:
| Configuration is a necessity for any tooling. Direnv is
| something you likely already use with dotenv anyway to
| configure your CI/CD pipelines. All you need to do is to drop
| in a few more extra environment variables into your git
| worktree.
| lapser wrote:
| > asdf is not a replacement for pyenv, rbenv, goenv and nvm,
| it's yet another abstraction on top of them
|
| No, asdf is a replacement for them. It's not an abstraction on
| top of them. It uses its own plugin system, and everything is
| written purely in bash. Everything gets installed in
| `~/.asdf/installs`
| wyuenho wrote:
| Their "plugins", are using exactly the same build plugins
| pyenv, rbenv and nodeenv use to build the language runtimes.
| All asdf does is set the point things to the right
| directories using various tricks like symlinks and shims and
| whatnot. This is something easily accomplished by direnv.
| rubyist5eva wrote:
| https://github.com/asdf-community/asdf-direnv
|
| > asdf version resolution is slow which makes every command
| execution pay that penalty. asdf reshim is needed for
| finding new executables, and some tools are not happy with
| their executables being masked by shims.
|
| > Perform asdf version resolution only once and defer
| environment loading to direnv.
| wyuenho wrote:
| Exactly the reason to use direnv directly.
| rubyist5eva wrote:
| but then I still have to use some kind of package manager
| to manage multiple language runtime versions and
| brew/apt/dnf are all woefully inadequate
| mjs wrote:
| I also prefer `direnv`. I am not entirely sure how `asdf`
| actually works; `direnv` is pretty straightforward: it hooks
| into the shell and modifies environment variables. (`PATH`, but
| also tool-specific variables if required.) So it's pretty easy
| to debug or replicate by hand if necessary.
|
| It can't actually install different versions though (looks like
| `asdf` can?), which is somewhat inconvenient.
| wyuenho wrote:
| Just install them with Homebrew or Macports. They all get
| installed into different paths anyway and you don't even need
| to build from source on your system. Building python and node
| are major PATA.
| vlunkr wrote:
| > install these language runtime versions with your favorite
| package manager like Homebrew or Macports
|
| Package managers often don't have a way to install specific old
| versions of a package, or multiple versions of a single
| package. That's the whole point.
| wyuenho wrote:
| Yes there is, but may not be granular enough to the patch
| level, they usually end at the major or minor level. But then
| again, I've never ran into or heard of anyone that needs a
| very specific semver of a language runtime for a project. It
| could happen, and when it does, you switch to pyenv, rbenv or
| nvm. There's still not a reason to use asdf, as a poster
| pointed out, asdf version resolution is slow and you need to
| reshim, which is yet another manual intervention you'll need
| to do.
| bin_bash wrote:
| I used asdf but found it caused a 200ms hit to launching any of
| the binaries: https://github.com/asdf-vm/asdf/issues/290
|
| Personally that's not something I can live with as these binaries
| are sometimes launched many times in subprocesses.
| lapser wrote:
| Seems the direnv plugin aims to resolve, but I struggled to get
| it working. 200ms for hasn't been a huge issue (yet), but not
| having the proper paths set up has been. Using shims isn't the
| greats experience (something direnv plugin also aims to
| address).
| thejosh wrote:
| direnv worked for me on both linux/mac.
|
| They just released a major new version (asdf-direnv), so
| might be worth giving that a shot and see if it works for you
| with asdf?
| noitpmeder wrote:
| Direnv is actually amazing
| samgranieri wrote:
| yeah, I love direnv combined with bundler binstubs. saves
| me from having to type bundle exec (whatever), or even
| using a shell alias for it
| bin_bash wrote:
| I had the same experience.
|
| To be clear since I think it might not be obvious: lapser is
| talking about asdf-direnv, not direnv by itself. See
| https://github.com/asdf-community/asdf-direnv
| tomatowurst wrote:
| wow, this needs to be top comment. This is a no go for me.
| since I use only pyenv and nvm, not too big of a problem.
| lapser wrote:
| I mentioned in my other comment that the direnv plugin aims
| to resolve this. Personally I didn't get it working but I
| haven't been determined enough. It has been working for
| others.
| vbrandl wrote:
| If you are looking for a language agnostic version manager, the
| nix package manager [0] might be worth a try. In combination with
| lorri [1] you will be dropped into a shell with all required
| development dependencies available, when you enter the project
| directory. This does not only include the interpreter/compiler
| versions but also any other dependency you can think of, like
| specific libraries the project links against (well almost any, it
| has to be available as a nix expression, best case directly in
| upstream nixpkgs).
|
| For me, it has come so far that I don't have any interpreters or
| compilers in my main OS environment and even for one off REPL
| sessions, I'll use `nix shell nixpkgs#python3 -c python3`.
|
| [0]: https://nixos.org/
|
| [1]: https://github.com/nix-community/lorri/
| giancarlostoro wrote:
| I mean to try it but some of these version managers work on
| Windows too, I believe Nix is specific to Linux. Lorri sounds
| amazing though! It's the kind of thing that made me fall in
| love with solutions such as virtualenv (which handles the "this
| directory needs these specific packages" installed locally, not
| globally problem that Debian gives you).
| baryphonic wrote:
| I believe Nix works on top of WSL. And IIRC, it can even
| handle building Windows GUI exe's, though that may be
| mistaken.
| vbrandl wrote:
| One thing that differentiates nix from things like virtualenv
| is that each (package + version) tuple exists at most once in
| your local nix store and is then linked into your project
| environments, while virtualenv will install new copies of the
| packages. On the other hand you have to run the nix garbage
| collector to get rid of unused packages.
|
| As for Windows support: It seems to be possible to use nix in
| WSL [0] but I've never tried that. I have used nix alongside
| apt and pacman on Ubuntu/Arch before I decided to go all the
| way and install the NixOS distribution. Using it alongside
| other package managers worked really well.
|
| [0]: https://nixos.org/download.html#nix-install-windows
| mauflows wrote:
| I've been running nix on wsl and works really well
| chriswarbo wrote:
| > I believe Nix is specific to Linux
|
| I've used Nix on Linux (i686, x86_64 and ARM), macOS and
| Windows (WSL2). I believe it also works (although not
| officially supported) on BSDs, etc. too, although I've not
| tried that yet.
|
| > Lorri sounds amazing though! ...which handles the "this
| directory needs these specific packages" installed locally,
| not globally problem that Debian gives you).
|
| You're describing Nix, not Lorri. Specifically:
|
| - nix-shell provides a directory-local environments (and
| more, e.g. #!nix-shell to fetch script dependencies
| automatically)
|
| - direnv can enter/exit this nix-shell environment when cd-
| ing in/out of a directory
|
| - lorri makes direnv+nix-shell faster, by building the
| environment asynchronously in the background (rather than
| causing our terminal to freeze)
| Aleksdev wrote:
| Looks cool! However it appears to have a bit of a learning
| curve to get up and running.
|
| Are there any other advantages to using nix?
| vbrandl wrote:
| In the end you could use nix to build docker/VM images and
| use the same, reproducible environment in
| prod/test/CI/dev/...
| rsanheim wrote:
| Yes! You can swoop into any thread discussing other packaging
| / build tools, and recommend nix, immediately sidelining any
| conversation about the original topic.
| nvarsj wrote:
| shell.nix + nix install is _the_ killer linux dev tool imo. It
| replaces all the various nonsense with a hermetic, fully
| reproducible build environment. You can then use the same
| shell.nix in your CI with the nix docker image. Now you have
| exactly identical dev environments across CI and locally. It's
| awesome once you've experienced it.
|
| The biggest drawback is installing Nix on OS X is a huge pain.
| Docker has lots of magical UX tooling to make it rather
| seamless in comparison.
| ris wrote:
| > The biggest drawback is installing Nix on OS X is a huge
| pain
|
| Did it on an aarch64 mac the other day. One command (two?),
| 15 minutes, done.
| mixedCase wrote:
| Installing NixOS on modern macOS nowadays is super trivial.
| Copy+paste command from website, follow instructions,
| optionally enable flakes in a config file. That's it.
| gmmeyer wrote:
| I've used Nix. It solves an interesting problem. It is so
| complicated as to be unusable by someone who is just looking
| for a quick and simple solution to something like a virtual
| environment
|
| There's a reason why, despite being almost 20 years old, Nix
| has at best a small cult following. It just doesn't solve the
| problems that most people are looking to solve, most other
| solutions are simply easier and in many ways better
| Fnoord wrote:
| Its not even 15 years old.
| 3836293648 wrote:
| 2022 - 2006 = 16
| kristjansson wrote:
| Here's a use case I pose every time Nix comes up in these
| discussions.
|
| For $REASONS I want python 3.7.10, and the full scientific
| stack (numpy, scipy, pandas, numba, matplotlib, ...) at the
| latest compatible versions along with pytorch. I don't mind
| waiting for things to compile. How, if at all, can Nix help me
| get into an environment that meets those constraints? Can it
| help without requiring me to find and copy hashes of things?
| zeec123 wrote:
| https://github.com/nix-community/poetry2nix
| ris wrote:
| > requiring me to find and copy hashes of things
|
| Put an incorrect hash in. Run the build. The error will tell
| you what hash it actually got. Copy that. Run build again.
| amelius wrote:
| This is how I use Conda, which uses libsolv under the hood to
| figure out compatible versions.
|
| This makes me wonder: does Nix use libsolv, or a similar
| idea?
| viraptor wrote:
| One place where asdf wins in convenience is when you actually
| really care about versions though. It allows you to install
| both new and old specific versions. In nix, while you can
| achieve that, you'll be guided to use the currently supported
| major versions like python 3.8, 3.9. Making sure you build with
| exactly 3.7.1 as well as a module which was merged to nixpkgs
| later is a serious pita.
| mixedCase wrote:
| It should not be. You're able to easily pin multiple specific
| nixpkg versions in a flake.
| viraptor wrote:
| And you can do that in theory, but in practice it ends up
| like this:
|
| - you want to use lang-x.y.z, so you'll just reference an
| old commit
|
| - turns out that version was never packaged, so you copy
| x.y.z-1 to your flake/overlay
|
| - you end up patching the build because it doesn't work by
| default with new environments
|
| - on the other hand a module you're using needs the new
| environment so you do your best to make everything happy -
| more overlaying
|
| - because you changed something crucial in a few steps,
| you're compiling llvm 20 times through this process and now
| it's 1am
|
| Compare that to putting "rust 1.59" in your .tool-version
| and running "asdf install". Guess who tried to use 3
| specific versions of rust at the same time recently and got
| very familiar with the process...
|
| (That's also ignoring the issue like Ruby being packaged
| without the platform identifier in nix which makes it
| unusable for some development purposes)
| speed_spread wrote:
| Yup, the versions of the toolchain components required
| for a project to build should be specifiable at the
| project in an format actionable by the main build tool.
| Running a project should never require more than "git
| clone $project && $build-cmd".
|
| It should never be Nix's business (or any other OS) to
| provide elements of a project's toolchain. Reciprocally,
| projects should go out of their way to not depend on OS
| installed toolchain components.
| mixedCase wrote:
| > It should never be Nix's business (or any other OS) to
| provide elements of a project's toolchain.
|
| Why? I used to think the same way, but that's because of
| the scar tissue of all the tooling being terrible at
| accepting that build concerns and packaging concerns are
| one and the same.
|
| Nix proves it and makes this ancient pain a non-issue:
| "My package depends on x, y and z, additionally, building
| it requires a, b, and c, please also make build
| dependencies available in my shell while I write code
| too". That's it.
|
| > Running a project should never require more than "git
| clone $project && $build-cmd".
|
| That's precisely what Nix allows you to achieve.
| Meanwhile doing this:
|
| > Reciprocally, projects should go out of their way to
| not depend on OS installed toolchain components.
|
| Means that you can never depend on anything you're not
| vendoring (read: maintaining a copy of, on your own). Or
| it means sacrificing productivity in the altar of shitty
| tools. Nix allows you to have your cake and eat it too.
| speed_spread wrote:
| Shared projects should not depend on a particular package
| manager. You may use Nix, but the next person might use
| something else because reasons. If you bind a project's
| lifecycle to a specific OS you'll be trading your
| convenience for those of others.
| viraptor wrote:
| I think you're missing the point of nix. Unlike other
| systems, yes you can make it mix's business to provide
| that. And the threshold for "OS installed" becomes blurry
| enough that projects can include the definition of what
| they depend on from the system.
| speed_spread wrote:
| You're missing my point, that building a public project
| from source should not require you to have a specific
| package manager installed.
| mixedCase wrote:
| If it's Rust, you can use
| https://github.com/oxalica/rust-overlay to get any
| version you want very easily without pinning an instance
| of nixpkgs just for it.
|
| asdf does not allow you to keep three different versions
| of the same language, so I'm not sure how that compares?
| It's not super-trivial to do in Nix, but at least you can
| do it.
|
| asdf is also no different than Nix when it comes to
| minor/major versions. You're at the mercy of what the
| plugin does, other than that you have to create your own
| plugin from scratch or make a fork. Nix has the option to
| patch things up more easily at least.
| viraptor wrote:
| > asdf does not allow you to keep three different
| versions of the same language
|
| Sure it does. You can have any number installed and
| either switch by changing the tools version file, setting
| it for your shell, or referencing the right path
| directly.
|
| > You're at the mercy of what the plugin does
|
| Again theory vs practice. In theory, sure, it depends on
| the plugin. In practice, almost every plugin lists every
| single released version to choose from. Which is much
| different from nix where the official choices are limited
| and you have to work to enable the rest.
| mixedCase wrote:
| > You can have any number installed and either switch by
| changing the tools version file
|
| Oh, that's cheating :) Nix can do the same thing!
| Derivations are just kept in the store until you garbage
| collect them. You can also configure multiple versions
| _at the same time_ to present you with multiple binaries,
| which is what I was referring to.
|
| > Again theory vs practice. In theory, sure, it depends
| on the plugin. In practice, almost every plugin lists
| every single released version to choose from. Which is
| much different from nix where the official choices are
| limited and you have to work to enable the rest.
|
| In practice, nixpkgs supports every maintained version of
| most tools. Certainly far more tools than asdf can ever
| hope to maintain. If not, there's an overlay or flake out
| there to support it. Can you come out with specific
| examples and a usecase?
| acatton wrote:
| My issue with Nix is thatm like docker, it is a dumb CLI which
| communicate with a powerful daemon running as root.
|
| I don't understand the concept/motivation behind it, everything
| that nix does should be able to be done in userland, like
| podman does, which I love.
| hamandcheese wrote:
| Because the nix store is global per system, it would be
| unsafe to allow any user to write to the nix store in a
| multi-user system. So the daemon instead mediates writes to
| the store.
|
| I'm not sure if there are other reasons.
| viraptor wrote:
| You can run nix without any daemon.
| https://nixos.org/manual/nix/stable/installation/single-
| user... It's not a requirement apart from multi-user
| environments.
| georgyo wrote:
| Indeed!
|
| The daemon is not powerful at all, almost all the logic
| happens in the client.
|
| The client does the parsing, job ordering, and tells the
| daemon exactly what to do.
|
| The daemon, only required in multiuser environments, does
| only two things really.
|
| - ensure that the /nix/store is protected as the daemon
| should be the only process able to _write_ to it.
| Everything can read from the nix store; no nix required at
| all.
|
| - execute build commands in a sandbox. Providing isolation
| between multiple builds.
| burke wrote:
| The daemon is actually very dumb. It's just there to keep
| /nix/store permissions clean.
| qudat wrote:
| Based on other comments like this, I gave it a try.
|
| Be prepared to sink dozens and dozens of hours learning the
| language, it's features, and switching everything over. Be
| prepared to get stuck when you are required to use an old
| version of a language and things don't install correctly -- and
| sometimes blocked by nix itself from installing it. Be prepared
| to get stuck if you are not in the happy path. Be prepared to
| get lost in the sometimes scant documentation.
|
| I love the idea of nix, but it did not work for me except for
| the simplest cases.
| packetlost wrote:
| Not as bad as the guix crowd IMO. They're like the nix/Rust
| "just use XYZ" crowd but also combined with the GNU zealots
| so you get the perfect overlap of annoying software purists.
|
| Really, I like the idea and I actually like the fact that
| they use Guile (scheme) instead of a home-built DSL, but guix
| is a horrible, poorly documented, doesn't work even on the
| happy path mess.
| sph wrote:
| I honestly never heard anyone on HN recommending Guix over
| Nix.
| packetlost wrote:
| I'm explicitly _not_ recommending it. I _want_ to like
| it, but it doesn 't give me any reason to
| b1n wrote:
| New GUIX user here. I've found the documentation to be good
| and the "zealots" seem very friendly - maybe take another
| look.
| robertlagrant wrote:
| Never heard of guix til now. An annoying piece of software
| with a cult-like following does sound up my street, though.
| soraminazuki wrote:
| I have the exact opposite experience. Nix is the best
| solution out there _especially_ in the sort of situations
| that you describe.
|
| Nix provides a level of flexibility that other solutions
| simply do not offer. If a package in Nix doesn't fit your
| requirements for whatever reasons, you can create a modified
| version of a package with ease. For example, say that you
| need a version of Nginx built against a custom version of
| OpenSSL. You can do just that with a few lines of code:
| let mypatch = pkgs.fetchpatch { url =
| "https://example.com/bugfix-for-openssl.patch";
| sha256 = "..."; }; openssl =
| pkgs.openssl.overrideAttrs (old: { # add build
| flags configureFlags = old.configureFlags ++ [ "
| --enable-foo" ]; # add dependencies
| buildInputs = old.buildInputs ++ [ pkgs.foo ];
| # add patches patches = old.patches ++ [ mypatch
| ]; }); in nginx.override { inherit
| openssl; }
|
| Nix will even know which packages it'd need to build locally
| instead of downloading a prebuilt binary.
|
| You can't do that with other common package managers. You're
| stuck with whatever the package manager provides you. So if
| you're not in the happy path, you're out of luck. You'd
| either have to give up or build from scratch. If you need a
| bugfix for a particular package, you'd have wait until the
| fix reaches the package repository. When I was using Ubuntu,
| that was often until the next major release. None of this
| stuff is a problem with Nix, which allows for customization
| with very little effort.
|
| It's easy to learn about Nix if you know where to look for.
| Nix Pills [1] would be a good start. The core language is the
| easiest part. It's JSON, but with functions and variables for
| proper abstraction. The documentation, while imperfect, is
| quite extensive compared to a majority of other high profile
| open source projects. If the documentation fails you, the
| Nixpkgs repository [2] is an even more rich source of
| information. The code is well organized, and I was able to
| get familiar with writing Nix packages fairly quickly by
| grepping the codebase. And finally, Nix has a sizable
| community so you could always ask if you're stuck.
|
| [1]: https://nixos.org/guides/nix-pills/
|
| [2]: https://github.com/NixOS/nixpkgs
| xrd wrote:
| This is the best comment I've read on nix yet!
|
| I've been using nix as my daily OS for six months and I'm
| able to be productive. At the same time, I'm often confused
| as to whether I should use nix-shell, nix-env or modify my
| configuration files.
|
| Thanks for the great comment.
| SkyMarshal wrote:
| I'm new to NixOS too. My general rule is to try to keep
| my core system build/description as minimal as possible.
| Default to nix-shell first, and only if I'm using that
| thing repeatedly do I then test-install it with nix-env,
| and if that succeeds then add it to configuration.nix.
| qudat wrote:
| Your example looks great! Indeed I trolled every piece of
| nix documentation I could find -- it was the only way to
| figure out what I should be doing.
|
| Here's the simplest example of an utter failure when I
| tried nix: installing ruby 2.2. It's very possible I did
| something totally wrong, but when I _have_ to use ruby 2.2
| and nix cannot do it, it sort of kills your awesome example
| of being able to override openssl.
|
| Another extremely frustrating experience was the
| bifurcation between nix and nixos. Clearly one experience
| is preferred over the other. So many blogs and docs talk
| about things like HomeManager and other configurations that
| are nixos specific.
|
| On top of nixos/nix, there's also shell.nix/flakes. There's
| just too much development in nix right now to make sense of
| what's going on.
| SkyMarshal wrote:
| Did you ever try using Ruby 2.2 in an isolated nix-shell?
| That's arguably the best feature of nix-shell, to create
| an isolated environment with only the packages you need
| for that particular project.
| Kaze404 wrote:
| You don't need NixOS for Home Manager. It's actually the
| first chapter in the installation section: https://nix-
| community.github.io/home-manager/index.html#sec-...
| soraminazuki wrote:
| > when I have to use ruby 2.2 and nix cannot do it
|
| Considering that ruby 2.2 was released in 2014, I can't
| say I'm surprised. It's too much effort to continue
| maintaining something so old. But if you really want to
| use it, you could try using the package definition from
| an older commit of Nixpkgs: let
| nixpkgs1709 = pkgs.fetchFromGitHub { owner =
| "NixOS"; repo = "nixpkgs"; rev =
| "e09c0adc63d10249dac8f90313f91e1050861d3c";
| sha256 =
| "sha256-Di9D0gvaESV3JmX/kW2uEJ68QDlAke23t19bImTXsJ8=";
| }; pinnedPkgs = import nixpkgs1709 {
| }; in pinnedPkgs.ruby_2_2
|
| > Another extremely frustrating experience was the
| bifurcation between nix and nixos.
|
| I'm not sure what you mean. Nix is the package manager
| while NixOS is an entire distro based on Nix the package
| manager and its configuration language. They have clear
| separation of concerns. Keeping it that way has actual
| benefits too, namely portability. It allows you to use
| Nix on a wide range of platforms including non-NixOS
| Linux distros, macOS, and even BSDs to some extent.
|
| > On top of nixos/nix, there's also shell.nix/flakes.
|
| Flakes are currently an experimental feature and not
| meant for wide use yet. So if you're just getting started
| with Nix, I'd recommend looking into it later until
| you're sufficiently comfortable with Nix.
|
| They're not fundamentally different, though. Flakes
| formalize the conventions for writing Nix expressions to
| make them more reusable.
| mikepurvis wrote:
| Flakes are actually part of the piece for how to sanely
| get access to old versions of things, as your project
| flake can bring in multiple instances of nixpkgs from
| different "stable" branches and then pull the specific
| packages off each one that you need.
| darkwater wrote:
| > It's easy to learn about Nix if you know where to look
| for.
|
| I understand you are saying it with the best intentions but
| it might sound like not the most correct way of defending a
| tool being criticized for its lack of documentation or easy
| of use.
| soraminazuki wrote:
| I feel that the criticism regarding documentation is too
| harsh. Speaking from experience, Nix has more
| documentation and community resources than most open
| source projects that I've encountered.
| jahewson wrote:
| Most open source projects have a woeful lack of community
| resources and documentation though, so that's not really
| saying much.
|
| Ultimately documentation doesn't solve usability problems
| it just makes note of them.
| bastardoperator wrote:
| What problem is this solving that a shell script couldn't
| handle for you? I don't know anything about nix, genuinely
| curious.
| SkyMarshal wrote:
| The example above is part of a declarative, reproducible,
| full system build. You include it in your build
| description file, and then Nix will rebuild the entire
| system in the exact same way across multiple machines.
|
| That's less of a guarantee with a collection of shell
| scripts.
| soraminazuki wrote:
| With a shell script, you get to enjoy all the pains of
| building software _and_ its dependencies from scratch.
| Even then, you wouldn 't be even halfway done. You'd also
| need to go through the process of turning your ad-hoc
| procedures into a reproducible script only to find out
| that it broke months later.
|
| Nix provides better abstractions and composability to
| make that experience far less painful. Note that in my
| previous example, I didn't need to specify how to build
| OpenSSL or Nginx. I was able to reuse the definition from
| Nixpkgs and apply my own customizations to it.
| X6S1x6Okd1st wrote:
| It is both great and terrible, a shining example for me was
| that I wanted to install a package built off a specific
| commit. That was trivial, I just over rode the source and
| the expected hash and the exact same build & install steps
| worked.
|
| Also rollbacks in nixos are amazing.
|
| On the other hand newly packaging something can be really
| daunting.
| flurie wrote:
| I think an easy trap with nix is the immediate desire to have
| it manage everything, which usually involves installing
| NixOS. It's the equivalent of being airdropped into the
| Romanian countryside with working knowledge of French. I am
| trying to adapt nix for use with some teams I'm consulting
| on, and my goal is to make its usage as dead simple as
| possible for really high value tasks.
|
| I also feel like nix is immediately tangible and beneficial
| in a way that makes us want to get to the power user level,
| but I don't feel like wanting to learn, say, kubernetes means
| one automatically wants to learn to be a cluster operator.
| The scope and complexity are great, and I have experienced
| all of the things you have described as well. Now that I am
| fairly confident in my abilities, I think it's important to
| make sure we find ways to help everyone get the most out of
| it, even if it doesn't involve scaling the wall that is the
| learning curve.
| xpe wrote:
| > I think an easy trap with nix is the immediate desire to
| have it manage everything, which usually involves
| installing NixOS.
|
| Can you clarity what you mean?
|
| What do you mean by "trap"? Which (if any) of the following
| concepts factor into your understanding? Broken
| expectations? Premature optimization? Over-engineering?
| Lock-in?
|
| Are you suggesting that many users of the Nix package
| manager "fall into a trap" and begin using NixOS?
|
| What do surveys and/or data show about usage of just the
| Nix package manager? Combined with NixOS?
| flurie wrote:
| Absolutely, and if it was not clear from the context, I
| call it a trap because that feels like what I'd fallen
| into at some point. Let me see if I can describe it
| better.
|
| The Nix ecosystem is attractive to me at a high level
| because it looks like it has the possibility to
| revolutionize a lot of the problems that come up with an
| increased focus on devops culture. The problem with the
| Nix ecosystem itself is that it is larger and more varied
| than it might appear.
|
| There are really three large entities in the ecosystem:
| nix the language, nix the package manager (nixpkgs), and
| nix the OS (NixOS). Despite the fact that there is a lot
| of overlap in some of the details of these three things,
| they are easily understood as separate entities. There
| are a bunch of smaller secondary entities such as nix-
| darwin, home-manager, devshell, and digga (formerly
| devos). The aim of these entities is usually to
| supplement, enhance, replace, or reproduce features of
| nixpkgs and/or NixOS.
|
| Each of these entities has its own learning curve. It
| takes surprisingly little nix knowledge to use it as a
| dev environment generator, but nixpkgs knowledge will
| help a bit more. Even within nixpkgs itself, each major
| language has a large ecosystem built up around
| facilitating package building and deployment, and these
| are extremely heterogeneous. Some of them require that
| you regenerate the entire matrix of package dependencies
| for the package manager (javascript, haskell). Some of
| them have tooling that allow you to get started with
| almost no effort (mach-nix for python). These all have
| their own peculiarities. NixOS has a ton of modules, all
| rather heterogeneous, on top of requiring more careful
| knowledge of how to put together a system than most other
| distros. And nix the language without nixpkgs is _very_
| small, and some of what it does have built in is
| necessary so it can bootstrap an actual stdlib from
| nixpkgs.
|
| The trap is sprung once you've been sold on the idea, and
| even seen it shine through some things, but feel like you
| have an entire wall to climb to get to a point of general
| competency. All of these things I've described, and even
| some of their subgroups, require new knowledge for a more
| complete understanding. I do not understand the details
| of many of the things I have described, and I would
| consider myself a power user at this point. What looks
| like one gigantic, insurmountable cliff is really dozens
| of much smaller cliffs, many of which are not necessary,
| at least not to most people.
|
| At one point I wanted to test my skills in a really
| generic way, so I helped fix broken packages on macOS
| during one of the two yearly regression periods where the
| community tries to get the CI builds passing. I picked
| the packages completely at random. Most of what I ended
| up fixing was C/C++ based, and I don't normally work in
| it, or I haven't for many years, but I feel like I
| learned a good bit about cmake. I don't really write
| Haskell, but I was able to fix a bug in a solving library
| having to do with a missing portability shim. I didn't
| have help for any of this, and in some cases I was either
| too stuck to fix the build or unable to fix it due to
| problems beyond my control.
|
| All of this is to say that I think getting to this point
| is a lot less painful if one takes a use case, preferably
| a narrow one, and focuses on learning whatever is
| necessary in the service of that use case. Some good
| topics are dev environments, using nix as an app/project
| builder, using nix to build containers, using nix to
| manage remote machines (the difference between these and
| personal machines is that the scope of these machines
| tends to be much smaller). I think people try to take on
| too many of these things simultaneously and end up
| overwhelmed. That's not to say that NixOS isn't worth
| learning, but I think many people could realize most or
| all of the real-world benefit if they use nix on an OS
| they're comfortable with. Coming back to my comparison of
| being dropped in the Romanian countryside, if you want to
| learn Romanian, start there. If you want to experience
| the countryside, maybe a guided tour is the way to go. If
| you want to learn about the life of Ligeti, you might not
| even need to learn Romanian! But total immersion is
| difficult and draining, and a failed immersion experience
| doesn't mean Romania is a bad place. So it is with Nix.
| lloeki wrote:
| Many use cases can be covered just by simple uses of
| nixpkgs, but it's often alluring to go beyond that. e.g
|
| package manager (whether on Darwin or any linux distro),
| to have some tools globally available:
| nix-env -iA nixpkgs.${some-package}
|
| per project, most shell.nix can just look like this:
| { pkgs ? import <nixpkgs> {}, }:
| let some_var = some_value; in
| pkgs.mkShell { buildInputs = [
| pkgs.some_package ... ];
| shellHook = '' # this is bash, so, whatever
| floats your boat '''
|
| e.g asdf, only much more generic with full non-leaking
| package management: { pkgs ?
| import <nixpkgs> {}, }: let
| some_package = pkgs.some_package_1_2; in
| pkgs.mkShell { buildInputs = [
| some_package pkgs.some_other_package
| ... ]; shellHook = ''
| export SOME_VAR="some_value" ...
| ''; }
|
| ruby/rvm/rbenv/bundle exec (example for rails >= 6):
| { pkgs ? import <nixpkgs> {}, }:
| let ruby = pkgs.ruby_2_7; python =
| pkgs.python27; node = pkgs.nodejs-14_x;
| in pkgs.mkShell { buildInputs = [
| ruby pkgs.sqlite python
| node pkgs.nodePackages.yarn ];
| shellHook = '' export RUBY_VERSION="$(ruby -e
| 'puts RUBY_VERSION.gsub(/\d+$/, "0")')"
| export GEM_HOME="$(pwd)/vendor/bundle/ruby/$RUBY_VERSION"
| export BUNDLE_PATH="$(pwd)/vendor/bundle"
| export PATH="$GEM_HOME/bin:$PATH" ''; }
|
| python/pyenv/venv: { pkgs ?
| import <nixpkgs> {}, }: let
| python_packages = python-packages: [ python-
| packages.pip ]; python =
| pkgs.python38.withPackages python_packages; in
| pkgs.mkShell { buildInputs = [
| python ]; shellHook = ''
| export PYTHON_VERSION="$(python -c 'import platform;
| import re; print(re.sub(r"\.\d+$", "",
| platform.python_version()))')" export
| PIP_PREFIX="$PWD/vendor/python/$PYTHON_VERSION/packages"
| export
| PYTHONPATH="$PIP_PREFIX/lib/python$PYTHON_VERSION/site-
| packages:$PYTHONPATH" unset SOURCE_DATE_EPOCH
| export PATH="$PIP_PREFIX/bin:$PATH" '';
| }
|
| mixing arm64 and x86_64 on an Apple Silicon machine:
| { x86_64 ? import <nixpkgs> { localSystem =
| "aarch64-darwin"; }, aarch64 ? import <nixpkgs>
| { localSystem = "x86_64-darwin"; } }: let
| foo = aarch64.foo; in aarch64.mkShell { # this
| makes nix-shell drop to an arm64 shell, change it to
| x86_64 to be intel/Rosetta2 buildInputs = [
| foo x86_64.bar aarch64.baz
| ]; }
|
| using an unstable/pinned/git package: {
| stable ? import <nixpkgs> {}, unstable ? import
| (fetchTarball http://nixos.org/channels/nixos-
| unstable/nixexprs.tar.xz) {}, pinned ? import
| (fetchTarball https://github.com/nixos/nixpkgs/archive/ca
| 2ba44cab47767c8127d1c8633e2b581644eb8f.tar.gz) {},
| git ? import (fetchGit { url =
| "https://github.com/nixos/nixpkgs/"; ref =
| "refs/heads/nixos-unstable"; rev =
| "ca2ba44cab47767c8127d1c8633e2b581644eb8f"; }) {},
| }: let foo = stable.foo; in
| stable.mkShell { buildInputs = [
| foo unstable.bar pinned.baz
| git.qux ]; }
|
| selecting a particular C/C++ compiler&stdlibc++ version:
| { pkgs ? import <nixpkgs> {}, }:
| let clang = pkgs.clang_12 in
| pkgs.llvmPackages_12.stdenv.mkDerivation {
| buildInputs = [ clang ]; }
|
| The trap is: You could handle all of that by using or
| writing nix features. You could even use NixOS instead of
| whatever distro you're used to. But then by going cold-
| turkey you have to learn whatever nix feature on top of
| all the basic (as in fundamental) things nix has to
| offer. Purists would say "no no no this is not the
| sanctioned way", which is kind of true but also setting
| yourself up for failure; it'd be like looking at a
| mountain and trying to jump right to the top, which of
| course you will fail to, when you could just be climbing
| it and be successful. Whatever practical gets you on
| board is fine. You can get to the "pure nix" stuff later,
| if you ever need to.
| robertlagrant wrote:
| > the equivalent of being airdropped into the Romanian
| countryside with working knowledge of French
|
| Thanks to a chatty Romanian taxi driver I had earlier
| today, I understand this.
| tmpz22 wrote:
| Don't threaten Linux users with a good time.
| randomblock1 wrote:
| Same, nix seems like such a perfect solution- but it's
| prohibitively hard. Sure, I could use it and sink 12 hours
| into reading the documentation, or I could just install
| pyenv/nvm/etc and be done in 1 minute.
|
| Look at Python with Nix: https://nixos.wiki/wiki/Python And
| then consider the simplicity of `pyenv install x.x.x` and a
| requirements.txt.
| tra3 wrote:
| Do you think it's worth it to sink the hours up front for
| future efficiency gains?
| lelanthran wrote:
| > Do you think it's worth it to sink the hours up front for
| future efficiency gains?
|
| " _possible_ future efficiency gains ".
| xpe wrote:
| Many things in the world are _possible_. I often find
| _probable_ to be a more useful concept than _possible_.
| Any tool with deterministic results (such as Nix) are
| highly likely to reduce a whole class of future problems.
|
| With this in mind, the question shifts from rather vague
| talk about _possible_ futures to various kinds of
| scenarios.
|
| What happens when software developers have a
| deterministic build system? On the whole, the benefits
| are significant. Sure, there are costs to get there. In
| my view, the benefits often outweigh the costs.
| lelanthran wrote:
| > Many things in the world are possible. I often find
| probable to be a more useful concept than possible. Any
| tool with deterministic results (such as Nix) are highly
| likely to reduce a whole class of future problems.
|
| I disagree with the whole "highly likely" bit. Just
| because something new has a large positive value, that
| does not mean that a large time-investment or effort-
| investment in it is likely to pay off.
|
| After all, it may never get traction to make the cost of
| learning it worthwhile - some other, newer, better,
| easier system could come along and be adopted instead.
|
| We've seen this play out again and again in the
| technology world[1]. In 2000, a large time and effort
| investment into UTF-16 would be wasted as the world
| adopted UTF-8 instead. A large time and effort invested
| into new languages with powerful features (Haskell, circa
| 2005) would be wasted as the industry largely ignored it
| in favour of (presumably) inferior alternatives.
|
| Is nix highly likely to provide solutions to a whole
| class of problems?
|
| Sure.
|
| Is it highly likely to be the dominant, or even a common
| tool, in the future that solves those problems?
|
| Doubtful; "easier-to-use" has always won against "can
| better solve problems". I see no evidence that this will
| change anytime soon.
|
| [1] I've found that by procrastinating about learning a
| new technology, I can sometimes manage to avoid having to
| learn it altogether, because the tech dies off before my
| procrastination does. By putting off learning VRML in
| 2001, it went away before I ever got to it.
|
| Likewise, I'll put off learning new things until I
| actually have a need for the solution they offer. At
| least then, even if they die out in a few years, I'd have
| at least solved my immediate problems.
| tra3 wrote:
| Thanks for your thoughts. I worded my question poorly, I
| suppose, but you got my meaning.
|
| I keep seeing nix mentioned alongside complaints about
| it's maturity but that seems par for the course for
| something new, doesn't it? Until a tool reaches critical
| mass sharp edges are to be expected.
|
| I'll look into nix again when I have an appropriate use
| case.
| xpe wrote:
| Word choice observation: someone arguing in factor of
| spending time up-front for future gains would probably say
| "invest".
| tra3 wrote:
| Appreciate you pointing this out.
| cr__ wrote:
| What really stings is that I could have written this exact
| comment in 2015, when I did the same thing.
| samhh wrote:
| I agree that the documentation story could be better. I also
| think it's a great shame that the language isn't statically-
| typed, so to understand how to use something I have to
| inspect its source code.
|
| I've found it to be quite flexible though. For example,
| here's a commit in which I apply a patch to a tool to solve a
| problem that the derivation hadn't taken into account (and
| absent a home-manager solution): https://github.com/samhh/dot
| files/commit/867dd3b4d4b3942a0aa...
| omaranto wrote:
| > I also think it's a great shame that the language isn't
| statically-typed, so to understand how to use something I
| have to inspect its source code.
|
| I understand everything in that sentence except the word
| "so".
| samhh wrote:
| Static types are documentative, and language servers
| often also show you things like JSDoc alongside the type
| signature. Nix has neither of these, hence I have to
| check the source code or run a build and see what
| happens.
|
| I really like Nix but in this particular way it feels
| like taking a big step backwards from the other languages
| I frequently use, particularly for a language in which
| you'll necessarily be constantly interacting with bespoke
| interfaces.
| omaranto wrote:
| Informative function signatures, allowing programmers to
| add per-function documentation, and tooling that displays
| those things are by no means limited to staticly typed
| languages.
|
| I guess knowing allowed types for each parameter is
| informative (more if very specific types are used rather
| than string or integer), but documentation usually
| specifies that anyway. And even knowing all the types
| isn't usually enough without documentation.
|
| My point is just that I've used plenty of libraries in
| dynamically typed languages without needing to read the
| source. And conversely, I've occasionally needed to look
| at the source of a function in a staticly typed language,
| to answer some question not answered by the types and
| documentation.
| SkyMarshal wrote:
| Being able to see a function's entire type signature at a
| glance is the single most useful way of documenting code
| I've ever come across. No alt-tabbing to docs or relying
| on an IDE to pop up hints, etc., it's just right there.
|
| Haskell is the ideal here, since it separates the type
| signature into its own separate line at the top of the
| function declaration. That confers readability and
| reduces cognitive overhead more than any other language
| I've used.
|
| It also changes the programming thought process. You can
| pseudo-code an entire program just with function type
| definitions. Then test that the type defs compile without
| error, and go back and implement the function
| definitions. As long as the function definitions adhere
| to their type signature, the program almost always works
| (barring some I/O errors).
|
| Not all statically typed languages are created equal. The
| ones fundamentally oriented around function type
| signatures, rather than variable types, are the ones I
| think parent had in mind in his comment.
| knowmad wrote:
| As someone who has been using asdf daily for over a year now
| I can say from experience it's awesome and should absolutely
| replace all individual language managers as the standard way
| we manage language versions.
|
| I recently gave nix a try because of a hacker news comment
| and agree with most of the points in this comment. The fact
| that new users are encouraged to read not 1, but 3 separate
| manuals to grok the tool is extremely discouraging for new
| users. Also the fact that nix is currently transitioning to a
| new feature called flakes and deprecating channels signals to
| me that the ecosystem is not stable enough for me to
| recommend that my team at work use it right now.
|
| That being said, I intend to read at least 2 of the manuals
| and use nix for dependency management in my side projects and
| to manage my system dependencies. I think nix takes all the
| great things that asdf can do, to a whole new level. The
| dream of having a nix section in my projects 'Getting
| Started' that only has 2 steps (1. Install nix 2. Run `nix-
| shell`) is something I desperately want.
|
| I think nix is much like erlang in that it's incredibly
| powerful and solves a lot of common problems, but it's used
| and maintained by an old guard that went through the trouble
| of learning how to use it and maintains the "I did it, why
| can't you" mindset. The erlang ecosystem has been given new
| life and been made easily available to new users thanks to
| the awesome work done by Jose Valim and the rest of the
| elixir team and community. However, I really hope that it
| doesn't take 30 years and a new language built on top of nix
| for it to become accessible to everyone.
|
| For those working on nix and the new flake functionality
| please look to things like elixir and asdf for inspiration on
| documentation, ease of use and pragmatism. Help make the tool
| you love ubiquitous. It will ultimately make your life easier
| when working with new developers because they will already
| have nix installed.
|
| I intended to do everything I can to help push nix in this
| direction (once I understand how the hell it works) if anyone
| reading this, has some time on their hands and wants to
| improve the state of software development for everyone please
| go down the nix rabbit hole and try to find ways to
| contribute to making it more accessible.
| shantnutiwari wrote:
| The Nix crowd is becoming a lot like the Rust crowd-- they
| hijack every conversation with "Have you tried Nix"?
|
| Like you, I find Nix hard to use and the docs confusing
| (beyond trivial examples). The amount of time the Nix mafia
| spends proselyting could be better spent making better docs
| tannhaeuser wrote:
| I agree. I've got nothing against nix, but it frankly sucks
| that the top comment isn't about asdf which I had hoped to
| learn about. There should be special "-1 offtopic" comment
| flagging because it's a frequent phenomenon.
| SkyMarshal wrote:
| As a fan of Nix and NixOS, I agree with this assessment.
| Too often people hijack threads about a topic to discuss
| something related but totally different and off-topic.
| It's kind of annoying, even when I'm a fan of the off-
| topic thing.
|
| I've never heard of asdf and popped in here to read about
| it, people's experiences with it, etc. If I want to read
| about Nix/OS I'll find a thread on that, or go to
| /r/nixos, etc.
| leaflets2 wrote:
| > Too often people hijack threads about a topic to
| discuss something related but totally different and off-
| topic
|
| And what are you doing :-)
| ravi-delia wrote:
| I mean, just minimize the comment? I like seeing
| discussion on many related aspects of an article, it's
| cool seeing how things compare.
| ris wrote:
| > the top comment isn't about asdf which I had hoped to
| learn about
|
| Oh I think you're using HN wrong. To learn about the
| subject you can read the actual article that's linked to.
| No guarantees about the comments section.
| jjgreen wrote:
| They do seem to share a fondness for executing curl output
| caymanjim wrote:
| This criticism is getting old. Piping curl output to bash
| is no less secure than installing a package via any other
| mechanism. Either you trust the source or you don't. I
| trust it more than I trust npm or pip because I'm
| typically getting it from the primary source instead of
| relying on a middleman that's got a poor track record.
| leaflets2 wrote:
| If there's a SHA hash, served by a different server, an
| attacker would have to compromise both -- now just one?
| jjgreen wrote:
| It may be old, but it is valid, and it allows me to mock
| Rust and Nix advocates at the same time; who could
| resist?
| cwp wrote:
| Yeah, frustration on both sides here. On one hand, I can
| see how the nix evangelism can get annoying, especially
| here on HN. On the other hand the nix crowd is seeing a
| parade of ad hoc, informally-specified, niche
| implementations of way-less-than-half of nixpkgs. This is
| an important problem, maybe one of this most important
| problems the tech world is grappling with right now, and
| _there is already a solution_.
|
| The reality is that the docs are fine. The reference
| material is comprehensive, and there are some very good
| tutorials and introductory essays to get people started.
| The problem is that it's difficult. Nix is very, very
| different from what people already know, and it takes quite
| a bit of effort to learn how it works. Most people just
| aren't prepared to do that much work just to install some
| software.
| airza wrote:
| If the problem is that it's difficult and the
| documentation is not making that difficulty simple enough
| to use, then the documents are not fine. Simple as.
| leaflets2 wrote:
| If the problem is that it's difficult to learn to fly a
| fighter jet, and the documentation is not making it
| simple, the real problem is that the documentation is not
| fine. Simple as that.
| arrakeen wrote:
| > The reality is that the docs are fine
|
| then show me where `nix-env`'s `--arg` and `--argstr` is
| documented.
| w-j-w wrote:
| The ad-hoc solutions you're talking about are often
| highly usable, a virtue often forgotten by software
| evangelists. Nix's learning curve will keep away people
| who are disciplined in avoiding yak-shaving, as the
| language specific version enables productivity now. Maybe
| adding a layer on top of Nix, like create-react-app is to
| webpack, can make it a better option for immediate
| productivity oriented developers.
| exdsq wrote:
| Nope nope nope. They're not fine. Nix is really really
| hard to use. I worked for a company that ran all the ops
| with Nix managed by some of the contributors. Every time
| I wanted to do something I'd have to get DevOps to do it
| for me. I've never ever had that level of difficulty
| doing stuff outside of a Nix environment and I really,
| really tried. Most of my colleagues who are fairly decent
| developers struggled with it too. It's the _only_ tech I
| advise against when it 's mentioned nowadays. And I
| _want_ to like it.
| lambdaba wrote:
| On the other hand only Nix (to my knowledge) guarantees
| truly reproducible builds.
|
| It helps that people who are into Nix most often use
| NixOS. NixOS is the only distro I've written packages
| for. It's not even particularly hard since there's a lot
| of inspiration to find in the official and community
| repos + people sharing personal configurations.
|
| The UX wasn't the best but it's improving.
|
| It remains extremely reliable which is not something that
| you could say of other systems.
| marcosdumay wrote:
| Hum... Every time I see Nix cited somewhere, it's somebody
| who tried it and didn't manage to get expertise out of the
| documentation maze. (You can add me there too.)
|
| But it's no wonder that people keep commenting that Nix is
| an awesome idea that solves a lot of problems. That's
| because it's an awesome idea that solves a lot of problems
| (yeah, like Rust).
|
| It is also completely undiscoverable (not completely unlike
| Rust), and that's why almost nobody uses it. But that's a
| matter of improving the documentation or maybe fixing one
| or two superficial problems on the language.
| oblio wrote:
| > But that's a matter of improving the documentation or
| maybe fixing one or two superficial problems on the
| language.
|
| Based on a quick check of their Wikipedia page, Nix has
| been around for 15 years, since 2007.
|
| So those for sure look like structural problems. I don't
| know which ones exactly, but paraphrasing Tolstoi,
| popular software products are popular in much the same
| way, while unpopular ones are unpopular in their own
| unique ways.
|
| In my experience almost no open source project that
| hasn't entered the mainstream in its first 10 years
| manages to turn the ship around, unless it lucks into a
| major change of environment. Programming languages are
| sometimes exempt if they have a "killer" library or
| framework pop up.
| whateveracct wrote:
| Nix ain't trying to win marketshare it's trying to be
| amazing. But it takes guts to be amazing and that means a
| lot of people don't manage to use Nix.
|
| HN tends to see success in terms of adoption rate &
| ranking. Nix is on another plane. It's beyond critical
| mass. And for people like me, it is a huge force
| multiplier. But I had to grind my way to the point where
| that is true.
|
| I tend to not bother selling people on it. People who
| want what Nix does will find it at some point. So I just
| focus on using Nix myself in a world of worse tools.
| Feels great.
| robertlagrant wrote:
| Oh, I'm not selling these cars. No, no. These babies sell
| _themselves_.
| grumbel wrote:
| Nix just got a huge overhaul, with most functionality
| getting bundle under a single easy to use 'nix' command
| (i.e. similar to 'apt'), rendering a lot of the old
| commands obsolete. That currently creates a bit of a
| confusing situation, as lots of docu is about old
| commands, while new commands that look similar can behave
| quite a bit different (e.g. 'nix-shell' != 'nix shell').
| Nix Flakes are another new thing and also create a bit of
| confusion and ugliness (e.g. packages being named
| 'nixpkgs#legacyPackages.x86_64-linux...').
|
| That said, I haven't found Nix hard to use, quite the
| opposite. The new 'nix' command is pretty self
| explanatory, it's just a little incomplete in spots. And
| the Nix language is quite simple and easy to understand
| if you have ever touched a functional language. Compared
| to all the other Linux stuff I have played with over the
| years, it was a very pleasant experience so far.
| marcosdumay wrote:
| Well, Nix is mainstream.
|
| Evidence of that is that you can just talk about it by
| name here on HN without explaining what it is.
| samb1729 wrote:
| That could just as easily be attributed to the word Nix
| being easy to search for. But even if everyone on HN is
| aware of what it is, that doesn't make it mainstream by
| any definition I can think of. There are countless
| examples in the world of things that most people (in a
| given audience) are aware of, but that almost none of
| them participate in.
|
| Is Haskell mainstream? What about committing murder?
| marcosdumay wrote:
| Your claim is that software doesn't go from non-
| mainstream into popular if it stays non-mainstream for a
| decade. I'm really not willing to discuss the definition
| of "mainstream", but there are plenty of cases of
| software that stayed a decade as "everybody knows it,
| nobody uses it" and eventually got popular.
| 3836293648 wrote:
| It is horrible searching for Nix stuff because of all the
| *nix stuff.
|
| And yes, this is 100% Nix's fault
| AceJohnny2 wrote:
| nix local package search sucks, and it is much more effective
| to use the online package search [1].
|
| It's astounding because the local version is doing a naive,
| case-sensitive regex search, which is pretty bad when you're
| looking for, say, the Perl package MIME::Lite which can have
| various junctions and capitalizations. In fact, that package
| has a listed name of perl5.34.0-MIME-Lite but is installable as
| nixpkgs.perl534Packages.MIMELite!? I can't even.
|
| I found the code a few months back (but can't be bothered to
| look it up just for a comment). I didn't fix it then because I
| had other priorities than learning how to modify my local copy
| of nix to test it.
|
| [1] https://search.nixos.org/packages
| norman784 wrote:
| I just tried but is really a pain in macOS right now, all went
| smooth until I got an error while trying to install the bundler
| gem, I skimmed quickly GitHub and seems to be an issue of macOS
| and the root dir being read-only, I suppose is too much hassle
| to worth it, at least if you compare with docker.
|
| As others pointed, the idea behind nix is cool, but it
| sometimes fall short, hope it get ironed all the issues in
| macOS so I could give a try in the future.
| Isthatablackgsd wrote:
| Same, tried Nix on my macOS (MBA M1) back in January. The
| installation is painless but it ran into an issue in the
| first run after it. It looks like that I skipped a step
| somewhere, I was following the guide and it should be working
| but its not. I suspected it was the guide that someone forgot
| to include it in the documentation or edge case I
| experienced. Now the uninstalling part is painful because
| there bare minimum documentation about this. Tried all
| different way to remove it and I couldn't. Took me a month to
| finally remove Nix from my macOS, I found the information
| through SO.
| LukaD wrote:
| I had the same exact same experience. If the uninstallation
| wasn't such a pain in the ass I might even give nix another
| try in the future.
| soraminazuki wrote:
| Were you trying to install the Bundler gem as root? If that
| is the case, I'd recommend against it even without Nix. In
| any case you can find Bundler in Nixpkgs instead:
| nix-shell -p bundler
| amelius wrote:
| > If you are looking for a language agnostic version manager,
| the nix package manager might be worth a try.
|
| Could it solve the issues we're seeing lately with NPM?
| paskozdilar wrote:
| For those who like the concept of Nix, but don't like the Nix
| language, the Guix package manager [0] might also be worth a
| try. It's an official GNU project, inspired by Nix, and it uses
| Guile Scheme instead of Nix language.
|
| [0] https://guix.gnu.org/
| javajosh wrote:
| Eventually someone will implement Nix in Javascript, and then
| it will explode.
| deepsun wrote:
| All languages get eventually implemented in Java and
| Javascript.
| recuter wrote:
| Figuratively?
| Arubis wrote:
| In both senses of the word "explode", I expect.
|
| (Tongue-in-cheek, but I'd actually like to see this happen:
| Nix is a fantastic concept obscured by space-alien syntax.)
| zeec123 wrote:
| Not exactly, but PureNix
| sleepydog wrote:
| The Guile compiler has a JavaScript frontend. You can write
| guix packages in JavaScript:
|
| https://lists.gnu.org/archive/html/guix-
| devel/2022-03/msg001...
| marius_k wrote:
| js code example still looks schemish to me.
| SkyMarshal wrote:
| Nix is mostly already Javascript. Just JSON with some addon
| functionality.
| rgoulter wrote:
| The code in nixpkgs does contain some code that uses
| relatively sophisticated programming language constructs,
| such as lazy evaluation, partial function application, or
| fixpoints. -- So I'll say there's at least some friction to
| Nix from the language being pure/functional.
|
| Mostly I think Nix would be just as hard if packages were
| expressed in any language.
|
| Stuff like "this program compiled by nix is using a
| different glibc than this library" is going to be confusing
| regardless of whether Nix expressions can be written with
| JavaScript; or stuff like some program assumes it can write
| to $HOME in its build script not playing nicely with Nix.
|
| A lot of why Nix is hard is because it's weird, and that
| even as an end user when things go wrong it may require
| understanding more about what's going on than what other
| solutions require.
| goodpoint wrote:
| Guix is much more usable.
| bronzecarnage wrote:
| My biggest gripe with nix is that you need sudo rights to get
| it running properly on another distro. I know you can do the
| nix-user-chroot thing, but it's not possible on machines where
| kernel namespaces aren't accessible for unprivileged users.
|
| I spent time understanding it, and getting it working, but it
| really doesn't work just universally. Once the install process
| becomes more like, say, miniconda -- where an unprivileged user
| can install it locally with ease -- nix will get a massive
| boost in my books. Until then, I can only yearn.
| ris wrote:
| It is possible to use from a completely unprivileged
| environment by putting the nix store in your home directory
| instead of `/nix`. _But_ (and it 's a big but) you'll then
| have to build all packages from scratch because the hashes
| for you packages will be different to the ones in the nix
| binary cache.
| yeti_or wrote:
| I like to use eclectica, it's not very popular, but it works for
| me
|
| https://github.com/markelog/eclectica
| 3836293648 wrote:
| Is this just low effort low compatibility nix?
| xwowsersx wrote:
| > And then one day, I found out about asdf-vm.
|
| FYI, the link to asdf there is a link to google.com
| KptMarchewa wrote:
| I thought it's a joke post after that, but apparently this is a
| real project: https://github.com/asdf-vm/asdf
| xwowsersx wrote:
| haha, I have to admit I also started to wonder if this was an
| April Fools' joke about how Google searches are a developer's
| main tool.
| derekzhouzhen wrote:
| There is a age old solution for managing multiple versions of the
| same software: gnu stow: https://www.gnu.org/software/stow/
|
| No crazy settings in your profile. Downsides are: You need to
| have root, you can switch any time but different versions don't
| work at the same time, and it does not work in windows.
| thingification wrote:
| > You need to have root
|
| This isn't true
|
| > different versions don't work at the same time
|
| Within one stow directory: true. But nothing prevents creating
| multiple directories (I don't think I've ever done that).
| derekzhouzhen wrote:
| You are right, I should have said to manipulate /usr/loca/
| you need to be root.
| akira2501 wrote:
| I feel like hardcoding shebang paths and not implementing a
| good widespread PATH whitelisting/safety mechanism has created
| far more problems than we should ever have had to deal with.
|
| It strikes me that not for the two above items we might be able
| to solve this with $PATH resolution and directories filled with
| symlinks to our "development versions."
| thejosh wrote:
| There is an issue that delays commands by 200ms, which isn't a
| massive issue for my usecase, but you might want to look into
| asdf-exec https://github.com/danhper/asdf-exec
| password4321 wrote:
| Both my thumbs up for the asdf extendable version manager, a big
| help when brew leaves older macOS editions behind.
|
| Previously: https://news.ycombinator.com/item?id=26018684 (Feb
| 2021: 98 comments)
| d3k wrote:
| It seems nice. When programming envs tend too complicated though,
| I tend to switch to docker: a given image will handle a given
| version and it's mostly hassle free. That probably would not make
| sense on macOS or Windows, if you want to run native versions.
| Ajedi32 wrote:
| This. Docker not only solves the language-specific venv
| problem, but also encapsulates other system-level dependencies
| that may not be covered by stuff like pyenv or nvm. Plus with
| docker-compose you can trivially satisfy other, way more
| complicated dependencies like "fully functional PostgreSQL DB
| is accessible over the network", _and_ you can deploy the
| entire thing to production with a few simple commands.
| brightball wrote:
| One thing that I realized after finally dabbling with Python
| after years in other languages was just how much Python _needs_
| Docker in development. There are so many details to get the
| right version of python working on a project because of system
| level assumptions that using Python without Docker is a huge
| pain.
|
| Conversely, the experience of using Ruby within Docker for
| development is often clunky and awkward by comparison.
| 0xedd wrote:
| Installing dependencies, per the docs, before installing
| python works well. Installing library dependencies for
| packages also works well. It's not dissimilar to compiling
| c/c++ where you have to read the installation docs. And the
| docs better be updated.
| robertlagrant wrote:
| I've not found this; pyenv and poetry sort it all out for me,
| and I could simplify and just make do with one.
| matsemann wrote:
| Same. Getting a python environment up and running can be a
| multi-day endeavor if you're unlucky. Just using docker compose
| on the existing Dockerfile for the project and mounting the
| local filesystem, means everything just works. PyCharm now
| works nicely with a remote interpreter inside docker compose.
|
| I've even tried doing that with all the devtools we have. Like
| normally a developer here has to install gcloud, terraform,
| kubectl, apt-get lots of stuff, and then 3-4 internal tools.
| Then configure all of it, and of course some stuff will just
| not work on someone's computer. But creating a docker image
| including all that means that I can grab a new computer and be
| up and running in minutes.
| hushpuppy wrote:
| I regularly use asdf-vm to manage kubectl, python, nodejs,
| and a bunch of other stuff.
|
| "Setting up a Python environment" takes about 10 minutes and
| it's dead nuts reliable.
|
| It is a very effective package manager.
| antholeole wrote:
| Hijacking this: does anyone know how to get vscode to use a
| python interpreter inside a docker container for linting?
| I've resorted to using the docker container, but also having
| a python env so that I can tell vscode to point to the python
| interpreter in the env to get linting. Otherwise,
| dependencies come up unresolved.
|
| I did some googling, but I'm unfortunately not experienced
| enough to be able to describe the problem succinctly to
| google.
|
| Using pylance, btw.
|
| TIA!
| nevalsar wrote:
| Try this: 1. Bring up command palette using `Ctrl+Shift+p`
| 2. Search and select "Python: Select Interpreter" 3. Select
| your Python environment.
|
| This works in VSCode over SSH perfectly fine - the locally
| running VSCode is able to detect and select Python
| environments in the remote server.
| antholeole wrote:
| The trouble with this is that the python environment is
| in the container - In step 2, id need to type path to the
| interpreter that it inside the container.
|
| Although, now you've got me thinking what would happen if
| I mounted the interpreter inside the container to
| something local that I can point to? Might try this
| later, thanks for the help!
| matsemann wrote:
| Have you tried VSCode's remote abilities? Should do exactly
| as you describe.
| https://code.visualstudio.com/docs/remote/remote-overview
| or https://devblogs.microsoft.com/python/remote-python-
| developm...
| antholeole wrote:
| I've seen these and got it to work this way. I'm not too
| fond of working inside the container as it's just a
| little more friction, but I've been working this way for
| a little bit now. I was hoping to avoid working in the
| container.
| kaladin_1 wrote:
| I like the idea. A package manager that can be used on several
| languages but simple enough to pick up in 15mins or less.
|
| Just today on trying to get rvm to do something I got angry at
| the number of language-specific package managers I have
| installed: nvm, jenv, rvm, rbenv(after I couldn't get rvm...)
|
| Of course, I know about Nix but it is too complex for my use
| cases.
| dangoor wrote:
| I recently discovered Hermit[1], which is a really
| straightforward tool for managing versions of tools for a given
| project. From the FAQ[2]:
|
| > Hermit is probably most similar to asdf, but their goals
| differ. Hermit's goal is to make isolated cross-platform tooling
| consistent, self-bootstrapping, and reproducible at the project
| level. asdf's primary goal is to allow developers to install and
| switch between multiple versions of languages and tooling.
|
| The downside for the author of this post, though, is that Hermit
| doesn't yet support Ruby because it's not static/relocatable.
|
| [1]: https://cashapp.github.io/hermit/ [2]:
| https://cashapp.github.io/hermit/faq/#asdf
| huac wrote:
| Hermit is great!
| bjourne wrote:
| Can someone give like the 120 seconds elevator pitch to why you
| need any of this stuff? I know _how_ to use pyenv, virtualenv,
| Poetry, autoenv, you name it. I just never have found any
| practical use for it. But what if you want to have multiple
| versions of the same package installed? I don 't. If some project
| I work on requires tf 2.x-1.0 and I only have tf 2.x.0 then I
| just rm -rf ~/.local/lib/python3.x and reinstall the dependencies
| anew. Furthermore, most Python maintainers do a terrific job in
| keeping up with breaking api changes in their dependencies (which
| I'm very grateful for) so it's quite rare that I need to use a
| Python package that is not the current release.
| dsabanin wrote:
| What if you work on 2-10 different codebases that all use
| slightly different versions?
| [deleted]
| haolez wrote:
| This probably makes a difference for hugo enterprise monorepos.
| I myself have a workflow similar to yours, but with the
| following mindset: my local environment is an approximation of
| what will be in production. I don't need every version to match
| and if I ever encounter a corner case where this is not true,
| I'll just deal with it and move on.
| roesel wrote:
| What if you're simultaneously working on two projects with
| different version requirements and don't want to rm -rf and
| reinstall every day?
| bjourne wrote:
| I don't do that.
| synergy20 wrote:
| for bash I use venv, nvm and gvm(more popular than goenv and
| worked well), worked fine so far.
| [deleted]
| mjs7231 wrote:
| This feels like Trillian for virtualenvs. So typical that as
| engineers we keep abstracting everything we can until we run out
| of people that understand the basics. This tool looks like a
| great resource until step 4 when we create a virtualenv protocol.
| In chat clients, the protocols came first. For development IDEs
| adding auto-complete, it seems the protocol LSP came second. It's
| happening to language virtualenvs now as well.
|
| 1. You need a tool for X to solve an issue so you write it. (AIM)
|
| 2. Guy over there needs a tool for Y to solve the same issue, so
| he writes it. (Yahoo Chat)
|
| 3. So many tools exist, we need to merge them all into one.
| (Trillian, Adium)
|
| 4. Merging all the tools created a monster, we need a common
| protocol they all understand.
|
| 5. No one remembers how tool X works anymore, updates become more
| sparse.
|
| 6. Both X and Y are no longer popular, big company took over.
| (Slack)
| e12e wrote:
| I've found asdf to be a reasonable choice when needing to manage
| more than one language - especially where there are good "native"
| version managers that asdf wrap (eg: python, ruby, node, to a
| certain extent rust/cargo).
|
| Not sure I'd do this, though: mkdir -p
| ~/.config/fish/completions cp
| ~/.asdf/completions/asdf.fish ~/.config/fish/completions
|
| But more probably add a symlink to the completions managed by
| git/asdf: ln -s ~/.asdf/completions/asdf.fish
| ~/.config/fish/completions/
___________________________________________________________________
(page generated 2022-04-05 23:00 UTC)