[HN Gopher] We want to make Nix better
___________________________________________________________________
We want to make Nix better
Author : biggestlou
Score : 169 points
Date : 2022-09-02 17:03 UTC (5 hours ago)
(HTM) web link (determinate.systems)
(TXT) w3m dump (determinate.systems)
| mixedCase wrote:
| Hot take as a NixOS user that uses Nix for work: "all" we need is
| a much better, sound, statically typed language to build better
| abstractions with.
|
| The only hard requirements I can think of are algebraic data
| types with exhaustive pattern matching to go with, row
| polymorphism, purity and good inline documentation support.
|
| I don't know if a good enough hostable language exists or if it
| should be a new version of Nixlang, but almost every single
| annoying problem that makes me go "Nix is getting in my way" can
| be traced back to the lack of a good, powerful type system
| leading to a house of cards situation whether it comes from
| nixpkgs or entirely of my own making.
| jhillyerd wrote:
| I think this is true for Nix in the deployment/ops space, where
| debugging a broken build can be very frustrating. Language
| improvements are going to be less useful for app developers,
| the Flake learning curve is not going to get better with a type
| system.
|
| Perhaps something like heroku buildpaks
| (https://github.com/railwayapp/nixpacks ?) would help devs get
| on the Nix train.
| tikhonj wrote:
| The problem is that we need not just a language like that, but
| a version of Nixpkgs-the-codebase designed _for that language_
| with corresponding sensibilities. And that 's a much bigger
| obstacle than coming up with a language!
| zuzuleinen wrote:
| "But there's a catch: to make that happen you need to write some
| Nix, use Nix tools, and probably consult several documentation
| sources."
|
| You can use bob[1] if you want a build tool which uses Nix to
| install dependencies in an easy manner: just list the package
| names for a task and then they will be installed.
|
| I'm looking forward for all the changes in Nix ecosystem and it's
| a good sign the fact that they also started working on an
| initiative to improve Nix documentation which was spread all over
| the places.
|
| [1] https://bob.build/
| biggestlou wrote:
| Bob is very cool and I think that there's room for lots of
| tools like this that build _on_ Nix but abstract away some of
| the trickier parts.
|
| Suggestion: make Bob installable with Nix!
| zuzuleinen wrote:
| Thanks for the suggestion! It makes sense to have it on
| Nixpkgs collection since anyway you need to install Nix
| alongside bob
|
| Already created an issue :)
|
| https://github.com/benchkram/bob/issues/193
| yewenjie wrote:
| How does Bob handle network requirements (downloading
| dependencies during the build)? I think that is one of the
| biggest pain points for nix packaing. Many projects have build
| instructions that depend on downloading stuff in a complex
| manner that cannot be easily abstracted/shaped to what Nix
| expects.
| ParetoOptimal wrote:
| > Many projects have build instructions that depend on
| downloading stuff in a complex manner that cannot be easily
| abstracted/shaped to what Nix expects.
|
| The basic way to fix this with nix is:
|
| 1) download the thing with a fixed ouput derivation: a
| derivation who's source has the URL and checksum of the
| artifact
|
| 2) copy the artifact from the nix store where said build tool
| stores it
| zuzuleinen wrote:
| I wrote a short guide[1] how bob uses Nix, but essentially we
| still use `nix-build` under the hood to build a list of
| packages you give under dependencies
|
| > Many projects have build instructions that depend on
| downloading stuff in a complex manner that cannot be easily
| abstracted/shaped to what Nix expects.
|
| Are you referring here to packages which are not available in
| Nixpkgs?
|
| [1] https://bob.build/docs/usage/package-management
| shiryel wrote:
| I love nix, I've been using it for the last 2 years, I have a
| very stable setup from these 2 years of effort [0], and I just
| can't recommend Nix for Linux beginners, why?
|
| It's not because of the nix language, It's not because of the
| CLI, it's because everything is scattered, you have to consult
| many places to find out how to do things with Nix, here is an
| example:
|
| Usually, when I need a new complex program, like Steam, I first
| check the system-wide configuration [1], the wiki [2] and the
| package list [3], if I just want it on my user, I need to check
| if Home Manager has an option [4], if it doesn't, I can try using
| the "home.packages" option. Now, if I need to override something
| on the package, I need to remember how to do it with [5] [6]
| (while checking the source code for the package in parallel to
| find the options).
|
| And then sometimes, on very rare occasions, I need to fine tune
| something with the nix language, so I need to check the
| builtins/lib docs [7], but some builtins are not there, so I need
| to either use nix-doc [8] or find the docs inside the code-bases
| [9] [10] (they are split between both repos)
|
| For me, this is one of the main pain points of using Nix / NixOS
| that needs to be solved.
|
| [0] - https://github.com/shiryel/nixos-dotfiles
|
| [1] - https://search.nixos.org/options
|
| [2] - https://nixos.wiki/wiki/Steam
|
| [3] - https://search.nixos.org/packages
|
| [4] - https://mipmip.github.io/home-manager-option-search/
|
| [5] - https://nixos.org/manual/nixos/stable/#sec-customising-
| packa...
|
| [6] - https://nixos.org/guides/nix-pills/nixpkgs-overriding-
| packag...
|
| [7] - https://teu5us.github.io/nix-lib.html
|
| [8] - https://github.com/lf-/nix-doc
|
| [9] - https://github.com/NixOS/nix
|
| [10] - https://github.com/NixOS/nixpkgs
| anderspitman wrote:
| For the first time in about 8 years an Arch update rendered my OS
| unbootable last week. I'd had graphics issues once or twice but
| this time it just hung in systemd somewhere. Still don't know
| what happened. Rolled all packages back about a month and that
| worked, but I'll need to update eventually.
|
| I'm considering Nix but don't like the custom language. I wish
| Guix were. more popular and less extreme on the freedom side.
| Chances are I'll just do a fresh Arch install or maybe try Debian
| rolling.
|
| These days I like to install almost all software and libraries by
| hand in my home directory. All I really want is a solid system
| that basically boots fresh then loads my home directory. Maybe I
| should do a net boot of some sort?
|
| Any other suggestions?
| Daegalus wrote:
| you can try something like Kinoite or Silverblue from fedora.
|
| you can also manage tools with asdf if it makes it easier.
|
| I personally run Fedora + Flatpaks + asdf and it covers
| everything, and is stable, up to date, and has a new release
| every 6 months. it feels midway between a rolling release and.
| apoint release
| jcynix wrote:
| I just started to experiment with EndeavourOS, a prepackaged
| version of Arch. Looks nice so far.
|
| https://endeavouros.com/about-us/
| grhmc wrote:
| With Eelco as a cofounder of DetSys, I feel really excited about
| where we're going here. I think the world is in _many_ ways
| primed and ready for Nix, as long as we can help Nix "meet them
| in the middle."
|
| We're working on making Nix more accessible and producing good
| and usable, production-ready workflows so teams can just pick it
| up and go. I'd love to hear what y'all think, to help make sure
| we're going in the right direction.
| tadfisher wrote:
| I might suggest making a typed superset of Nix, basically
| adding record types and function signatures. That in itself
| would be a huge boon for tooling and interacting with nixpkgs.
| nixpkgs describes a huge and varied meta-API for derivations in
| various build ecosystems, which makes it hard to use because
| you ultimately need to dive into source code.
| yewenjie wrote:
| https://github.com/tweag/nickel
| tazjin wrote:
| Some community members have looked into this quite a bit, and
| personally I have come to the conclusion that types don't
| really give that much benefit to Nix (and I say this as a
| Rust & Haskell developer).
|
| Nix expressions are almost always[2] short-lived snippets
| that evaluate to a data structure, not long-running programs
| where the distinction between "static analysis time" and
| runtime is extremely relevant.
|
| In fact, deriving the full set of potentially relevant type
| constraints is _only_ possible at runtime due to how the
| import system works and the shape of most expressions. If you
| pick a single file from nixpkgs without context, you can 't
| get much information from it at all (other than a lot of
| unknown types being passed to things that accept an unknown
| type) - it's only in the context of an evaluation of a graph
| node that uses that file that you can infer anything
| meaningful about it (and even then that information is only
| relevant in that context).
|
| I think what people actually want is a way to derive more
| sensible documentation information from the code so that the
| various builders etc. could have checked documentation pages,
| as well as tooling for auto-completing members of attribute
| sets, names of (formals) function arguments and so on.
|
| fwiw, there's an alternative implementation of the Nix
| language[0] that we (TVL[1]) are slowly open-sourcing at the
| moment, and we're aiming to design it in such a way that
| things like an LSP can be implemented on top of it, and to be
| able to dump out static information from a context that can
| help with documentation etc.
|
| [0]: https://cs.tvl.fyi/depot/-/tree/tvix/eval
|
| [1]: https://tvl.fyi/
|
| [2]: Yes, I'm aware that Nix is turing-complete, but surely
| nobody would write a web framework[3] and HTML templater[4]
| in it.
|
| [3]: https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md
|
| [4]: https://cl.tvl.fyi/c/depot/+/3410
| hamandcheese wrote:
| > If you pick a single file from nixpkgs without context,
| you can't get much information from it at all (other than a
| lot of unknown types being passed to things that accept an
| unknown type) - it's only in the context of an evaluation
| of a graph node that uses that file that you can infer
| anything meaningful about it (and even then that
| information is only relevant in that context).
|
| This is exactly where types would be the most beneficial,
| and what makes nix so hard to read. If a nix function could
| specify the types of its arguments, then I might have a
| chance of better understanding it.
|
| Just because most nix functions currently don't have many
| constraints on their inputs doesn't mean that they couldn't
| in the future.
|
| > it's only in the context of an evaluation of a graph node
| that uses that file that you can infer anything meaningful
| about it (and even then that information is only relevant
| in that context)
|
| That's how nixpkgs works now. But we can make functions
| that work only in more limited contexts.
| tadfisher wrote:
| > tooling for auto-completing members of attribute sets
|
| That's a great start, but it would be nice to know what the
| expected type of the member is! Often it's another
| attribute set, a list, or even a function, and it's
| impossible to tell what type is expected without looking at
| where the member is used.
| grhmc wrote:
| We've heard this feedback pretty frequently. We have a few
| ideas of ways to add some types to Nix in place. I think even
| simple naive approaches would add a lot of safety and improve
| the usability of Nix.
|
| However, one of the bigger problems here is the general API
| design of Nixpkgs. Most of the function interfaces use some
| named parameters and then use `...` to accept any additional
| parameters. All of those then get passed down to some other
| function, and on and on. This makes a mess! It is hard to
| tell what functions use what parameters, and none of them can
| restrict their interface.
|
| We could see some significant DX here by creating smaller,
| more specific interfaces that actually restrict their inputs.
| mikepurvis wrote:
| I think the biggest win with some progressive typing would
| be better error diagnostics. Right now the errors are
| _horrifyingly_ bad in many cases, puking out massive
| recursive stack traces that have nothing to do with the
| real problem. Being able to pinpoint exactly where the
| square peg is going into a round hole would be amazing.
| setheron wrote:
| I wish this blog post had more meat to it. It was pretty
| minimal on information.
|
| This is also due to the fact that I tend to value Graham's
| posts highly :)
| biggestlou wrote:
| Consider this post more of a "hello world." Substance is on
| the way :)
| flurie wrote:
| Hi, Graham, I have a few questions.
|
| Does Eelco's departure from Tweag have any impact on Nickel as
| a possible replacement for nix?
|
| There are lots of fragmented attempts at making nix easier/more
| immediately valuable/lower barrier to entry. Some of these are
| personal projects, some are businesses. Some of these just make
| nix easier, and some attempt to put something in between nix
| and the user. I'm thinking about things like Cachix, Flox, and
| divnix (formerly devos). How do you see your work interacting
| with these?
|
| One of my barriers to adoption at work has been ensuring
| maintenance continuity. That's always going to be the case for
| tools that aren't ubiquitous, but I worry with nix that teams
| without my assistance will revert to other tools they know
| better, even if I've invested a good bit of time in teaching
| them enough to keep things going. Do you seek to address that?
| nbp wrote:
| My thinking is that this is long overdue!
|
| I am working on a very large project where multiple solutions
| have been invented to solve problem which are all handled by
| Nix:
|
| - Bake in the repository the clone of the dependencies.
|
| - Compile the tool chain to create reproducible builds.
|
| - Create "artifact" builds, to avoid large recompilation times
| for many developers.
|
| - Use various tools to cache and distribute builds.
|
| - Use a bleeding-edge version of some dependencies.
|
| - Use Node, Java (Android), C++, Rust, ...
|
| If only Nix were to run natively on Windows, this would check
| all the boxes.
| bpye wrote:
| There was some work being done to make Nix work on Windows
| [0] - but it might have fizzled out. I too would really love
| Nix to work on Windows. I already use it on both macOS and
| Linux and it's great.
|
| [0] - https://github.com/nix-windows/nix
| mamcx wrote:
| My main gripe is documentation.
|
| Look at this:
|
| https://nixos.wiki/wiki/PostgreSQL
|
| Now, the real "workflow" with nix is look at other setups, or
| just look at the code:
|
| https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/s...
|
| And look at the code is the only reliable way to see what
| exactly is supported.
|
| My second gripe is the fact is hard to see what nix do. Today I
| hit this trouble:
|
| https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_...
|
| The thing is: Nixos not tell what is doing. `verbose` is too
| much noise.
|
| What I wish now is something like: nix change
| nothing (if my changes somehow don't do anything, like in my
| issue) nix setup users... nix setup
| postgresql
| ontouchstart wrote:
| I have more than 20 years of Unix experience and was getting into
| DevOps and CI/CD in the recent years with GitHub Actions, Azure
| DevOps, AWS etc. I literally discovered Nix yesterday via Andrew
| Kelley's Zig demo on YouTube. I had the similar feeling as I
| first learned Unix tool chains two decades ago.
|
| I believe live demo and CI/CD pipeline might be some of the best
| use cases to get people interested in Determinate Systems. Don't
| get distracted by language features and syntactic sugars. If it
| works, people will learn.
| JustLurking2022 wrote:
| I've worked with large scale monorepos in FAANG - as a rule,
| they all suck. Eventually you always end up in this place where
| you have to understand, and be prepared to debug the entire
| universe just to build hello world.
| ontouchstart wrote:
| Totally agree. For complicated system, this essential
| complexity won't go away no matter how you make the UX
| "friendly". But good tooling will help users grow to
| understand the challenge of the problems as well as to
| appreciate the power of the solution. For people who are
| really interested in the solution, they will eventually dive
| deeper: https://edolstra.github.io/pubs/phd-thesis.pdf
| [deleted]
| Arcuru wrote:
| Since Eelco (often treated as Nix's BDFL) and Graham (also very
| prominent in the Nix community) are involved hopefully this group
| will actually be able to make traction on some of these issues.
| This is not even close to the first of these posts saying "we're
| going to help make Nix more user friendly!", but maybe this group
| has a chance of making changes to the Nix ecosystem since they
| can just approve it themselves.
|
| I've had concerns about Nix's governance in the past but it
| sounds like they may be going in the right direction, so I'm
| excited to see what they're planning.
| skybrian wrote:
| The external dependency problem is somewhat solved in npm (at
| least, as far as many users are concerned) by writing a module
| that downloads the appropriate binary.
|
| For example, esbuild is written in Go, which is compiled to a
| different binary for each system. The NPM for esbuild has 21
| optional dependencies, one for each binary that it makes
| available. A post-install script [1] chooses which dependency to
| install.
|
| It seems like a lot of work for the maintainer? But most users
| don't need to care.
|
| It probably helps that the Go SDK builds static binaries.
|
| [1] https://github.com/evanw/esbuild/blob/master/lib/npm/node-
| in...
| ParetoOptimal wrote:
| > The external dependency problem is somewhat solved in npm (at
| least, as far as many users are concerned) by writing a module
| that downloads the appropriate binary.
|
| This complicates working on node projects with nix in my
| experience.
|
| Is there a way to tell npm "no, just use my binary"?
|
| Best i've found is letting npm download it and then overwriting
| the npm binary with the nix one in a postinstall script.
|
| Why you ask? Npm doesn't always have an aarch64 or arm32
| binary.
|
| Nix being a source based package manager typically does support
| it or at least makes it possible by overriding build/configure
| options.
| skybrian wrote:
| If you know how to build the binary from scratch, could you
| teach the npm package's continuous build to do it? Sending a
| pull request fixing the npm package would fix it for
| everyone, not just nix users.
|
| Also, I wonder if nix would be a suitable tool for generating
| npm packages that contain binaries? Maybe this is a way for
| people to benefit from nix without actually using it.
| lykahb wrote:
| The name Nix is used for the language, package system, and
| sometimes the whole ecosystem. There is a recent post that tells
| the difference https://www.haskellforall.com/2022/08/stop-
| calling-everythin...
|
| Knowing functional programming helps but Nix still is a hard
| language to learn. I have a lot of Haskell experience but the Nix
| language was very confusing for me. At first I couldn't even tell
| a variable name from a keyword (it is a convention to call
| variables "self" and "super" in some contexts). The Nix language
| has been created for research in build systems and has evolved
| over a long time, so it has accumulated quite a lot of
| idiosyncrasies and cruft.
|
| I hope that a simpler typed language like
| https://github.com/tweag/nickel replaces it.
| Jeaye wrote:
| Writing the packaging expressions should not be harder than
| writing the program, for the average developer. The average
| developer doesn't know FP, at this point. Nix needs:
|
| 1. Approachability for those not indoctrinated in lazy,
| declarative, functional programming (i.e. Haskell); nope, Nix
| Pills are not sufficient for the average dev
|
| 2. Editor tooling to help guide the writing of expressions (just
| as anyone learning C# or Rust can use LSP); a better type system
| for Nix would help here
|
| 3. Better documentation for practical things like "Using Nix to
| manage the dependencies and package a C++ program using Meson",
| rather than having people piece this together from a bunch of
| disparate docs
|
| 4. A much better CLI UX than `nix-env -qa` and the like (this is
| ongoing and experimental, but even that broke recently, causing
| lots of confusion; now it requires `nix --extra-experimental-
| features nix-command --extra-experimental-features flakes search
| nixpkgs`)
|
| 5. To seriously answer the question: is the Nix language required
| for the Nix packaging system to exist? Laziness is required, to
| some degree, but can the next iteration provide an on-ramp which
| doesn't involve learning a new lang and paradigm? Guix folks sure
| think so.
|
| I feel like Nix folks have been focused so long on solving the
| tough problems of declarative, deterministic packaging that they
| haven't been able to focus on the UX. I also feel like folks for
| whom Haskell is comfortable may not realize just how absurd it
| feels to everyone else. Perhaps like the early days of Git.
|
| I do really hope Nix succeeds in this, though; I've been using
| it, or it's been using me, for several years. [1] and [2] for
| more info.
|
| [1]: https://blog.jeaye.com/2015/11/24/nixos/
|
| [2]: https://blog.jeaye.com/2017/07/30/nixos-revisited/
| avgcorrection wrote:
| > Writing the packaging expressions should not be harder than
| writing the program, for the average developer.
|
| Nah. The logical conclusion of this is that Nix should be some
| Java-like, Python-like, or whatever else qualifies as something
| that the average developer knows.
|
| Nix needs to be declarative which rules out the average PL
| paradigm.
|
| > [...] , but can the next iteration provide an on-ramp which
| doesn't involve learning a new lang and paradigm? Guix folks
| sure think so.
|
| Does the average developer know Scheme? I think you just played
| yourself.
| seqizz wrote:
| > I also feel like folks for whom Haskell is comfortable may
| not realize just how absurd it feels to everyone else.
|
| Yup. I can't even imagine what could I accomplish if I'd have
| something like a "python library" for nixpkgs. I still use
| nixos but gave up fiddling with nix for a while, after trying
| to wrap some stuff I am using/developing.*
|
| * (Yes, I know it's easy. I just need flakes or correct amount
| of "lib: prev: { self"s. Sadly not for me.)
| amelius wrote:
| Also: it should be able to deal with NVidia cards.
| ParetoOptimal wrote:
| NixOS handles nvidia cards better than any linux distro i've
| ever uses including Debian, Ubuntu, and Archlinux.
|
| I've only had to add a few lines to my configuration over 10
| PCs and laptops.
| amelius wrote:
| Ok, that is good to know. But how well does it run on
| NVidia's Jetson line of systems?
|
| https://developer.nvidia.com/embedded-computing
| theptip wrote:
| Having just done some prototyping with Nix, this list is spot
| on in enumerating the pain points I hit.
|
| I think most of the docs problems could be solved with the
| meta-strategy of "deeply understand all the things the Rust
| ecosystem got right with docs, discovery, and general
| pedagogical structure, and replicate that". Actually have a
| canonical set of docs. Link liberally between packafes and
| stdlib. Figure out typing so you can get those hints and links
| in your IDE. Fix the package docs so they actually tell you
| what functions the packages export, instead of relying on the
| wiki for instructions. Etc.
|
| Basically, pay a couple excellent docs people for a few years.
|
| I found the language a bit weird but not terrible to learn. But
| if it's possible to build some porcelain for common usecases
| that skips there need to grok the language, that could be quite
| transformative. I do wonder if the "dev env/packaging" use-case
| might be simpler than the "run and configure a whole OS"
| usecase.
| techdragon wrote:
| Point 4 is what lost me when I tried using NixOS for a machine
| I explicitly wanted as reproducible as possible. I needed to
| get a bunch of stuff working, then freeze/bundle/lock the
| useland as a documented working state I could rebuild later if
| anything went wrong.
|
| When documented examples began steering me towards the still
| experimental nix flakes and using the sorts of with ever more
| flags required commands, I bailed and built it using salt. I
| want to like nix but the current state of things was a massive
| put off.
| dial9-1 wrote:
| flakes are anything but experimental at this point, they've
| been out for many years and I pretty much use them for
| everything. you don't have to type those flags if you add
| them to the configuration file
| mook wrote:
| A newcomer is going to treat anything documented as
| experimental as that, regardless of what the community
| thinks; if it's no longer experimental, it needs to be
| presented as such at all the normal entry points. (None of
| this is unique to Nix; imagine you needed to start
| packaging for Windows for some reason.)
| ParetoOptimal wrote:
| > When documented examples began steering me towards the
| still experimental nix flakes
|
| There's a push to make them not be experimental anymore since
| so much of the community including myself is using them.
| rssoconnor wrote:
| > To seriously answer the question: is the Nix language
| required for the Nix packaging system to exist? Laziness is
| required, to some degree, but can the next iteration provide an
| on-ramp which doesn't involve learning a new lang and paradigm?
| Guix folks sure think so.
|
| I'd love to hear from someone deeply familiar with Nix and Guix
| about laziness.
|
| I'm deeply familiar with Nix and I've concluded that lazy
| semantics is absolutely critical for a configuration language.
| It lets me refer to other attributes of my configuration from
| anywhere. For example, I can refer to port number from my
| whatever service in my firewall. Nix's system of overlays
| depends on laziness too to provide efficient late-binding
| familiar from OOP.
|
| I don't need to topologically sort the evaluation of the
| various inter-dependencies of my configuration. So long as
| there exists an evaluation order, laziness finds it.
|
| Laziness is compelling enough that I managed to convince the
| author of Jsonnet <https://jsonnet.org/> of it when he was
| designing it, and in turn he helped me design what is now known
| as overlays in Nix.
|
| I don't even understand how Guix manages to work without
| laziness, though clearly it does somehow. I'm curious as to how
| that is possible, though I fear I will only ever truly
| understand by diving into Guix.
| infogulch wrote:
| I'm not convinced that any of the things you mentioned as
| benefits of lazy evaluation are a necessary condition to
| getting those benefits.
| mikepurvis wrote:
| As an 18 month Nix packager/user, I continue to be blown away
| by how powerful laziness is and how expressive the
| interactions between multiple overlays can be.
| jackac wrote:
| Recursion and merging semantics are also necessary. I use Nix
| and Jsonnet a lot, but Nix is much more expressive for
| complex structures, but the tooling being tightly coupled
| with the package manage make it impossible to adopt for
| common use cases.
| tikhonj wrote:
| Having used Nix a bunch, the system has been hard to use and
| learn in exactly the same ways that messy legacy codebases I've
| worked with have been hard--which makes sense since Nixpkgs is
| an absolutely massive codebase that's evolved over the years
| with very distributed development. People latch onto the Nix
| language and the paradigm because that's the most _tangible_
| difference but, everything else held equal, Nix + Nixpkgs would
| be no easier to work with if everything were in Python. (And,
| if my limited experience is any guide, would be _worse_ with an
| ad-hoc DSL _pretending_ to be Python a la Starlark!)
|
| This is a pattern I've seen over and over: if a project tries
| to do something different, the novel aspect gets blamed for
| issues that would not be surprising in any other context. When
| faced with a massive awful codebase in Java, well, sometimes
| legacy code can be a pain, right? It is what it is. People
| rarely jump to "OOP is a mistake" or "Java is completely
| unusable". But that's exactly what happens all the time with,
| say, functional programming.
|
| Pointing to an uncommon choice as the underlying cause of a
| problem makes for a nice, pat story; the pervasive, diffuse
| problems inherent to messy legacy code do not. After all,
| here's a single thing that, presumably we can actually change!
| Nix being functional is an objective fact while people can--
| reasonably or not--disagree on how well-designed and well-
| written Nixpkgs-the-codebase is. Besides, even if we could
| agree, what could we do about it? We all know how much momentum
| legacy code has and how hard it is to make anything beyond
| local improvements. Being a socially convenient story doesn't
| mean it's a particularly strong explanation, but it does mean
| that people will believe it even if it isn't.
| qbasic_forever wrote:
| Yeah IMHO nix just needs someone like Brian Kernighan to sit
| down and write the definitive book about it and its
| internals. Something that clearly explains the practical and
| theoretical uses of nix and nixpkgs.
|
| Nix pills, the official manual, etc. aren't really there yet
| --they talk about all kinds of paradigms that might (?) be
| going away like nix-env and its ilk. It's just extremely
| painful to track down the current best practice and way to do
| things in nix--you're bouncing off all kinds of docs and
| finding multiple conflicting or out of date ways to do
| things. Someone needs to give folks a slim book about nix
| flakes and the flakes way of doing things, the nix language,
| and a tour of nixpkgs, and IMHO there will be a lot more
| people using it.
| barkingcat wrote:
| I dislike that the way nix is onboarded for new folks is
| nix-env, and then for the next documentation to say: nix-
| env is bad don't use it.
|
| If it was bad why show it to me at all?
| catern wrote:
| Fascinating take! I've never heard this view before, thank
| you for sharing it!
| rgoulter wrote:
| > they haven't been able to focus on the UX. I also feel like
| folks for whom Haskell is comfortable may not realize just how
| absurd it feels to everyone else.
|
| The majority of communication I've seen from Nix maintainers
| over the last few months has been about exactly this.
|
| The submitted post is about exactly that while Nix powerful,
| they want more people to be able to use it without having to
| pay a steep penalty to learn it.
| tinco wrote:
| It also doesn't have anything to do with Haskell or it
| looking somewhat like Haskell. I'm very comfortable in
| Haskell, and Nix looks awful to me. It's not that it's
| functional or it's declarative, it's that it's badly designed
| and not intuitive, making it hard to remember and therefore
| awkward to code in.
| rgoulter wrote:
| I agree with the preamble and the "our mission" stuff. I wish
| them success at bringing the benefits of nix to more users, but
| without it being limited to enthusiasts, and people who can
| afford to deal with rough edges.
|
| The "external dependencies" thing shows how Nix can be a tough
| sell to those who don't click with it.
|
| I'd say that the other solutions are:
|
| - a README which has a bunch of "apt-get install" commands you
| can copy-paste from,
|
| - a "setup.sh" script that installs things,
|
| - a container or VM image with the toolchain setup,
|
| - or a tool like asdf.
|
| IMO, Nix is a much nicer solution than these other options, and
| has the benefits, avoids the downsides of these (but an entirely
| different set of downsides).
| wizwit999 wrote:
| How about Bazel, doesn't it solve the same issues?
| rrgok wrote:
| Beside being invasive on MacOS, as said by @ridiculous_fish, it
| took me more than 3 hours (and it didn't yet finish, I just quit
| all) to use QMK. I just cloned the Github repository of QMK and
| did `nix-shell` as they provide shell.nix file.
|
| 1. Does every nix-shell require building the whole world from
| ground-up? Seems impractical to me. 2. What is the right
| approach?
|
| This is not to bash Nixpkgs, because I installed NixOS and it
| took me 10min to install a whole OS with Sway, Neovim and some
| other tools. I guess I'm doing something wrong. But, on MacOS,
| nixpkgs was not a pleasant experience at all.
| rgoulter wrote:
| > 1. Does every nix-shell require building the whole world from
| ground-up? Seems impractical to me.
|
| It's the same as homebrew:
|
| If there's a cached binary already, it will download the cached
| binary. If not, it will fetch the source and build it.
|
| The QMK nix shell needs different GCCs for different targets.
| For the QMK shell.nix specifically, you can disable the targets
| you don't need (e.g. if you don't use a teensy board).
| rrgok wrote:
| That makes sense. Is there a `nix-shell` option that outputs
| which packages will be compiled from source and which will
| get a precompiled binary? Something like `nix-shell --dry-
| run`?
|
| I have to learn more about the nix language, but looking at
| the shell.nix seems that I have comment/remove lines from 25
| to 30 according to my board.
| ParetoOptimal wrote:
| > Something like `nix-shell --dry-run`?
|
| Exactly that? Lol :)
| rrgok wrote:
| Really? It is not written in the official nix-shell
| manual
|
| https://nixos.org/manual/nix/stable/command-ref/nix-
| shell.ht...
|
| I still will try when I get to my macbook. Thank you!
| SAI_Peregrinus wrote:
| Just tried it, it does work. Not in the `--help` though.
| ParetoOptimal wrote:
| I misread that as `nix-build --dry-run`, sorry!
|
| You can use nix-build on a shell.nix though i'm pretty
| sure.
| rgoulter wrote:
| The easiest is to change the default: { avr
| ? true, arm ? true, teensy ? true }:
|
| Change `true` to `false` as needed, only keep the ones you
| need for you keyboard(s).
| rrgok wrote:
| Thank you, I will try again with your suggestions!
| ParetoOptimal wrote:
| > 1. Does every nix-shell require building the whole world from
| ground-up? Seems impractical to me.
|
| Agreed! Most repos with a shell.nix and heavy dependencies use
| their own binary cache with the service cachix.org (free for
| public repos).
|
| > But, on MacOS, nixpkgs was not a pleasant experience at all.
|
| Were there reasons besides the QMK nix shell building the
| world?
| ParetoOptimal wrote:
| > Agreed! Most repos with a shell.nix and heavy dependencies
| use their own binary cache with the service cachix.org (free
| for public repos).
|
| Created an issue:
|
| https://github.com/qmk/qmk_firmware/issues/18248
| 0x457 wrote:
| Nix is source-first, but with binary cache. Many things have to
| be installed again on non-NixOS because nix is hermetic - it
| can't use library X from underlying system, it has to come from
| nix store.
|
| MacOS has issues with binary cache because of apple's
| requirement on where macOS can be run.
| ianai wrote:
| I hope this goes the way of RedHat/SUSE. i.e. The corporate side
| of a FOSS project so that contracts can be signed for appropriate
| purposes, etc.
| ghoward wrote:
| Disclaimer: I'm working on a build system that will eventually do
| what Nix does but make it much easier to use.
|
| If they do manage to create a system where Nix is hidden, and end
| users never have to directly touch it, I think this could work
| and make my work never see the light of day.
|
| But I have my doubts that they will be able to do that, and it
| boils down to one simple reason: declarative is not powerful
| enough.
|
| Don't get me wrong, for 90%, possibly more, of use cases, it's
| enough. And it is preferable to keep things declarative as long
| as possible, so much so that my build system will have a way of
| restricting itself to purely declarative code when possible (and
| will error otherwise).
|
| However, when more power is necessary, it is _required_ ; it is
| not possible, by the definition of Turing-completeness and
| declarative, to do the same thing with a declarative language
| that can be done with a Turing-complete language. So when that
| power is missing, the only option is to work around it.
|
| This is why Nix is really just a front for bash. For all of its
| faults, at least bash is Turing-complete.
|
| But this fact means that there will be things that Nix will fail
| to do on its own. I suspect this means that the abstraction
| around Nix will always be leaky. Maybe it won't be unacceptably
| leaky, but I'm not very hopeful.
|
| I think Nix would be better served by doing the following:
|
| 1. Rewriting the language. This would require an auto-transformer
| to the new language in order to not throw away the entirety of
| nixpkgs, but transforming a declarative language to a Turing-
| complete one is easy.
|
| 2. Spend gobs of time on user experience. Make the usual commands
| short, easy to remember, and easy to use with few options. Make
| usual things easy and powerful things safe. For example, to make
| a powerful thing safe, make sure that screwing it up will not
| screw up their system.
|
| 3. But not only should they _be_ safe, they should _feel_ safe,
| giving users every opportunity to back out without consequences.
| This is where git goes wrong: it does not feel safe, so users are
| scared of it. [0] I believe Nix is the same way, even though it
| is safer than git.
|
| 4. Spend gobs of time on documentation. Use Fred Rogers' list of
| rules for talking with children [1] in each piece of
| documentation. This will make it easier to avoid the trap of not
| explaining something the user needs because the writer forgot
| that they didn't know it. In essence, the documentation should
| treat new users as Fred Rogers treated children: ignorant, but
| capable.
|
| 5. The documentation also needs to have a different focus.
| Instead of focusing on how great Nix is, it should focus on
| helping users get stuff done. Nix enthusiasts should be able to
| say, "Oh, you want to set up a development environment for your
| project? Great, go to such-and-such tutorial. It will tell you
| exactly how to do that even if you don't have Nix installed."
| This should be done for as many use cases as people have,
| including less common ones. Some examples: using Nix as a build
| system, using Nix to install multiple versions of glibc and how
| to switch between them, using Nix to set up systemd, using Nix to
| replace a Docker container in production, using Nix to distribute
| builds, etc.
|
| [0]: https://xkcd.com/1597/
|
| [1]: https://www.mentalfloss.com/article/547536/mr-rogers-
| rules-f...
| nonsequitur wrote:
| Nix is turing complete (it can implement the Lambda Calculus),
| even conveniently so.
| ghoward wrote:
| Then people shouldn't say it is, such as in the article under
| discussion.
| vdijkbas wrote:
| > This is why Nix is really just a front for bash. For all of
| its faults, at least bash is Turing-complete.
|
| Nix is Turing-complete. This was even sometimes considered a
| problem because it makes the language too powerful:
|
| https://nixos.org/~eelco/talks/guix-feb-2018.pdf
| ghoward wrote:
| This is news to me, I admit. But if this is true, then people
| shouldn't say Nix is declarative.
| throwamon wrote:
| Although it's debatable, it's very common for functional
| languages to be categorized as declarative.
| ParetoOptimal wrote:
| > However, when more power is necessary, it is required; it is
| not possible, by the definition of Turing-completeness and
| declarative, to do the same thing with a declarative language
| that can be done with a Turing-complete language. So when that
| power is missing, the only option is to work around it.
|
| I fall heavily into the use less powerful languages and
| "liberties constrain, constraints liberate" camps.
|
| I'd be very interested in a real world example of this, though
| I know that class of examples is hard to come by.
| ghoward wrote:
| > I fall heavily into the use less powerful languages
| "liberties constrain, constraints liberate" camp.
|
| For what it's worth, so do I. That's why my build system will
| allow users to set those constraints. It's a great idea.
|
| > I'd be very interested in a real world example of this,
| though I know that class of examples is hard to come by.
|
| There are many, but I'll say one: anything that requires
| iteration until a fixed point is reached. A concrete example
| of this is building documentation in LaTeX.
|
| Or it may even be simpler than that. Maybe your build
| requires some sort of special code should two arguments have
| specific values. In other words, edge cases often require
| Turing-compete power to resolve.
| HowTheStoryEnds wrote:
| Would you consider lisp to be (possibly) declarative?
| ghoward wrote:
| No, it is Turing-complete.
|
| I was going off of the article and Nix documentation that
| says Nix is declarative. But if it actually is not, then
| there shouldn't be a problem with power.
|
| However, Nix proponents shouldn't claim it is declarative
| either.
| setheron wrote:
| Funny enough I introduced Nix to our company which was acquired
| by Google -- so Google has software leveraging Nix. (I got Google
| to in fact sponsor Nix which was nice too. A small amount but it
| was meaningful to me).
| beermonster wrote:
| Is it Nix or NixOS?
| qbasic_forever wrote:
| Both exist. Nix is a functional language for describing how
| software is built. Nix is also a package manager of sorts that
| installs and manages software built with its language. NixOS is
| an operating system that is described entirely by a nix
| language configuration file that builds a bootable linux-based
| OS using nix language packages.
| ridiculous_fish wrote:
| I installed nix on my Mac but quickly backed out due to the
| complexity. I assumed the nix store would just be an ordinary
| directory with a tool for managing it, similar to brew. I
| discovered it creates a new Unix group, adds a separate APFS
| volume, installs a daemon. This was too invasive for a tool I was
| unsure if I even wanted to use, so I uninstalled it.
|
| What is the reason for all this machinery? I went with the
| recommended multi-user install, should I have just used the
| single user mode instead?
| edolstra wrote:
| It's because Nix wants to install into /nix. Once upon a time
| doing "sudo mkdir /nix" wasn't a problem, but recent macOS
| releases have made that very hard.
|
| Nix could switch to an alternate location on macOS (e.g.
| /opt/nix) but that has a lot of downsides for interoperability
| with other systems.
| qudat wrote:
| Yikes, I had no idea. What a tough pill to swallow all
| because the directory nix runs out of is hardcoded.
| Macha wrote:
| This is inherent to having binary packages. Binaries
| compiled to look for libraries in one location cannot just
| be copied to a system with a different layout. Nix also
| can't just use the OS conventions, since part of the point
| of Nix is that it does not use the global system state and
| only its own isolated, reproducible world. So then they had
| to make a decision and decided to just use /nix everywhere
| which worked fine for some years. And now MacOS has
| changed, choosing something else would invalidate all
| historical binary packages.
| vamega wrote:
| It's not hardcoded, but you end can't take advantage of the
| binary caches if you change the directory. A company could
| certainly create their own binary cache and distribute that
| to it's users.
| rnhmjoj wrote:
| Not entirely true, there are many ways in which you can
| use a custom location and still take advantage of the
| binary cache. You can do it with chroot, file system
| namespaces, bind mounts and so on. There's also a nice
| user friendly tool that does exactly this [1].
|
| [1]: https://github.com/DavHau/nix-portable
| grhmc wrote:
| On Linux, users can put their Nix store in their home
| directory or other places and at run-time Nix remaps the
| directory using user namespaces. Unfortunately this isn't
| workable on macOS: the kernel doesn't support the features
| we need.
|
| Using /nix and a separate group and daemon means the store
| can be read-only and be protected from modification in
| several ways. This is pretty helpful, as a lot of tools try
| very hard to write "next to" where they are installed --
| corrupting the Nix store.
|
| I sort of wonder if it would be more palatable if the Nix
| installer was a bit less in your face about what's going
| on? This would be similar to how Docker's works.
| pseudalopex wrote:
| It would be less palatable when I found out. The group is
| fine. Why the daemon when other package managers use sudo
| is unclear. Even Homebrew moved to /opt.
| kaba0 wrote:
| It's not actually hardcoded, but you would have to compile
| everything from scratch as the hash would depend on that as
| well, throwing out the whole binary cache.
| wmertens wrote:
| Actually, there's quite a few hard-coded references to
| /nix in nixpkgs I believe. It's possible but a little
| more work than just changing the prefix.
| grhmc wrote:
| There may be, but at least one BigCorp deploys Nix to a
| different prefix.
| ParetoOptimal wrote:
| > What a tough pill to swallow all because the directory
| nix runs out of is hardcoded.
|
| Yikes? Well then... how would you solve remote binary
| caching with something like Nix on a platform such as OSX
| without userns remapping support?
| 0x457 wrote:
| APFS volume is required to have a read-only nix-store at /nix.
| The daemon is only required in multi-user installation, and you
| can opt out of it (by using a single-user install). New groups
| are for letting people submit jobs to builders in multi-user
| mode.
| Macha wrote:
| Note that the multi-user installation is these days the only
| suggested approach by the Nix team for new MacOS
| installations - the old instructions for single user
| installation on MacOS have been removed from their site.
| scoates wrote:
| It's because builds are completely reproducible, and therefore
| they have to be built in the same place (in `/nix`), and you
| can't create directories off of `/` on modern Mac setups.
| rgoulter wrote:
| > What is the reason for all this machinery?
|
| Nix wants to use /nix, but Apple wants macOS to be locked down
| and secure.
|
| The compromise was putting /nix on a separate volume.
|
| > I discovered it creates a new Unix group, ... installs a
| daemon
|
| This is at about the same level as Docker, fwiw.
| chrisseaton wrote:
| > The compromise was putting /nix on a separate directory.
|
| Why does a separate directory need a whole other volume?
| These are just files, right? Why do they need to be on a
| separate volume?
| Cloudef wrote:
| Many binaries when compiled gets hardcoded paths compiled
| into them which are determined during configuration time.
| This means you cant use binary from a cache if the file
| hierarchy isnt identical. On linux this isnt a problem due
| to namespaces (feature originating from plan9).
| ParetoOptimal wrote:
| OSX requires mounting root folders like /nix with
| synthetic.conf IIRC.
| chrisseaton wrote:
| Right but the comment said they compromised on using a
| different directory in the end, so it could be under /opt
| like Homebrew, and not a root directory.
| rgoulter wrote:
| One of Nix's benefits is it aims to isolate where
| programs are installed; so you can have multiple versions
| of the same program available without conflict. (e.g.
| programs built with different feature flags, or multiple
| minor versions).
|
| Nix achieves this by storing packages in a path as some
| hash of its inputs. -- This then allows either compiling
| or downloading a package, with confidence that it will
| behave the same way regardless.
|
| But, since the files were under /nix, if you put them
| under /opt, then you wouldn't be able to make use of the
| compilation caches for /nix.
| chrisseaton wrote:
| I guess I don't understand what you meant by 'The
| compromise was putting /nix on a separate directory'
| then?
| rgoulter wrote:
| Oops. Wrong word. I meant "separate volume".
|
| /nix could still be used if it's on a separate volume.
| Which still seems less nice than just having /nix on the
| same volume.
| danieldk wrote:
| _But, since the files were under /nix, if you put them
| under /opt, then you wouldn't be able to make use of the
| compilation caches for /nix._
|
| Until a very short time ago there was no binary cache for
| Apple Silicon Macs.
| nodejsthrowaway wrote:
| ParetoOptimal wrote:
| The install process was simpler before recent OSX updates.
|
| > I discovered it creates a new Unix group, adds a separate
| APFS volume, installs a daemon. This was too invasive for a
| tool I was unsure if I even wanted to use, so I uninstalled it.
|
| Since you'd already installed it, wouldn't trying it in some
| capacity before uninstalling it have made sense?
|
| > What is the reason for all this machinery
|
| Enforcing reproducibility basically.
|
| I don't know all but...
|
| The daemon and APFS volume otherwise readonly /nix that only
| the daemon can write to can't be created.
|
| The group is probably for the daemon to be able to write to
| /nix.
|
| The path /nix is important because the remote binary cache
| paths will miss otherwise and you'll compile everything from
| source.
|
| > I went with the recommended multi-user install, should I have
| just used the single user mode instead?
|
| I'm guessing it would work in the way you want, but I always
| opt for the daemon.
| ridiculous_fish wrote:
| > Since you'd already installed it, wouldn't trying it in
| some capacity before uninstalling it have made sense?
|
| Fair observation. I installed nix as a prerequisite for
| DevBox, discussed here
| https://news.ycombinator.com/item?id=32600821 . I thought
| DevBox sounded really cool (and still do!), but the
| Quickstart took frustratingly long, and ended up not working.
| Faced with the prospect of debugging it, I opted to cut my
| losses and uninstall it instead.
|
| That said I'm very much open to trying nix again in the
| future. Also I want to acknowledge how much effort went into
| getting /nix to work on the Mac; it appears that was a heavy
| lift indeed.
| grhmc wrote:
| Single-user Nix installs are lighter weight, but also have a
| major compromise: the store is writable by the user. This means
| it is pretty easy for software you're using to accidentally
| change the store underneath Nix, and create unpredictable and
| unreproducible behavior.
|
| I think the multi-user install is a better _Nix_ experience,
| even if the install process is spookier.
|
| Like is mentioned elsewhere, Docker does similar contortions to
| install itself. I wonder if it would have been more palatable
| if the Nix installer was less forward about what it is doing?
| ParetoOptimal wrote:
| > I wonder if it would have been more palatable if the Nix
| installer was less forward about what it is doing?
|
| I bet most wouldn't consider it spooky if the installer
| didn't print it out.
|
| If docker started printing it, I bet lots of people would
| complain it seems similarly complex.
| rgoulter wrote:
| I recall the sqlite changed their tmpfiles on Windows from
| "sqlite" to "etilqs".
| tadfisher wrote:
| > What is the reason for all this machinery?
|
| > new Unix group
|
| This is used for the daemon, so it doesn't run as root and
| expose your system to Nix build code.
|
| > adds a separate APFS volume
|
| I think this is required because of macOS security restrictions
| preventing direct modification of the root directory. The Nix
| store has to be housed in /nix because all references to
| runtime dependencies in the store are absolute paths in /nix,
| and that can't really change per system because it would break
| caching and reproducibility. The separate volume is added to
| /etc/synthetic.conf so it can live in /nix.
|
| > installs a daemon
|
| For multi-user installs, this allows non-privileged users to
| add build outputs to the Nix store, which ultimately allows
| these users to share build outputs. More info here:
| https://nixos.org/manual/nix/stable/installation/multi-user....
| danieldk wrote:
| _The Nix store has to be housed in /nix because all
| references to runtime dependencies in the store are absolute
| paths in /nix,_
|
| This is true, but in some sense Nix on Apple Silicon was a
| missed opportunity. It started as a blank slate (fresh binary
| cache) and it would've been a good opportunity to move the
| store to a writable path like _/ opt/nix_. This would have
| solved the whole dance needed with volumes and
| _synthetic.conf_. I know that there are infrastructure issues
| (like Hydra using a single Nix store), but it would 've made
| the macOS Nix story so much better.
|
| It's a shame that only Apple can make firmlinks, because that
| would've been another possible solution ( _/ nix_ could be a
| firmlink to the actual store location).
|
| The same problem occurs on e.g. Fedora SilverBlue, because
| you can also not make arbitrary root directories. But at
| least on Linux, you don't want to throw away more than one
| decade of a x86_64 binary cache.
| wmertens wrote:
| Actually the cache isn't that important, most binaries get
| rebuilt quite often due to their dependencies changing.
|
| Although it is my hope that rfc 17 eventually makes it
| through: https://github.com/wmertens/rfcs/blob/master/rfcs/
| 0017-inten...
| danieldk wrote:
| _Actually the cache isn 't that important, most binaries
| get rebuilt quite often due to their dependencies
| changing._
|
| Sure. But it's still nice that e.g. on Linux x86_64 you
| can use a nixpkgs commit from 2015 and get all stuff from
| the binary cache.
| tadfisher wrote:
| > It started as a blank slate (fresh binary cache) and it
| would've been a good opportunity to move the store to a
| writable path like /opt/nix.
|
| That breaks caching cross-compiled outputs and x86_64
| outputs, which do still run on Apple Silicon.
| oxff wrote:
| Make the documentation up to the modern standards. If I have to
| open a single random blog or Github repository to find out what
| to do and piece it together like a puzzle - it is impossible to
| adapt at organizational scale.
|
| It is also complex enough to require a modern Language Server.
___________________________________________________________________
(page generated 2022-09-02 23:00 UTC)