[HN Gopher] The Unreasonable Effectiveness of Makefiles
___________________________________________________________________
The Unreasonable Effectiveness of Makefiles
Author : srijan4
Score : 163 points
Date : 2022-08-12 13:49 UTC (9 hours ago)
(HTM) web link (matt-rickard.com)
(TXT) w3m dump (matt-rickard.com)
| LAC-Tech wrote:
| I wanted to learn makefiles to generate a static website.
|
| Quickly ran into some massive limitations - one of which is that
| it completely broke apart when filenames had spaces in them.
|
| "But why would you do that you're doing it wrong" - don't care
| wanted spaces in filenames.
|
| Ended up switching Rake (a make like DSL written in ruby) and
| never looked back. Not only can you do all the make declarative
| stuff, but you get the full power of ruby to mess around with
| strings.
| rightbyte wrote:
| Next thing you ask for leading tabs in filenames ...
|
| And the feature bloat slippery slope is sliding towards Bazel!
| Mister_Snuggles wrote:
| My most horrible abuse of `make` was to write a batch job runner.
|
| Most of the targets in the Makefile had a command to kick off the
| job and wait for it to finish (this was accomplished with a
| Python script since kicking off a job involved telling another
| application to run the job) followed by a `touch $@` so that make
| would know which jobs it had successfully run. If a process had
| dependencies these were declared as you'd expect.
|
| The other targets in the Makefile lashed those together into
| groups of processes, all the way up to individual days and times.
| So "monday-9pm" might run "daily-batch", "daily-batch" would have
| "daily-batch-part-1" (etc), and each "daily-batch-part-..." would
| list individual jobs.
|
| It was awful. It still is awful because it works so well that
| there's been no need to replace it. I keep having dreams of
| replacing it, but like they say there's nothing more permanent
| than a temporary solution.
|
| All of this was inspired by someone who replaced the rc scripts
| in their init system with a Makefile in order to allow processes
| to start in parallel while keeping the dependencies in the right
| order.
| jbboehr wrote:
| > All of this was inspired by someone who replaced the rc
| scripts in their init system with a Makefile in order to allow
| processes to start in parallel while keeping the dependencies
| in the right order.
|
| Any sufficiently complicated init system contains an ad hoc,
| informally-specified, bug-ridden, slow implementation of half
| of systemd.
| ithkuil wrote:
| My most horrible abuse of make was a distributed CI where I put
| a wrapper in the MAKE env var so that recursive make executions
| would invoke my wrapper which would enqueue jobs for remote
| workers to pick up
| [deleted]
| scott_s wrote:
| Congrats! You invented Apache Airflow:
| https://airflow.apache.org
| Mister_Snuggles wrote:
| Interesting:
|
| > Airflow was started in October 2014 by Maxime Beauchemin at
| Airbnb. It was open source from the very first commit and
| officially brought under the Airbnb GitHub and announced in
| June 2015.
|
| I believe I starting building my tool somewhere around 2010,
| possibly 2011. The core mechanism has been completely
| unchanged in that time. If Airflow was a thing at the time,
| I'd have hopefully looked into it. I looked at a handful of
| similar products and didn't find anything that was a good
| fit.
|
| Based on a really quick skim of the Airflow docs it seems
| like it checks all of the boxes. Off the top of my head:
|
| * LocalExecutor (with some degree of parallelism, assuming
| the dependencies are all declared properly) seems to do
| exactly what I want.
|
| * I could write an Operator to handle the interaction with
| the system where the processes actually run. The existing
| Python script that does this interaction can probably get me
| 90% of the way there. Due to the nature of what I'm running,
| any job scheduler will have to tell the target system to do a
| thing then poll it to wait for the thing to be done. To do
| this without any custom code, I could just use BashOperator
| to call my existing script.
|
| * It's written in Python, so the barrier to entry (for me) is
| fairly low.
|
| * Converting the existing Makefile to an Airflow DAG is
| likely something that can be done automatically. We
| deliberately keep the Makefile very consistent, so a
| conversion program can take advantage of that.
|
| I think my dream of replacing this might have new life!
| rhacker wrote:
| But airflow is an abomination... that I am forced to use at
| my current job.
| MonkeyMalarky wrote:
| With the added bonus of not having to learn or maintain
| Apache Airflow!
| cerved wrote:
| That's beautiful, not awful
| josteink wrote:
| > All of this was inspired by someone who replaced the rc
| scripts in their init system with a Makefile in order to allow
| processes to start in parallel while keeping the dependencies
| in the right order.
|
| Sometimes the most interesting thing is not the story itself,
| but the story _behind_ the story.
|
| This has my interest _peaked_. Is there anywhere else I can
| read about this?
| pjot wrote:
| Just a friendly tip that it's _piqued_ :)
| shawabawa3 wrote:
| You never know, maybe it is the peak of their interest and
| it's all downhill from here
| [deleted]
| topspin wrote:
| My most elaborate use of make was ten years ago to build a
| bootable, reproducible embedded system image including the OS,
| application, media files and cryptographic signatures. It's still
| in active use and no one is complaining.
| hyperupcall wrote:
| Honestly, I only find Makefiles useful when I have a tiny C/C++
| project and need stuff just to compile quickly and easily without
| the overhead of a real build system.
|
| For literally everything else, I found myself using it more as a
| task runner - and Make doesn't do a great job at it. You end up
| mixing Bash and Make variables, string interpolation, and it
| becomes really messy, really fast. Not to mention the footguns
| associated with Make.
|
| I found bake (https://github.com/hyperupcall/bake) to suit my
| needs (disclaimer: I wrote it). It's literally just a Bash script
| with all the boilerplate taken care of you - what a task runner
| is meant to be imo
| evilotto wrote:
| The 2 biggest things I find missing in makefiles is the ability
| to have the "modified date" of a target be something other than
| the mtime of the target file, and for the dependencies of a
| target to be generated rather than statically specified. The
| latter there are workarounds for, but not the former. The
| author's suggestion of "better support for make docker" is
| completely the wrong thing to do - you don't need to support
| docker, you need to support mtimes and dependencies in a way that
| docker could use.
| h4l wrote:
| I've found these guidelines for Makefiles make for a pretty good
| experience using make: https://tech.davis-hansson.com/p/make/
|
| The advice on output sentinel files for rules creating multiple
| files helps keep rebuilding dependencies reliable. Avoiding most
| of the cryptic make variables also helps Makefiles to remain
| easily understandable when you're not frequently working on them.
| And using .ONESHELL to allow multi-line statements (e.g. loops,
| conditional etc) is great. No need to contort things into one
| line. or escape line breaks.
|
| Seems like you could even use a more serious programming language
| instead of sh/bash by setting SHELL to Python or similar. That
| may be a road to madness though...
| carapace wrote:
| > Seems like you could even use a more serious programming
| language instead of sh/bash by setting SHELL to Python or
| similar. That may be a road to madness though...
|
| TIL. SHELL=/usr/bin/python .ONESHELL:
| all: @from plumbum.cmd import ls
| print(ls["-a"]())
|
| It totally works... _Mwoooo ha ha ha haaaa!_
| zelphirkalt wrote:
| The problem with .ONESHELL is, that it is for the whole file. I
| so wish it was per target. That would be really useful. But for
| the whole file? Maybe I need each line to be a separate shell
| anywhere in the file and that will make it impossible to use
| .ONESHELL for the entire file.
| h4l wrote:
| It is per target, that's how it works when I've used it. For
| example: # Makefile SHELL := bash
| .ONESHELL: .RECIPEPREFIX = > thing1:
| > FOO=bar > echo "$${FOO:?}" .PHONY: thing1
| thing2: > echo "$${FOO:?}" .PHONY: thing2
|
| Results in: $ make thing1 thing2
| FOO=bar echo "${FOO:?}" bar echo "${FOO:?}"
| bash: line 1: FOO: parameter null or not set
|
| Note how the bash error for unset FOO is line 1 for the
| second target.
|
| Edit: Maybe I misinterpreted, do you mean you'd want to
| choose whether a given target is ONESHELL or not?
| philsnow wrote:
| > File-watcher or live-reloading. You can create a build/deploy
| loop fairly easily
|
| When I worked with latex more, I kept a ~/.Makefile-latex and a
| shell function that would pretty much just do
| inotifywait -e modify -e move --recursive --monitor . | while
| read; do make --makefile ~/.Makefile-latex; done
|
| and I kept emacs and xpdf in side-by-side windows. Whenever I'd
| save a file in emacs (or xfig or whatever), a second later xpdf
| would refresh the pdf (and stay on the same page). It took away
| some of the pain of working with latex.
|
| _edit: I used this complicated setup instead of LyX or whatever
| other "(la)tex IDE" because I had ancillary files like xfig
| diagrams that would get compiled to .eps files and gnuplot
| scripts that would render data into graphs, and the makefile knew
| how to generate everything._
| kwantam wrote:
| Very nice! It turns out that latexmk has this functionality:
| latexmk -pvc -pdf foo.tex
|
| (It can be configured to HUP your pdf reader if needed, too.)
|
| I usually add something like this command as the `auto` target
| in my latex Makefiles, which works pretty nicely.
| earthscienceman wrote:
| This is funny, because I just wrote a (fish) shell script that
| does this as well because all of the tex IDEs are so painful.
| Mostly because the entire efficiency of latex is that you're
| editing text and can do it in a _text editor_ like emacs and
| move things around very quickly. I don 't want a new interface!
|
| But. I'm kind of proud. My shell script monitors the tex files
| for character changes and then once a (configurable) threshold
| of changes is met, it kicks off the compilation process. But
| the real game changer is that every time it compiles, if the
| compile is successful it commits the recent edits to a git
| branch. Then if I want, I can go through the git branch and see
| the entire history of edits for only versions of the document
| that compiled. It's a game changer in a big way. When I finish
| a section, I squash the minor edits into one commit and give it
| a good message and then commit the full thing to the main
| branch. Then there is where I can make sure my manuscripts look
| the way they should and do major revisions or collaborative
| edits.
|
| The icing on the cake is that the fish script monitors for a
| special keypress and will take actions. So I hit "c" to compile
| the tex, "b" to rerun the massive bibliography compilations,
| "s" to squash the commits into the main branch, and "l" to
| display log errors. It's a dream! Now I don't think about
| compilation at all, or when I should commit something minor to
| git (and fiddle with the commands). I just type away and
| watch/request the pdf refresh when I need it... and _actually_
| get work done. My god. So happy.
|
| I just finished this today.
| shepherdjerred wrote:
| I'd encourage anyone thinking of using make to look at
| alternatives. Make is great, but is quickly becomes a ball of
| duct-tape. Make works very well when you spend the time to
| express your dependency tree, but realistically that never
| happens and people tend to add hacks upon hacks for Makefiles.
| Not only that, but they don't scale well as your project adds
| more components, such as integration testing, documentation, etc.
|
| I found Earthly[0] to be a great replacement. Everything runs in
| Docker. Your builds are reproducible, cache-able, and
| parallelizable by default. I've heard Dagger[1] is another good
| tool in the space.
|
| [0]: https://earthly.dev/
|
| [1]: https://dagger.io/
| keepquestioning wrote:
| Now we need to banish CMake
| Koshkin wrote:
| Yeah, there is something deeply wrong about the whole thing...
| But what do we replace it with? (Out of desperation, perhaps, I
| even dreamt of a build system based on a C library, with the
| usage being, say, tcc -run mybuild.c
|
| :)
| mdaniel wrote:
| People keep saying that, but of the systems I've encountered
| that use CMake they've worked the most predictably, as opposed
| to autotools, meson, scons, bazel, or heavenforbid ./build.sh
|
| *WRITING* CMake is the 11th circle of hell, but CLion is
| gradually getting more support for it because they use(d?) it
| as the first-class project definition when it launched
|
| I would pay good money for CMake 4.x to switch to
| skylark/starlark
| klodolph wrote:
| Make is fantastic at what it does well.
|
| These days, I would absolutely not use Make to compile code
| written in C, except for the smallest personal projects. It is
| just too fussy to construct the Makefile correctly, in a way that
| you can get correct incremental builds. Nearly any other build
| system is better at building projects written in C, in the sense
| that it is easy & straightforward to get your build system to do
| correct incremental builds.
| EddySchauHai wrote:
| Absolutely! Basically every company I've worked with over the
| last couple years as a contractor followed this methodology and
| it's grown to be my default runner as it's language agnostic.
| properparity wrote:
| > incremental builds
|
| I've found that to not matter that much these days - unless
| your projects is hundreds of thousands (or maybe millions even)
| of LOC, full builds are instant on modern machines.
| klodolph wrote:
| That's definitely not true for:
|
| - Most C++ or Rust projects
|
| - Medium size or larger C projects
|
| - Anything which is built with tools written in JavaScript
| (due to Node startup time overhead)
|
| Stuff where a full build is close enough to instant:
|
| - Most Java, C#, Go projects
|
| - Small C projects
|
| - Tiny/trivial C++ or Rust programs, or C++ programs written
| for embedded systems
|
| This is just my experience. YMMV.
| stjohnswarts wrote:
| any sizeable rust or c++ project is gonna take a while to
| compile in my experience? I tend to break things off into
| libraries that are faster to compile. Some of my coworkers
| don't like it but they learn to deal (c++)
| cassepipe wrote:
| Advertisement time (not affiliated, just want to share the joy)
| : I personally use Xmake and try to advertise it every time I
| get the chance : FOSS, no DSL it's just Lua, dead simple yet
| featureful, and it is ninja fast, or at least claims to be I
| never bothered to check that out, it's fast enough for me.
|
| https://xmake.io/#/
| est31 wrote:
| Make is solving many complicated tasks, like keeping in mind when
| to re-run some target (there is communication going on with the
| compilers that provide make .d files so that they know which
| source files influence a binary), or running job servers managing
| parallelism that also support nested sub-makes. But it also has
| many ugly warts. It's hard to design something that solves both
| those tasks as well as make does, and also as _generalist_ as
| make does. Often they are solving a subset, what currently itches
| the main developer. But something that is both as general, and as
| comprehensive as make, those tools are rare. Ninja for example
| checks many of the boxes, but lacks make jobserver support.
| bXVsbGVy wrote:
| Ninja looks clean because it is new. Give it some decades and
| it is likely inherit a few of warts make has.
| aappleby wrote:
| It's not that new, and in practice there aren't very many
| ways for it to get warty because it's basically a dependency
| graph with "run this command line if this node is dirty"
| attached to the edges.
|
| I like it very much.
| belkarx wrote:
| There was a post on HN (I believe) a couple of months ago
| concerning a research paper that explores "What defines a
| makefile" concluding that by their definition, Excel can be used
| as one. I've looked around the internet and can't find it, so if
| anyone else remembers it and has it upvoted or otherwise has the
| link, posting it would add to this conversation. It had
| interesting discussion of graphs and requirements and was overall
| a worthwhile read.
| yaris wrote:
| Was it "Build Systems a la Carte" paper? I could not find
| mentions of it on HN after 2020 though.
| belkarx wrote:
| Yes, thank you!
|
| Here's the PDF in case anyone else is interested:
| https://www.microsoft.com/en-
| us/research/uploads/prod/2018/0...
|
| I can't find any actual conversation about it on HN so I
| won't post the HN link
| eichin wrote:
| Aww, noone mentioned that `debian/rules` files are almost always
| makefiles? (To the point of starting with `#!/usr/bin/make -f`
| ...)
| aappleby wrote:
| Recently I've been much more pleased by the "Unreasonable
| Effectiveness of Ninja Plus A Few Trivial Scripts".
|
| A raw Ninja file is verbose but easy to read and understand, and
| there is basically no magic happening behind the scenes (well,
| except for "deps = gcc", but that's minor). Any action that's too
| complicated to go directly in the Ninja file goes to a "rule
| command \ command = ${command}" that just runs a python or shell
| script.
|
| The only thing I'd like to add to my setup would be "Ninja with
| glob support", but again that can be handled in a few lines of
| Python that spit out another chunk of .ninja rules so it's not
| really a blocker.
| falcolas wrote:
| I love Make. It's a terrible tool for quite a few things, but
| it's awesome at the thing I use it most for - abstracting away
| complex series of shell commands behind one or two words. It's
| like shell aliases that can follow a repo anywhere.
| make test make format make clean
| make docker-stack
|
| Fantastically useful stuff, even if all it's doing is calling
| language specific build systems in the background.
| JackFr wrote:
| Never loved make. First used it in the early nineties and found
| the syntax obscure and error messages cryptic.
|
| My response to this article would be, if make is so great why did
| they have to invent 'configure' and 'xmkmf'? And why do people
| continue to create new build tools every couple of years?
|
| Yeah, I mean I guess it worked, but unreasonably effective?
| Hardly.
| jhallenworld wrote:
| Eh, they solve different problems. Make is too simple to
| customize your build to deal with system differences- it just
| builds your code.
| falcolas wrote:
| > why did they have to invent 'configure'
|
| Cross-architecture and linux distro compatibility, mostly.
| compiler-guy wrote:
| Err, pedantically, configure was not for cross Linux distro
| compatibility, but for cross unix compatibility. It existed
| long before Linux was a sparkle in Linus's eye.
|
| And even then, it handled even some non unix environments as
| well.
| bch wrote:
| > ... why do people continue to create new build tools every
| couple of years?
|
| Seems like a rite[0] of passage to some degree. Perhaps similar
| to people talking a stab at The Next Actually Correct CMS, and
| The Next Object System That Doesn't Suck, or The Next Linux
| Distro For Smart People.
|
| [0] edit: corrected "right V. rite" per
| https://news.ycombinator.com/item?id=32442473
| erik_seaberg wrote:
| (btw, it's https://en.wikipedia.org/wiki/Rite_of_passage)
| stjohnswarts wrote:
| i've turned to cmake to do some really weird dependency
| management for various script calling. It's much more
| scriptable/friendly than make in its modern form but obviously
| no python :)
| ogogmad wrote:
| Is there a Python library that can check a timestamp and update
| some files according to a lambda? I don't want to learn Make
| syntax again.
| Jtsummers wrote:
| if os.stat(object).st_mtime < os.stat(dependency).st_mtime:
| ...
|
| Use a dict to contain each rule: rules["a.c"] =
| (["b.c", "c.c", "b.h", "c.h"], action) ...
|
| That can be simplified as well and then run a simple parser
| over it that takes a simpler representation and turns it into
| that dict. Then: def make(rules, rule):
| (deps, action) = rules[rule] run_rule = len(deps) == 0
| # if there are no dependencies, the rule always runs
| for dep in deps: make(rules, dep) if
| os.stat(rule).st_mtime < os.stat(dep).st_mtime: run_rule = True
| if run_rule: action()
|
| Of course, this doesn't actually validate the DAG itself.
|
| ----------
|
| An amendment: You'll probably want to pass both the rule name
| and the dependencies into the action function/lambda/object so
| that you can parameterize it and maybe reuse the action (like a
| common compiler command): if run_rule:
| action(rule, deps)
| badrabbit wrote:
| Autogen/autotools mess however... not so pleasant when it breaks.
|
| Curious for those who published source with autotools,etc... do
| you really sit down and figure that out. It's just a mystery that
| works or doesn't to me. I never have or would go beyond a simple
| make file. It really seems tedious on top of the actual code you
| write. A bit impressive tbh.
| jbboehr wrote:
| Yes, I used autotools[0]. It's definitely hairier than plain
| make, but you get a lot of useful features on top of it.
| There's thousands of examples all over the internet so it's
| easy to reference them.
|
| I like the elegance of pure make, and do use it when
| appropriate, but I wouldn't really want to reimplement the
| things autotools does myself in it.
|
| [0]:
| https://github.com/jbboehr/handlebars.c/blob/master/configur...
| thinkingkong wrote:
| I like make. But these days to me the best part about it is that
| it's a common entry point. Most popular languages come with their
| own make-esque tools that provide the same experience to
| developers and systems.
|
| Tying together multiple projects, source from different
| locations, etc Id probably use make or a script.
| dijit wrote:
| A company I worked for used make to execute docker, which felt
| somewhat odd.
|
| Correct me if I'm wrong, but I always assumed `make` to be one of
| those build tools that could incrementally build targets based on
| dependencies.
|
| The arcane and esoteric language that constitutes `make` is
| almost universally avoided, surely if people need a simple task
| runner there could be better options?
| falcolas wrote:
| Better? Absolutely.
|
| More ubiquitous? Not really.
|
| I use it for executing docker as well, because the docker
| command is actually about 5 commands with long lists of
| parameters. But it's just `make docker-stack` for everyone now.
| jwilk wrote:
| It's 500 Internal Server Error for me.
|
| Archived copy:
|
| https://web.archive.org/web/20220812135641/https://matt-rick...
| jcoq wrote:
| This is a remarkably stupid comment but not everything is
| "unreasonably effective". Mathematics was noted as unreasonably
| effective for modeling the universe because most areas of
| mathematics were not invented for the applications they meet...
| like discovering that your coffee maker doubles as an
| exceptionally good waffle maker.
|
| Makefiles on the other hand, are not unreasonably effective in
| this sense. Makefiles are, in fact, _reasonably_ effective...
| like a coffee maker that brews coffee well.
| [deleted]
| bjourne wrote:
| The short article conflates popularity with quality. Windows 3.11
| became the most sold os in history despite being utter trash.
| Make is popular because it was the first build system, not
| because it is not utter trash.
| BiteCode_dev wrote:
| Just got out of a python training session with one of my
| student running w11. Can confirm. So many problems.
|
| Seems like one version out of 2 of windows being trouble stills
| stand.
| Nexialist wrote:
| Slightly tangential but I've worked for several companies now
| that use `make` as a simple command runner, and I have to say
| it's been a boon.
|
| Being able to drop into any repo at work and expect that `make
| init`, `make test` and `make start` will by convention always
| work no matter what the underlying language or technology is, has
| saved me a lot of time.
| ReadTheLicense wrote:
| This is standard in Node.js ecosystem and I love it. Each
| package has scripts in package.json that you can run with _npm
| run [name]_ , and some of these like start, test or build (and
| more) are standardized. It's really great DX.
| patrickthebold wrote:
| But it's npm, so when you switch to a java project, for
| example, you have different commands.
| r3trohack3r wrote:
| Quite a few companies I've contracted with have lifted the
| pattern up into Bazel or Gnu Make - for node projects `make
| lint` can be a pass through.
|
| In the project repo, either work.
| txutxu wrote:
| Conventions are great, but that doesn't look like anything
| specific to make, a shell wrapper could do that:
| #!/bin/sh case $1 in init)
| ... do whatever for each project init ;;
| start) ... do whatever for each project start
| ;; test) ... do whatever for each
| project tests ;; *)
| echo "Usage: $0 init|start|test" >&2 exit 1
| ;; esac
|
| In my home/personal projects I use a similar convention (clean,
| deploy, update, start, stop, test...), I call those little sh
| scripts in the root of the repo "runme".
|
| The advantage could be, maybe, no need to install make if not
| present, and no need to learn make stuff if you don't know it.
|
| Sometimes they don't match the usual words (deploy, start,
| stop, etc) but then I know that if I don't remember them, I
| just type ./runme and get the help.
|
| For my scenario, it's perfect because of it's simplicity.
| kazinator wrote:
| You can make "make init" work on Windows _and_ Unix if you
| work at it, out of the same Makefile.
|
| The above won't.
| dymk wrote:
| A shell wrapper could do that, but Makefiles are a DSL to do
| exactly that with less boilerplate.
| marcosdumay wrote:
| And have a nice inbuilt graph runner if you decide one task
| depends on another...
| tom_ wrote:
| Complete with automatic parallelization if you ask for
| it! And automatic KEY=VALUE command line parsing, default
| echoing of commands (easily silenced), default barf on
| subprocess failure (easily bypassed). The variable system
| also interacts reasonably sensibly with the environment.
|
| I've never rated Make for building C programs, but it's
| pretty good as a convenient cross-platform shell-agnostic
| task runner. There are also several minimal-dependency
| builds for Windows, that mean you can just add the exe to
| your repo and forget about it.
| marcosdumay wrote:
| To tell the truth, make sucks incredibly for building
| modern C programs. There are just too many targets. It's
| why all of them generate their makefile with some
| abomination.
|
| But it is still a great task runner.
| natrys wrote:
| Tbf, that particularity is easily achieved in shell
| scripts too: task1() { echo
| hello } task2() {
| task1() echo world }
| "$@"
| garblegarble wrote:
| But now update it to not re-run tasks unnecessarily -
| it's already wordier than a shell script right now.
|
| Meanwhile, in Make that's task1:
| echo hello task2: task1 echo
| world
| natrys wrote:
| True, that's where Make shines. Though given the
| popularity of so many Make alternatives (the strictly
| subset of command runner variety, like just[1]) who keep
| its syntax but not this mechanism, I wonder if for
| command runner unnecessarily re-running dependencies is
| really a big deal. Because quite often the tasks are
| simple and idempotent anyway, and then it's a bit of a
| hassle to artificially back the target by a dummy file in
| Make (which your example doesn't do here e.g.).
|
| [1] https://github.com/casey/just
| whateveracct wrote:
| make gives you autocomplete more easily for free. One reason
| I use it always.
| sanderjd wrote:
| This was the nicest thing about blaze at google. I'm a big
| believer that having a single standard tool for things is a
| huge value add, regardless of what the tool is. I didn't really
| like blaze particularly, and I don't really like make
| particularly, but it's _amazing_ to just have a single standard
| that everybody uses, no matter what it is.
| pornel wrote:
| Rust's Cargo has the same appeal. There are 90,000 libraries
| that support cargo build/doc/run/test with no fuss.
| shoo wrote:
| I've worked on a few projects that apply this pattern of using
| a Makefile to define and run imperative commands. A few people
| develop the pattern independently, then it gets proliferated
| through the company as part of the boilerplate into new
| repositories. It's not a terrible pattern, it's just a bit
| strange.
|
| For many junior colleagues, this pattern is the first time
| they've ever encountered make -- hijacked as some kind of
| imperative command runner.
|
| It's quite rare to run into someone who is aware that make can
| be used to define rules for producing files from other files.
|
| I find it all a bit odd. Of course, no-one is born knowing
| about middle-aged build tools.
| arinlen wrote:
| > _It 's quite rare to run into someone who is aware that
| make can be used to define rules for producing files from
| other files._
|
| Is it, though?
|
| That's literally what Make does as part of its happy path.
|
| GNU Make even added support for pattern rules, as this use
| case is so pervasive.
|
| What do you think people think make is about?
| shoo wrote:
| oh i agree, that's why i find the situation odd!
|
| i'm talking working on projects with people whose first
| encounter with make is in a project where someone else has
| defined a Makefile to wrap imperative actions, e.g. `make
| run-unit-tests`, `make deploy`. If they think about make at
| all, there's a good chance they think make is for
| performing imperative actions, and has nothing specifically
| to do with producing files from other files using rules and
| a dependency graph, or the idea of a target being a file,
| or a target being out of date.
| 3836293648 wrote:
| This is what I do for all non-rust projects. I knew what it
| was supposed to do, but wow if it took me forever to figure
| out how to do it (the connection between rule name and file
| name is really poorly documented in tutorials, probably
| should've just read the man page)
| jbboehr wrote:
| Yeah, I did this too. It's not that surprising considering
| that typically end-users only interact with phony targets
| (all, clean, install, etc).
| pak9rabid wrote:
| $ make run-dev
|
| That command (to run an Angular/nodejs dev instance has staved
| off carpel-tunnel syndrome for me for maybe another 5 years.
| shadowgovt wrote:
| Does make still basically fail to handle filenames with spaces in
| them?
|
| That was the deal-breaker for me last I checked.
| davidpfarrell wrote:
| As these types of post often come around to make's sub-optimal
| use as general runner, I'd like to point out my project, Run:
|
| https://github.com/TekWizely/run
|
| It feels like a makefile but is optimized managing and invoking
| small tasks and wrappers, and auto-generates help text from
| comments.
| [deleted]
| PaulKeeble wrote:
| Its unfortunate that other build systems haven't taken over. Make
| is terrible for incremental builds and its reliance on binaries
| often means issues getting it to run and being very platform
| dependent. It is better than using a bat or shell file for the
| same purpose but its a long way behind many of the other language
| specific tools. I am surprised something better hasn't become
| popular, Make is the CVS of the build tools.
| olliej wrote:
| I find a bunch of the decisions in Make to not be great - but
| it's all just syntax stuff.
|
| At its core all a Makefile is is a dependency graph, which is
| necessary in all the more advanced config managers, only those
| others are much more heavyweight than the vast majority of
| projects ever need. Most code doesn't need to vary arguments or
| parameters based on the target environment, and so that
| complexity is unneeded, in which case a Makefile can do it all.
| BiteCode_dev wrote:
| Make dsl is terrible, it's the yaml of build systems.
|
| If you are a python dev, give doit a try.
| Koshkin wrote:
| What is so "terrible" about it? I think the rule syntax is as
| simple as it gets.
| vitiral wrote:
| Required tabs, lines are each their own "script" instead of
| blocks (allowing variables), not allowing other executors
| (i.e. python, TCL, etc would be better than sh).
| nottorp wrote:
| Interesting that everyone who recommends an alternative to Make
| picks... something different. I doubt there will be two people
| recommending the same thing in the whole discussion.
| qbasic_forever wrote:
| For a task runner I really like just and its Justfile format:
| https://github.com/casey/just It is heavily inspired by make but
| doesn't focus on the DAG stuff (but does support tasks and
| dependencies). Crucially it has a much better user experience for
| listing and documenting tasks--just comment your tasks and it
| will build a nice list of them in the CLI. It also supports
| passing CLI parameters to task invocations so you can build
| simple CLI tools with it too (no need to clutter your repo with
| little one-off CLI tools written in a myriad of different
| languages).
|
| If most of your make usage is a bunch of .PHONY nonsense and
| tricks to make it so developers can run a simple command to get
| going, check out just. You will find it's not difficult to
| immediately switch over to its task format.
| gurgeous wrote:
| Seconded - I love just & Justfile. Such an upgrade after trying
| to force things into package.json scripts. Chaining commands,
| optional CLI arguments, comments, simple variables, etc. Very
| simple and a breath of fresh air.
| davidpfarrell wrote:
| For those looking for a powerful task runners that feel like a
| makefile, please take a look at Run:
|
| https://github.com/TekWizely/run
|
| It's better a managing and invoking tasks and generates help
| text from comments.
| kbd wrote:
| Just seems neat, but except for the dependencies it's just a
| way to package multiple shell scripts into one file, no?
|
| I've thought about trying it out a few times but can never see
| its value over scripts in ./bin.
| qbasic_forever wrote:
| Scripts in bin have no documentation, no easy way to
| enumerate them, etc. There is definitely a time and a place
| for bin scripts, especially as things grow in complexity.
| However the beauty of just is that there's one file (the
| justfile) that defines all of your project's actions. You
| don't have to go spelunking into bin to figure out how to
| tweak a compiler flag, etc. And since just will run anything
| there's no reason why your complex bin scripts can't just be
| called from a simple one liner task in a justfile.
|
| Could your write a bash script that does stuff like enumerate
| all the bin scripts, pull out documentation comments, etc.?
| Absolutely, and people have followed that pattern for a while
| (see https://github.com/qrush/sub) but it's a bunch of
| boilerplate to copy between projects. Just pulls out that
| logic into a simpler config file.
| niedzielski wrote:
| Just looks soooo promising! I don't think I can use it until
| conventional file target and dependencies are supported though.
| Right now everything's tasks (phonies) so conventional makefile
| rules like the following are impractical: tic-
| tac-toe: tic.o tac.o toe.o cc -o '$@' $^
| %.o: %.c; cc -c $^
| qbasic_forever wrote:
| You might find checkexec useful to pair with just, it is
| basically a tool that only does the file-based dependency
| part of make: https://github.com/kurtbuilds/checkexec
| dahfizz wrote:
| I don't understand the use case of `just`. It drops every
| useful feature from `make`. It doesn't look like it has
| parallelism or the ability to not needlessly re-run tasks.
|
| Even if `just` was installed on a standard Linux box, I don't
| see the benefit of it over a bash script.
| throwaway787544 wrote:
| Declarative isn't important. I wish people would stop talking
| about it. It's not even useful to think about most of the time
| because of how many ways it can be interpreted. It's a
| thereotical categorization, not a functional design principle.
|
| Just make the program do useful things for the user. Make the
| computer work for the human rather than the other way around.
| shepherdjerred wrote:
| > Make the computer work for the human rather than the other
| way around.
|
| Ironically is a very concise definition of `declarative`.
| samatman wrote:
| This is almost too devoid of content (opinion is not that) to
| usefully reply, and I suspect you're gesturing at some more
| specific point I might agree with, or at least understand.
|
| Declaration is absolutely a coherent and functional design
| principle. A declarative system is one in which the outcome is
| specified and the process is not.
|
| This has big payoffs in domains where it's natural. A good
| example being grammars. It also has hazards, a good example
| being the performance of algorithms to parse grammars.
|
| We can see where a declarative build system might be a mixed
| bag, because the process itself is imperative: make is an early
| attempt to reconcile imperative build processes with a
| declaration of what circumstances require their triggering.
| Basically every build system since make has improved on make,
| but they all do more-or-less what make does.
|
| The design, in short, is proven, as well as declarative but not
| purely so.
|
| And the ability to reason about the degree to which make is
| declarative shows that 'declarative' is in fact a coherent
| idea. But you can't declare software into existence, you must
| compile it.
| nimih wrote:
| I dunno, the "declarative-ness" of `make` is a pretty important
| component of its usefulness. In particular, the property of
| `make`, wherein the structure/details of the computation is
| implicit in the provided configuration and invoked command
| rather than being explicitly written down somewhere, is central
| to its utility, since the alternative of just writing a shell
| script is anecdotally a much less popular option. If you want
| to propose a more appropriate word to describe such a property
| which is less buzzwordy, feel free, but in the context of "why
| does `make` have such enduring popularity", I think the
| article's author is being quite reasonable in bringing it up.
| throwaway787544 wrote:
| You could replace the word "declarative-ness" with
| "automation" and it'd mean the same thing. And literally all
| configuration of functionality implies the structure and
| details of computation - that's the point of configuration,
| to tell an already assembled program that already has
| structure and computational details what to do with it.
| There's no overt distinction between "declarative
| programming" and "configure a function with value X".
|
| Makefiles are simply configuration files that use whitespace
| and a couple characters to create the configuration, and what
| Make's inbuilt functions do with that determine the extent to
| which the result becomes "more intelligent". Yes they are
| used to build a graph and execute it, but so is Dotfile
| notation, and software package configuration files. But we
| don't call those declarative programming. Many of those
| configuration files create multiple levels of instructions
| and require several passes to execute properly. But we just
| call them "config files" because we don't feel they are
| intellectually superior enough to be called a form of
| programming. And on the other hand, we don't call declarative
| programming "configuration", but they're often the same
| thing.
|
| Nobody says they "imperatively configure" some software, but
| they do consider themselves "declaratively configuring" it.
| Because they've overloaded the word "declare" as if it means
| something other than "write down a thing I want a computer to
| eventually do with some automation". People bring up the
| declarative thing because they want to imagine there's some
| intellectual value to considering it, but there isn't. You're
| basically saying "I want to configure a program rather than
| write one". Which is fine. But just say that and stop
| pretending that's going to immediately lead to a better
| result.
| Banana699 wrote:
| Configuration doesn't have to be declarative, for example
| see https://lukeplant.me.uk/blog/posts/less-powerful-
| languages/, in particular the section about python
| configuration, where an imperative configuration language
| is discussed. How imperative? Very. It's a line-oriented
| language, where the program reads a line and changes
| something in an internal data structure accordingly then
| continue reading the file. This is imperative, the person
| writing the configuration file has to think about state and
| time while writing the file, not just abstract goal states
| and facts.
|
| Declarative vs. Imperative is a spectrum. For instance,
| there is a declarative language hiding inside most
| imperative languages : Infix Math. 1+2*3/71**7 is
| declarative because it under-specifies the order of
| operation, only the data flow dependencies implied by
| operator precedence needs to be respected. In the
| precedence hierarchy I had in mind when I wrote it, You can
| do 2*3 first or 71**7 first, it's unspecified and
| irrelevant. I only ask that you do both before you perform
| the division of their results, and that the addition is the
| last operation. Meanwhile, in Forth, math is imperative,
| you have to unroll the expression tree into an exact
| sequence.
|
| Declarative is any language that under-specifies the task
| being described. Therefore, every language worth using is
| declarative to some degree or the other. After all, that is
| the very purpose of a high level language : to under-
| specify a task by describing only the most essential of
| details, all the abstracted details are taken care of by
| either inference (compiler figures it out, possibly
| according to rules that you need to be aware of) or
| exhaustive checking (compiler generates all possible cases
| and code to select among them at runtime, or very generic
| code that can handle all cases uniformly). If, like Alan
| Perlis says, "A low level language is that which requires
| attention to the irrelevant", then every good language is
| already declarative in some sense, you omit things and they
| get taken care of automatically, that's what Decorative
| means.
|
| You can say you hate buzzwords, I empathize. You can just
| say that make is bad software (trivially true, or we
| wouldn't have needed software to _generate_ makefiles,
| effectively making them a machine code that isn 't meant to
| be written by humans) and that being declarative doesn't
| make it any less bad. Declarative vs. Imperative are just
| names for design decisions, they guide a language designer
| but don't have the power to make a language good single-
| handedly.
| wnoise wrote:
| The builder pattern is imperative configuration.
| kazinator wrote:
| Anyone who finds make unreasonably effective must be working with
| GNU Make.
|
| If I had to use some barely POSIX conforming thing from BSD thing
| or wherever, I'd instead write a write a top-to-bottom linear
| shell script full of conditionals.
| makapuf wrote:
| In that case,a good habit is to name those GNUMakefile. Works
| the same, but announce gnu make (which is the one worth it)
| bXVsbGVy wrote:
| I'm trying to understand why so many people seems to hate make.
|
| I hate building system that don't use Makefile, or that use but
| don't respect the variable convention. It makes really quite
| annoying to do things like changing allocation library, add
| compilers flags, etc.
| TylerE wrote:
| Because gnu autotools blows goats. Oh, and the tab thing.
| gbrown_ wrote:
| make != autoconf
| TylerE wrote:
| Yes, but 99% of the makefiles you encounter in the wild
| comes from autotools.
| dima55 wrote:
| This isn't at all true, in my experience. If it's true
| for you, please consider that your issues are about
| autotools and not Make, and direct your complaints in
| that direction.
| stjohnswarts wrote:
| the vast majority of the ones I encounter are from
| cmake...
| tpoacher wrote:
| surely it blows gnus rather than goats?
| qbasic_forever wrote:
| As a build system make wasn't really designed to handle stuff
| like partial rebuilds, caching, or distributed building. Modern
| build systems like bazel are just orders and orders of
| magnitude faster and better for complex projects.
| dima55 wrote:
| Yeah. As far as I can tell, most people complaining about Make,
| and building its replacements haven't figured out how to use
| Make, or bothered to read the manual. It's really not that
| complicated...
| nhooyr wrote:
| 100% agreed. Half these comments make no sense and
| demonstrate a real ignorance of make. Please everyone read
| the manual and judge make for make, not autotools...
| [deleted]
| jhallenworld wrote:
| To go along with Make, use this technique to automatically
| generate and include dependency information:
|
| https://scottmcpeak.com/autodepend/autodepend.html
|
| And this, avoid recursive make:
|
| https://accu.org/journals/overload/14/71/miller_2004/
___________________________________________________________________
(page generated 2022-08-12 23:00 UTC)