[HN Gopher] Bun: Fast JavaScript runtime, transpiler, and NPM cl...
___________________________________________________________________
Bun: Fast JavaScript runtime, transpiler, and NPM client written in
Zig
Author : firloop
Score : 373 points
Date : 2022-07-05 20:41 UTC (2 hours ago)
(HTM) web link (bun.sh)
(TXT) w3m dump (bun.sh)
| cyansmoker wrote:
| Congrats!
|
| This is making me reconsider using the node ecosystem in some of
| my projects.
| switz wrote:
| I'm really excited about bun - it represents an opportunity to
| deeply speed up the server side javascript runtime and squeeze
| major performance gains for common, real-world uses-cases such as
| React SSR.
|
| Also, Jarred's twitter is a rich follow full of benchmarks and
| research. If nothing else, this project will prove to be
| beneficial to the entire ecosystem. Give it some love.
| Jarred wrote:
| I started building Bun a little over a year ago, and as of about
| 20 minutes ago, its in public beta
|
| One of the things I'm excited about is bun install.
|
| On Linux, it installs dependencies for a simple Next.js app about
| 20x faster than any other npm client available today.
| hyperfine "bun install --backend=hardlink" "yarn install --no-
| scripts" "npm install --no-scripts --ignore-scripts" "pnpm
| install --ignore-scripts" --prepare="rm -rf node_modules"
| --cleanup="rm -rf node_modules" --warmup=8 Benchmark
| #1: bun install --backend=hardlink Time (mean +- s):
| 25.8 ms +- 0.7 ms [User: 5.4 ms, System: 28.3 ms]
| Range (min ... max): 24.4 ms ... 27.6 ms 76 runs
| Benchmark #2: yarn install --no-scripts Time (mean +-
| s): 568.4 ms +- 15.3 ms [User: 781.6 ms, System: 497.4
| ms] Range (min ... max): 550.8 ms ... 604.5 ms
| 10 runs Benchmark #3: npm install --no-scripts
| --ignore-scripts Time (mean +- s): 1.261 s +-
| 0.017 s [User: 1.719 s, System: 0.516 s] Range
| (min ... max): 1.241 s ... 1.286 s 10 runs
| Benchmark #4: pnpm install --ignore-scripts Time
| (mean +- s): 1.343 s +- 0.003 s [User: 601.3 ms, System:
| 151.6 ms] Range (min ... max): 1.339 s ... 1.348
| s 10 runs Summary 'bun install
| --backend=hardlink' ran 22.01 +- 0.85 times faster
| than 'yarn install --no-scripts' 48.85 +- 1.51 times
| faster than 'npm install --no-scripts --ignore-scripts'
| 51.99 +- 1.45 times faster than 'pnpm install --ignore-scripts'
| panzerboiler wrote:
| Congratulations for the release! You are doing an impressive
| work with bun. I find particulary exciting the built-in sqlite,
| and I cannot wait to move all my projects to bun. Egoistically
| speaking (my 2012 mbp doesn't support AVX2 instructions), I
| hope that now that the project is public, since you are going
| to get a lot of issue reports about the failure on install, you
| will find some time to get back to issue#67. Thank you, and
| keep up the excellent work.
| andai wrote:
| Dang, nice work! Any idea where the slowness comes from in the
| other tools, how Bun manages to be so much faster?
| nicoburns wrote:
| Yeah, the install function of npm/yarn/pnpm are all _incredbly
| slow_. And also seems to get slower super-linearly with the
| number of dependencies. I have one project where it can take
| _minutes_ (on my 2015 MacBook - admittedly it's quicker on my
| brand new machine) just to add one be dependency and re-resolve
| the lock file. If that can solved by a reliable tool I'd
| definitely switch!
| oreilles wrote:
| This is one of if not the most insane thing in web dev at the
| moment. Git can diff thousand of files between two commit in
| less time than it takes to render the result on screen. But
| somehow it can take actual minutes to find out where to place
| a dependency in a simple tree with npm. God, why ?
| capableweb wrote:
| > it can take actual minutes to find out where to place a
| dependency in a simple tree with npm. God, why ?
|
| npm is famous for a lot of things & reasons, but none of
| those are "because it's well engineered".
|
| To this day, npm still runs the `preinstall` script after
| dependencies have actually been downloaded to your disk. It
| modifies a `yarn.lock` file if you have it on disk when
| running `npm install`. Lots of things like these, so that
| the install is slow, is hardly surprising.
| tempest_ wrote:
| People building language tooling often use the language
| itself, even if it is not very suitable for the task at
| hand.
|
| This happens because the tooling often requires domain
| knowledge which they have and if they set out to write
| tooling for a language they tend to be experienced in that
| language.
| nicoburns wrote:
| In fairness, I suspect your average node_modules folder has
| a lot more files than your average git repo (maybe even an
| order of magnitude more)
| zebracanevra wrote:
| It seems like bun caches the manifest responses. PNPM, for
| example, resolves all package versions when installing (without
| a lockfile), which is slower. The registry does have a 300
| second cache time, so not faulting you there, but it means your
| benchmark is on the fully cached path, which you'd only hit
| when installing something for the first time. Subsequent
| installs would use the lockfile and bun and PNPM seem fast* in
| that case.
|
| If I install a simple nextjs app, then remove node_modules, the
| lockfile, and the ~/.bun/install/cache/*.npm files (i.e. keep
| the contents, remove the manifests) and then install, bun takes
| around ~3-4s. PNPM is consistently faster for me at around
| ~2-3s.
|
| I'm not familiar with bun's internals so I may be doing
| something wrong.
|
| One piece of feedback, having the lockfile be binary is a HUGE
| turn off for me. Impossible to diff. Is there another format?
|
| * I will mention that even in the best case scenario with PNPM
| (i.e. lockfile and node_modules) it still takes 400ms to start
| up, which, yes, is quite slow. So every action APART from the
| initial install is much MUCH faster with bun. I still feel
| 400ms is good enough for a package manager which is invoked
| sporadically. Compare that to esbuild which is something you
| invoke constantly, and having that be fast is such a godsend.
| Jarred wrote:
| > It seems like the main thing that bun does to stay ahead is
| cache the manifest responses. PNPM, for example, resolves all
| package versions when installing (without a lockfile), which
| is slower.
|
| This isn't the main optimization. The main optimization is
| the system calls used to copy/link files. To see the
| difference, compare `bun install --backend=copyfile` with
| `bun install --backend=hardlink` (hardlink should be the
| default). The other big optimization is the binary formats
| for both the lockfile and the manifest. npm clients waste a
| lot of time parsing JSON.
|
| The more minor optimizations have to do with reducing memory
| usage. The binary lockfile format interns the strings (very
| repetitive strings). However, many of these strings are tiny,
| so it's actually more expensive to store a hash and a length
| separately from the string itself. Instead, Bun stores the
| string as 8 bytes and one bit bit says whether the entire
| string is contained inside those 8 bytes or if it's a memory
| offset into the lockfile's string buffer (since 64-bit
| pointers can't use the full memory address and bun currently
| only targets 64-bit CPUs, this works)
|
| yarn also caches the manifest responses.
|
| > If I install a simple nextjs app, then remove node_modules,
| the lockfile, and the ~/.bun/install/cache/.npm files (i.e.
| keep the contents, remove the manifests) and then install,
| bun takes around ~3-4s. PNPM is consistently faster for me at
| around ~2-3s.
|
| This sounds like a concurrency bug with scheduling tasks from
| the main thread to the HTTP thread. I would love someone to
| help review the code for the thread pool & async io.
|
| > One piece of feedback, having the lockfile be binary is a
| HUGE turn off for me. Impossible to diff. Is there another
| format?
|
| If you do `bun install -y`, it will output as a yarn v1
| lockfile.
|
| If you add this to your .gitattributes:
| *.lockb binary diff=lockb
|
| It will print the diff as a yarn lockfile.
| zebracanevra wrote:
| > If you add this to your .gitattributes:
|
| Not applicable to GitHub etc though.
|
| I'm also not seeing any speed differences when using
| -y/yarn lockfile. Why not make it the default?
| the_duke wrote:
| The benchmark source code links on the homepage are "Not
| found".
|
| Also a few questions:
|
| What do you attribute the performance advantage to? How much of
| it is JavascriptCore instead of v8 versus optimized glue
| binding implementations in the runtime? If the latter, what are
| you doing to improve performance?
|
| Similarly for the npm client: how much is just that bun is the
| only tool in a compiled /GC free language versus special
| optimization work?
|
| How does Zigs special support for custom allocators factor in?
| Jarred wrote:
| will fix the broken links shortly
|
| edit: should be fixed
| claytongulick wrote:
| It's a deeply impressive achievement, absolutely blows my mind
| that you were able to do it in a year.
| truth_seeker wrote:
| Roadmap goals looks promising, especially under the Runtime
| section.
|
| https://github.com/Jarred-Sumner/bun/issues/159
|
| Best of luck.
| adamwathan wrote:
| I've been following Jarred's progress on this on Twitter for the
| last several months and the various performance benchmarks he's
| shared are really impressive:
|
| https://twitter.com/jarredsumner/status/1542824445810642946
|
| Easy to dismiss this as "another JS build tool thing why do we
| need more of this _eye roll_ " but I think this is worth a proper
| look -- really interesting to follow how much effort has gone
| into the performance.
| slimsag wrote:
| Agreed, I've also been following his work for a while now.
| Jarred is pulling off something very impressive here :)
| Congratulations on the public beta!
| immigrantheart wrote:
| The best part of being a JavaScript programmer is that you have
| the entire world working for you for free.
| brunojppb wrote:
| This is an impressive achievement! Congrats Jarred for this
| initiative. This is going to help JS ecosystem to move further
| ahead.
| yes_but_no wrote:
| gratz, what was using Zig like? What kind of problems you had?
| bogwog wrote:
| > curl https://bun.sh/install | bash
|
| This type of thing needs to stop
| asadlionpk wrote:
| I disagree. This reduces friction for users. Users should be
| aware of what/who they are trusting.
| capableweb wrote:
| Ok, let's say I have a experimental tool I want to distribute
| to people with a easy install. What way would you prefer to
| install it?
|
| Going through package registries/repositories is a slow
| process, so obviously want something faster.
|
| Just GitHub releases? Would you be fine if the URL instead
| pointed to GitHub in that case?
| forrestthewoods wrote:
| All I want is a download link to a single .exe. Or a zip if
| it has runtime content/dependencies. GitHub release is fine.
| capableweb wrote:
| So then Bun is doing fine here? They have GitHub Release
| activated and are using it for releases.
| vore wrote:
| Why? If it's over TLS you can trust it's being served by the
| owner of the website. You're having to trust the person who
| wrote the script anyway. And before anyone says "I'm going to
| inspect the shell script before I run it", do also note that
| its purpose is to download and run binaries that you are never
| going to inspect.
| SahAssar wrote:
| IMO the main problem is that it isn't clear how updates will
| work. Some of the curl-to-bash scripts don't do anything in
| regards to updates at all, some add a PPA/similar on
| ubuntu/debian/fedora/etc.
|
| It'd be nice to know what and how I should manage updates.
| Arnavion wrote:
| The other thing to be careful about is that the script is
| written in a way that a truncated script has no effect. This
| is because sh will execute the stdin line-by-line as it reads
| it, so if the download fails in the middle of a line the
| downloaded fragment of that line will be executed.
|
| It can be the difference between `rm -rf ~/.cache/foo` and
| `rm -rf ~/`
|
| The standard way to solve this problem is to put the entire
| script inside a function, have the function name be the last
| line of the script, and name it in such a way that substrings
| of the name are unlikely to map to anything that already
| exists. Note that the bun.sh script does _not_ do this, but
| also from a quick glance the only thing that could noticeably
| go wrong is this line: rmdir
| $bin_dir/bun-${target}
|
| It might run `rmdir $bin_dir` instead, in which case it'll
| end up deleting ~/.bun instead of ~/.bun/bin/bun-${target},
| which is probably not a big deal.
| TimTheTinker wrote:
| True, the only real counterpoint is someone who clones the
| repo, inspects it, and builds from source.
| metadaemon wrote:
| You could add additional security to the process by first
| validating some cryptographic signature or verifying that the
| downloaded content's hash matches one that the author
| published.
|
| Both of those just push the overall security a bit down the
| line, but both are ultimately not completely safe. The only
| truly safe action to take is to not download it at all.
| vanviegen wrote:
| And we need to ban binary downloads from vendor sites as well,
| as those pose the exact same risks? Good luck with that!
| dementiapatien wrote:
| Techniques like this could make curl|bash more prone to
| malicious activity:
| https://www.idontplaydarts.com/2016/04/detecting-curl-
| pipe-b...
| vore wrote:
| You're running untrusted binaries anyway in the end, so I
| don't think this is anything more than a neat party trick.
| dementiapatien wrote:
| But this technique lets you serve malicious code to a
| small number of people using curl|bash, rather than
| hosting obviously-bad binaries that anyone can inspect
| and call you out on. It also lets you target the attack
| to specific users or IP blocks.
|
| The previous HN discussion said it better than I can:
| https://news.ycombinator.com/item?id=17636032
| [deleted]
| metadaemon wrote:
| Yeah, people should be using fish! /s
| 0des wrote:
| someone demonstrated a while back how based on user agent you
| could serve innocuous code for a browser checking the code
| first, and then a different malicious payload for curl.
|
| thanks to dementiapatien below for the link
| https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
| WaffleIronMaker wrote:
| Note that the bun install [1] seems to be hosted as an HTML file,
| not as a text file. I'm not sure to what extent that causes
| issues, but it seems atypical.
|
| [1] https://bun.sh/install
| alexarena wrote:
| Congrats! Cool to see a new class of JS runtimes springing up.
| Lots to be excited about here, but cold start time seems like a
| game changer for building at the edge.
| jakearmitage wrote:
| Any special magic going on with the interpreter code? Did zig
| allow you to write a more performant parser/AST walker?
| fuu_dev wrote:
| Zigs makes cross compiling easier and is on par with c/c++ in
| terms of performance. There is no magic trick else just
| personal preference i assume.
| conaclos wrote:
| Bun parser is a translation in Zig of ESbuild's parser. ESbuild
| parser is already well tuned. Bun takes zig advantages to go
| further.
| mwcampbell wrote:
| The benchmark numbers for React server-side rendering are really
| impressive. How does bun manage to be so much faster, especially
| considering that React SSR is actually fairly compute-intensive
| (building up a DOM tree and all)?
| gardenhedge wrote:
| Awesome. The homepage explains it very well which is impressive.
| tylerchurch wrote:
| Any detailed comparison of this vs. Node.js vs. Deno?
| throwawaymaths wrote:
| I did a quick comparison for my own reasons of using them as an
| "absolutely stupid runner" which boots a fresh VM, runs some
| JavaScript that converts a piece of JSON, and gets out (this is
| likely mostly measuring VM boot and cleanup only). Bun was
| crazy fast: a factor of 2 over libmozjs, and a factor of 3 over
| nodejs
| ccheever wrote:
| I've been following this project for a while now, and it's
| incredibly ambitious. It will take a while to reach its full
| potential but Jarred is doing an extraordinary amount of very
| good work on it, and I think there is a very good chance this
| will turn out to be a pretty big deal.
| m0meni wrote:
| As far as I know, one guy made this
| (https://twitter.com/jarredsumner) working 80h-90h on it a week.
| His twitter has some cool performance insights into JS and JS
| engines. It's the biggest Zig codebase too I think.
|
| Congrats on the release :)
| asciiresort wrote:
| Not to be a cynic, but I wonder how much of the motivation to
| create a competing runtime in recent months is in response to the
| eye gauging ( I know, I know. It's all relative ) tens of
| millions in funding Deno just raised.
|
| I don't intend this as a knock on this project. Competition is
| good and, this space, unlike the rest of JavaScript, could do
| with more players. There are some promising numbers and claims
| here. I hope it works out.
|
| I'm genuinely posing this intellectual question of financial
| incentive as an theory for JavaScript fatigue as a whole.
|
| High profile threads on JavaScript fatigue trend on HackerNews
| multiple times a week. The wide range of theories about why web
| developers reinvent the wheel strangely leave out the financial
| incentives.
|
| Everyone claims their framework is the fast, powerful, light,
| batteries included, modern, retro futuristic, extensible,
| opinionated, configurable, zero config, minimal (3kb, gzipped
| minified, of course ). The list goes on. A few days ago, I was
| chatting with someone how all these JavaScript libraries make
| these words have no meaning. To demonstrate, I screenshared a
| bunch of landing pages. At this point, I haven't exhaustively, in
| one sitting, cross referenced these landing pages. 90% of the
| libraries shared the same layout. 3 columns / cards with one of
| those hyperbolic words.
|
| Previously I thought it was pretentious and weird that Svelte
| marketed itself as "cybernetically enhanced web apps". What does
| that even mean? Then again, none of the descriptor words like
| light, dynamic, and ergonomic mean much. At least Svelte was
| memorable.
|
| Occasionally, one of these libraries would describe their
| interface as being ergonomically designed. As if other developers
| designed their interfaces to not be ergonomic. It's like how we'd
| all like to think we're nice, good, decent people. The majority
| of people would not do bad stuff if they perceived it as such.
|
| I do think most JavaScript developers have good intentions. Then
| I've also seen DevRel / Evangelist types who shill their library,
| with disingenuous benchmarks, knowing full well there are better
| solutions, or that they can help make existing solutions better,
| to everyone's benefit. The spoils include consulting, training,
| speaking fees, books, swag, collecting Patreon ( there are some
| controversial projects which come to mind ), resume building,
| GitHub activity social capital ( I've talked to some recruiters
| who were fixated on the idea that you publish code on GitHub,
| specifically Github, because to them, VCS=Git=GitHub, or it
| doesn't exist )
| kristoff_it wrote:
| These are legitimate concerns. Looking at the history of bun
| and its current homepage the main point is that it offers a
| dramatic speed improvement (plus misc. quality of life stuff).
|
| At this point the ball is in your (and everyone else's) court
| to put these claims to the test. It should not be terribly hard
| to see if the speedup is worth your while or not, JS surely
| doesn't lack bloated projects that you can try to build. My own
| personal blog is a bloated Gatsby mess that takes half a minute
| to build.
|
| That's the one true part of the experience that nobody can
| falsify.
| asciiresort wrote:
| > Replace npm run with bun run and save 160ms on every run
|
| Maybe you can't falsify this, but it's a question of risk vs
| reward.
|
| It's currently at 0.1 release. Chances are it has a much
| higher chance of breaking. And when that happens, it would
| likely take occupy way more time to debug than the hundreds
| of ms saved.
|
| Also by being new, it means it has not had a chance to cover
| all the cases yet. That's the devil. It's fast now, but it's
| an apples to orange comparison until Bun is at a stable
| release.
| kristoff_it wrote:
| Yes, making up your own mind with first-hand experience
| requires investing time and effort, that's why people like
| to have other people tell them what to think.
|
| Now we've come full circle.
| andyfleming wrote:
| I think some creators pursue products as ways to fund their
| passions. For example, was deno deploy always the endgame or is
| it just a way to fund working on deno? Aside from motivation,
| how it's implemented and how it affects the community matters.
| asciiresort wrote:
| > it has the edge in mind
|
| This part, very early on in the Bun page, stood out. That's a
| monetizable product, even if the code is open source. That to
| me felt like positioning itself as a potential drop in
| replacement for Deno Deploy / Edge Functions.
|
| There was serverless, and now the next trend is with edge
| computing. It's already happening, but now specifically about
| runtimes on that edge.
| maclockard wrote:
| I'm also pretty cynical of most JS rebuild/reinvention
| projects. I'm very tentatively excited by this one _because_ it
| looks like all it does is incrementally improve. Having
| something that is a drop-in API compat replacement for yarn
| 1/npm and node makes it potentially really easy to get the
| benefits of incremental perf improvements _without_ needing to
| reinvent the wheel like yarn 2 or deno.
| forty wrote:
| 100% this. Being compatible with nodejs API makes it possibly
| useful, unlike some other projects which want to throw away
| the huge npm ecosystem. Why on earth would anyone use JS (or
| even TS) server side if not to benefit from the ecosystem?
| Unlike on the web, there are plenty of better languages to
| use if you don't want npm.
| Tomuus wrote:
| Deno isn't the only company offering a not-Node JS server
| runtime. Cloudflare, Shopify, Fastify, AWS, and probably more
| all have skin in this game.
| asciiresort wrote:
| Deno showed there is space for not-Node, and that developers
| would be receptive to this.
|
| And yes, Deno is just one player in edge, but you can agree
| there is much more money involved with all those other
| players you listed.
|
| It's going to be a battle of eyeballs from those edge
| providers then wouldn't it? Whether that's consulting or
| licensing fees, or just an acquisition / acquihire player.
|
| Maybe you're suggesting these players would build a runtime
| themselves. From my experience, only a fraction of companies,
| rarely, tackle ambitious projects like this. It'd be hard to
| justify to management who need quarterly results. Instead,
| they'll fork an existing technology and make it better,
| because you can show incremental progress but keep your
| baseline. For example, Facebook didn't rewrite their PHP code
| right away. They wrote a faster interpreter.
| firloop wrote:
| Jarred's been working on bun for over a year. I don't get the
| sense that it's a reaction to anything recent at all, Jarred is
| just super passionate about building the best JavaScript
| runtime.
| asciiresort wrote:
| I don't doubt that.
|
| The fact that this project uses Zig suggests to me the
| developer is talented , passionate, and willing to challenge
| the status quo.
|
| When you choose a lesser known language like that to tackle a
| hard problem, chances are you are confident in yourself and
| the language.
|
| The problems I pointed out with the JavaScript ecosystem as a
| whole is that it's low hanging fruit. It's not that there
| aren't financial opportunities elsewhere, outside JavaScript.
| There definitely is. But the perception of financial
| incentives is the low hanging fruit, plus high reward.
|
| In this case, it will boil down to how much Bun innovates
| versus just being a thin wrapped around existing solutions.
| And again, I don't doubt this. Skeptical, in general, but not
| ruling Bun out.
| latchkey wrote:
| Definitely beta, but really awesome to see someone working on
| this stuff.
|
| "I spent most of the day on a memory leak. Probably going to just
| ship with the memory leak since I need to invest more in docs. It
| starts to matter after serving about 1,000,000 HTTP requests from
| web streams"
|
| https://twitter.com/jarredsumner/status/1543236098087825409
| julienb_sea wrote:
| This is super cool and if I was working on more personal projects
| I would be tempted. In an enterprise context, moving to a
| reimplementation of core Node APIs is a terrifying prospect.
| There are infinite possible subtle problems that can appear and
| debugging even within official Node has been a challenge in the
| past. I don't know how this concern can be alleviated without
| seeing proven usage and broader community acceptance.
| andyfleming wrote:
| It is scary, but it already seems more stable and backward-
| compatible than Deno. With some testing and further
| stabilization, I have a feeling bun might be a much more
| feasible and beneficial move.
| kylemh wrote:
| A special dev, releasing a special thing. Chuffed for Jarred. I'm
| hoping Bun hops to me even quicker than it can run.
___________________________________________________________________
(page generated 2022-07-05 23:00 UTC)