[HN Gopher] Speeding up the JavaScript ecosystem - Polyfills gon...
___________________________________________________________________
Speeding up the JavaScript ecosystem - Polyfills gone rogue
Author : jviide
Score : 106 points
Date : 2023-09-21 19:31 UTC (3 hours ago)
(HTM) web link (marvinh.dev)
(TXT) w3m dump (marvinh.dev)
| e_y_ wrote:
| > Polyfills that don't polyfill
|
| This is what's sometimes called a "ponyfill". The idea is to
| avoid messing with global scope (monkeypatching), which could be
| problematic if you have multiple polyfills for the same API or
| polyfills that don't perfectly match the native behavior.
|
| This can be a good thing in some situations, but in general it's
| probably best to leave polyfill decisions to the bundler so you
| can decide which browsers you want to support. Or even produce
| multiple versions, a lightweight one for modern browsers and one
| with tons of polyfills that gets served to ancient ones.
| mhagemeister wrote:
| Author here.
|
| Good point. Agree that the ideal scenario would be that the end
| user (or the tools they use) have the final say in which
| polyfills to load. It's a bit of a bummer that they are shipped
| as part of npm packages without an easy way to get rid of them.
|
| I wonder if our industry will move to publishing the original
| source files to npm in the long run. Only the last piece of the
| chain, the developer using these dependencies, knows what their
| target environments are. So the bundler could then downlevel or
| polyfill the code for the specified targets.
| no_wizard wrote:
| Marvin is doing wonderful work in the JS ecosystem around
| performance. It has been largely focused on tools and node,
| however, he did have an interesting set of things to say about
| how they optimized performance in Preact as well on his website
| that was really interesting too.
|
| One thing I've noticed is the rampant duplication of polyfills
| and babel helpers. To the point that I now have overrides setup
| via pnpm and I re-write many imports of polyfills to point at my
| own shims, which simply re-export existing functionality native
| to the language, most of the time.
|
| For smaller utility packages, I often simply clone the repo and
| copy things over that we need, or copy the src right out of the
| node_modules folder if possible, then I strip away all the
| superfluous imports (and often convert from commonjs to ESM if
| needed)
|
| Saves so much headache, its better for users, smaller builds etc.
| Dextro wrote:
| That sounds awesome. I've dabled with something like that but
| only for lodash (makings sure all different flavours get
| aliased to a single thing) but I never went too far with all
| the other stuff.
|
| You wouldn't happen to have an example of what you're doing
| laying around would you? I'd be genuinely curious to try stuff
| like that out.
| mhagemeister wrote:
| Author here.
|
| Thanks for the kind words! It's feedback like this that
| encourages me to keep writing about it.
|
| I share your experiences regarding babel helpers and haven't
| found a good solution myself. Similar to you, I often patch
| unnecessary stuff out via patch-package, but that approach
| doesn't scale well.
| romellem wrote:
| There is some interesting [drama][1] with this, since this
| article noticeably doesn't mention any PRs they opened to remove
| some of these older polyfills.
|
| The reason those PRs were never opened/merged is the maintainer
| of many of those libraries [has a strong stance on "breaking"
| changes][2] in software:
|
| > I have developed an intense avoidance for breaking changes in
| all of my packages, because I don't want to inflict hundreds of
| millions of dollars of person-hour cost on the entire industry
| unnecessarily.
|
| IMO this argument avoids the _opposite_ claim, that people then
| spend a ton of time (and money) trying to make old tech work with
| newer tech since not everyone maintains to the same standards of
| backwards compatibility.
|
| But regardless, no one is required to stick to a particular way
| of creating open source software, so the one benefit here is that
| you are free to [fork the library][3] (assuming its license
| allows for that) to remove some backwards compatibility that
| isn't relevant to you.
|
| [1]: https://twitter.com/ljharb/status/1704912065486618915
|
| [2]: https://github.com/import-js/eslint-plugin-
| import/pull/2447#...
|
| [3]: https://www.npmjs.com/package/react-outside-click-handler-
| li...
| potsandpans wrote:
| Ljharb is harmful to the ECMAScript community
| silverwind wrote:
| I'm thankful for his work, but I do agree, this polyfill
| madness has to stop.
| gibolt wrote:
| Is there not a config for minimum supported JS version? That
| would appease everyone, while maintaining backwards
| compatibility.
|
| Docs should show the recommended version (modern) and show what
| options are available to go deeper.
|
| Obviously adding those settings for every pollyfill in non-
| trivial, but burdening everyone with every pollyfill ever is
| also suboptimal. If anything, this would make cleanup easier
| going forward since it would all be classified
| petetnt wrote:
| I don't really care to comment about the practice itself, but the
| "Polyfills that don't polyfill" section is missing the point: the
| function is called directly instead of patching the global object
| so that the global object is not polluted by an possibly non-
| standard implementation. Additionally it does use
| Object.defineProperty if available - furthermore it doesn't even
| call itself a polyfill in the first place. If it's needed in 2023
| is a valid point however.
| fiddlerwoaroof wrote:
| I think it would be better to just expect the standardized
| functions to be present and then document that the project
| needs them (e.g. via peer dependencies), allowing users to
| install them themselves as needed.
| shpx wrote:
| That's a lot more work than your library just working in more
| places.
| fiddlerwoaroof wrote:
| I don't think it's all that much more: basically every
| bundler I've used uses browserslist to include polyfills
| for the developer's target audience.
|
| But, also, I think this sort of easy path inflicts a huge
| cost on the ecosystem as a whole: writing to the standards
| and expecting your users to supply a compliant environment
| solves a lot of N*M problems in the dev process.
| ComputerGuru wrote:
| I maintain a few JavaScript libraries that I manually verify
| compatibility against IE6 (and have lints to catch violations). I
| manually polyfill a few necessities and quality-of-life
| improvements up top in the script. Out of curiosity, I removed my
| polyfills and tried swc and babel both, followed by an eslint
| pass, and the results were absolutely atrocious. Everything gets
| polyfilled, even stuff that has been supported in every IE
| version ever. The usage-based detection is completely borked, and
| is completely based off string searching property/function names.
| Using toString() anywhere pulls in the polyfills for Date to
| string, Regex to string, and Object to string. Using regex
| anywhere pulls in a bunch of regex polyfills. It was a nightmare
| and the size of my library increased by orders of magnitude!
|
| (I tried opening an swc issue about optionally using typescript
| ast info (via a plugin, not in swc core) to have more correct
| usage-based polyfill detection, but that was closed as unlikely
| to be acted upon.)
| mhagemeister wrote:
| Author here.
|
| That mirrors my experience too on working in various projects.
| The automatic polyfilling story is such a good thing in theory,
| but reality isn't as rosy and much more polyfills than
| necessary are included.
| ComputerGuru wrote:
| Thanks for writing your article and sharing! One thing I do
| on my other libraries is also to polyfill a few heavier
| things asynchronously instead of making everyone pay the
| price upfront, so for example I detect if a JSON polyfill is
| required before asynchronously loading that polyfill. I think
| the expectation for most things is that the browser will
| support them by default, and it's ok if all/any polyfills are
| loaded separately and asynchronously.
| spankalee wrote:
| Why are you supporting IE6?
| ComputerGuru wrote:
| I replied to a sister comment to yours but it's an old
| library that started off with IE6 support maybe 10 years ago
| and just kinda never lost it.
| H1Supreme wrote:
| > I maintain a few JavaScript libraries that I manually verify
| compatibility against IE6
|
| Genuinely curious why anyone would target IE6 in 2023. Is it a
| personal goal to have massive coverage for your library?
| ComputerGuru wrote:
| It was used in the checkout flow on our website and the
| marginal cost of supporting IE6 wasn't worth a lost sale.
| That was many years ago, and since then I don't think we have
| anyone on less than IE8 due to TLS version issues with
| CloudFront, but the code already supported IE6 and there
| aren't many polyfills extra compared to IE10, so :shrug:
| smallnamespace wrote:
| The reason libraries call polyfills directly is because it's
| impolite for a library to change global scope underneath you.
|
| Usually it's the top-level application's author who chooses and
| configures polyfills.
|
| Now one may reasonably ask, why doesn't the library just call
| Object.defineProperties directly, and tell the user to install
| the appropriate polyfill?
|
| I'm going to guess that a library that Just Works after an npm
| install will see much better adoption than one that requires each
| user to configure their babel/swc/etc. correctly, especially
| since the library can be a dependency of another library.
|
| There's currently no standardized mechanism in the npm ecosystem
| to do the equivalent of "Install this library, and also configure
| your environment to pull in all required polyfills" so that the
| required functionality is available in global scope. One reason
| is because the transpilers that automatically polyfill into
| global scope are third-party tools.
|
| Maybe a standard mechanism like this _should_ exist, but it doesn
| 't today, hence the quite reasonable choice of library authors to
| directly use polyfills because doing so:
|
| 1. Avoids pollute the global namespace by avoiding applying a
| polyfill globally
|
| 2. Works as a dependency without additional configuration by the
| user
|
| 3. Preserve backwards compatibility
|
| A somewhat cheap fix to at least reduce duplication of polyfills
| would be for libraries that need polyfills to accept a wide
| version range. That would give the package manager room to pick a
| version that's compatible across call sites.
| tedunangst wrote:
| But why are we polyfilling a function that exists in every
| version of node? When did this code _not_ just work after
| installation that it required a polyfill?
| [deleted]
| azemetre wrote:
| This series is great!
|
| You should seriously think about consolidating them into a book.
| Something I notice other engineers struggle with is how to
| properly assess performance, read heap snapshots, or even
| understand how to read a flamegraph for stack tracing tools. It
| would be nice to point, or buy, them a resource showing this.
|
| I'd definitely buy a copy.
| bjnewman85 wrote:
| Second that, I would definitely buy a book based on this
| series.
| mhagemeister wrote:
| Thanks for the kind feedback! It's definitely something in
| the back of my mind. I feel like I need to collect a little
| more content to fill a whole book, but I'm enticed by the
| thought of writing one nonetheless.
| leipert wrote:
| Part 3 send me down a debugging route of eslint performance
| in the GitLab project and we were able to move from 25 min
| of listing time in CI, down to 5. So, thanks for the
| inspiration!
| jluxenberg wrote:
| For what it's worth; `eslint-plugin-react` has been around for a
| long time and seems to support running in very old versions of
| Node.JS (back to v4[1] apparently! tho I can't find anything
| documenting that for sure.)
|
| I was surprised to learn that Object.values is only supported in
| Node >v7, Object.fronEntries was added in v12, etc. So for this
| project maybe the polyfills are needed.
|
| [1] https://github.com/jsx-eslint/eslint-plugin-react/pull/1038
| mhagemeister wrote:
| Yeah, engines are a moving target. I'm all for backwards
| compatibility, but I'm worried about promoting old node
| versions with known unpatched security issues. Given that
| eslint itself only supports node >= 12.22.0 it seems like it's
| time to get rid of the polyfills.
|
| I wish we as in the industry would find a better solution to
| adapt to this. It's a bit unfortunate that the polyfills as
| part of the library code itself, which makes it difficult to
| get rid of them once they're not needed anymore.
| gigel82 wrote:
| Genuinely curious, are there people out there using newer
| versions of this package with old / unsupported versions of
| Node (in production)?
| silverwind wrote:
| Not really. Adoption of new Node versions is quite quick,
| given their short support periods.
|
| https://nodejs.org/metrics/summaries/version.png
| cxr wrote:
| You'll never get NPM apologists to acknowledge this. One of their
| only skills is making non-specific appeals to the necessity of it
| all (as essential infrastructure) and vague arguments that boil
| down to "you need to trust the wisdom of the crowds" (and e.g.
| the fact that it exists and everyone else is using it means that
| anyone who disputes its value just doesn't understand it--bonus
| points for them if they manage to work in a slight that's
| designed to paint you, implicitly or explicitly, as a junior),
| despite not being able to attest to any firsthand knowledge of
| the real _why_ of anything they 're defending.
|
| > The new dependencies were all polyfills for JavaScript
| functions that have long been supported everywhere. The
| Object.defineProperties method for example was shipped as part of
| the very first public Node 0.10.0 release dating back to 2013.
| Heck, even Internet Explorer 9 supported that. And yet there were
| numerous packages in that dependend on a polyfill for it.
| CharlesW wrote:
| > _You 'll never get NPM apologists to acknowledge this._
|
| How should NPM prevent archaic dependencies, or the "even more
| bizarre" (author's words) problem of developers calling
| polyfills directly instead of the function that the polyfill
| fills?
| whstl wrote:
| The parent is not criticising NPM the tool/registry, but
| rather the ecosystem and culture.
| cxr wrote:
| I'm happy to criticize NPM the tool. The whole thing is
| designed as a second, crummier version control system that
| lives in disharmony with and on top of your base-level
| version control system (so it can subvert it). It's a
| terrible design.
|
| There's basically at most one reasonable use for npm: as a
| glorified download manager, i.e. to quickly fetch a module
| by name (right before you check it in to version control
| with Git). This differs wildly, of course, from how it's
| actually used, which is as a drug that sweeps mountains of
| unaudited code under the rug so people can trick themselves
| and others into thinking that none of it's really there on
| the basis that it's not visible when anyone first clones
| the repo.
|
| To answer the other commenter's question, "How should NPM
| prevent archaic dependencies": it shouldn't; it's okay for
| programmers to be responsible for their work.
| spankalee wrote:
| How is npm a version control system?
|
| npm is a fairly standard package manager, much like many
| others, pretty good even.
|
| I don't know of anyone who says that package managers and
| source control solve the same problems. They both happen
| to use the word and concept of "version" but to mean
| _different_ things. Yes, some projects vendor in their
| dependencies into their source control system, but they
| must either manually verify package version compatibility
| or use a package manager like npm to help them do it. And
| vendoring doesn 't work for actual packages published to
| the package repo. If they vendored dependencies then
| every dependency would be duplicated always, defeating
| the very purpose of a package manager!
| cogman10 wrote:
| Back in 2011ish or so when npm was just getting started and
| Ruby on rails was all the rage. "Do not repeat yourself" was
| seen as a gold standard for programming. The end result has
| been a mess, particularly for npm. There were FAR too many
| articles talking about how "there's no such thing as too small
| a dependency" and talks given about how much a virtue it was to
| create "is-odd" or "is-even" "look you saved 3 lines of code
| that you don't have to test!"
|
| Unfortunately, that compounded with browser of the era
| (Internet explorer...) having basically 0 support for modern
| javascript led to a proliferation of dependencies, polyfills,
| etc that are nearly impossible to remove from the ecosystem.
|
| I've not seen a lot of node apologists that are fine with the
| current ecoystem. The problem is righting the ship is going to
| be terribly hard. Either existing frameworks/libraries need to
| go through the effort of saying "Ok, do I really need is-even,
| let's remove it" or we need new frameworks/libraries to abandon
| tools and the ecosystem in favor of fatter and fewer
| dependencies.
|
| I think the issue all stems from the fact that before 2010ish,
| there was one library and one framework, jquery (Ok, there were
| others... but were there really?) and that added a good 1mb to
| any webpage. The notion was we do more with less if we had a
| bunch of smaller deps that didn't need to be brought in.
| no_wizard wrote:
| I wonder if there would be any interest from folks for an
| alternative npm registry that would automatically cleanup
| dependency chains like this (among other things, possibly),
| remove polyfills etc.
|
| I always thought about something like this, with on the fly
| manipulation of packages via SWC would be pretty fast I think
| Klonoar wrote:
| jQuery is the one that's outlasted them all, but yes - there
| were absolutely other frameworks in use. It also discounts
| the detour into the Backbone era before Angular and React
| took off.
| arbol wrote:
| MooTools springs to mind
| whstl wrote:
| Most people fine with the problems of the ecosystem have
| financial incentives.
|
| Stating that you maintain 800 NPM libraries brings more clout
| and money than maintaining a foundational one.
|
| Even with foundational packages things tend to go wrong. Why
| add features to an existing package if I can write several
| plugins? Or even worse in some cases: why use the existing
| configuration file if I can instead just ask users to install
| dozens of dummy packages that only exist to trigger a feature
| in my Core package?
| josephg wrote:
| > "Do not repeat yourself" was seen as a gold standard for
| programming.
|
| I remember this era. I've been using nodejs before npm
| existed and so many silly things have happened in that time.
|
| I think the core problem the JS ecosystem has always had is
| that most JS developers are relatively inexperienced. (JS is
| very beginner friendly and this is the price we pay). I still
| vividly remember being at nodecamp in 2012 or something
| listening to someone tell me how great it would be if the
| entire OS was written in javascript. It didn't matter how
| much I poked and prodded him, he couldn't hear that it might
| not be an amazing idea. I think he thought it would be easier
| to reimplement an OS kernel in JS than it would be to just
| learn C. And there were lots of people around with a sparkle
| in their eye and those sort of wacky ideas - good or bad. It
| was fun and in hindsight a very silly time.
|
| So yeah, of course some idiot in JS made is-even. And is-odd
| (which depends on is-even). I see all of this as the
| characteristic mistake of youth - that we go looking for
| overly simple rules about what is good and bad (JS good! C
| bad!) and then we make a mess. When we're young we lack
| discernment about subtle questions. When is it better to pull
| in a library vs writing it yourself inline? When is JS a good
| or a bad idea? When should you add comments, or tests? And
| when do you leave them out?
|
| Most of the best engineers I know made these sort of stupid
| philosophical mistakes when they were young. I certainly did.
| The JS ecosystem just suffers disproportionately from this
| kind of thing because so many packages in npm are written by
| relatively new developers.
|
| I think that's a good thing for our industry as a whole. But
| I also get it when Bryan Cantrill describes JS as the failed
| state of programming languages.
| TeMPOraL wrote:
| > _The JS ecosystem just suffers disproportionately from
| this kind of thing because so many packages in npm are
| written by relatively new developers._
|
| I think it also suffers because it grew in the age of
| Internet and Open Source, which made the problem
| compounding. Programmers write a lot of stupid code when
| learning, it's part of the process - but it used to be that
| the stupidity was constrained to your machine and maybe a
| few poor souls who ended up reading or using your code. In
| JS ecosystem, all that stupidity gets published, and ends
| up worming its way, through dependency chains, into
| everything.
| beepbooptheory wrote:
| I thought is-even/is-odd were satirical? Granted, satire
| echoing these same points... but, you know, just for-the-
| record.
| phire wrote:
| It's hard to tell.
|
| The author of both packages has since moved the github
| repos to an organisation named "i-voted-for-trump" and
| labelled the packages with the "troll-bait" tag.
|
| But the author also claims those packages where created
| when they were learning to program, and they seem to have
| stuck with the idea of small libraries (just not quite
| that small). The changes seem to be due to grief the
| programming community has given him about those packages,
| as some of the worst examples of the pattern.
|
| I suspect the author was being slightly satirical by
| choosing to do the simplest possible NPM library. But at
| the same time they probably believed it was a useful
| package, allowing new programmers (like himself) to
| calculate the evenness/oddness of numbers without needing
| to know or google the mod-2 trick. And if you are going
| to learn how to create npm packages, might as well start
| small.
|
| If they were aiming to be slightly satirical, they were
| not expecting that level of outrage.
|
| It's worth pointing out that the libraries do slightly
| more than just mod-2. It checks the argument passed in
| was actually an integer, and throws descriptive error
| messages if the argument is not a number or not an
| integer.
| pphysch wrote:
| Those may be satirical, but `leftpad` is real.
|
| https://qz.com/646467/how-one-programmer-broke-the-
| internet-...
| phire wrote:
| _> I still vividly remember being at nodecamp in 2012 or
| something listening to someone tell me how great it would
| be if the entire OS was written in javascript._
|
| That reminds me of Gary Bernhardt's "The Birth & Death of
| JavaScript" talk[0], which one of the best comedy
| programming talks. Despite the comedy, it's also a half-
| decent idea.
|
| [0] https://www.destroyallsoftware.com/talks/the-birth-and-
| death...
___________________________________________________________________
(page generated 2023-09-21 23:00 UTC)