RFC: Rethink polyfilling story

[nicolo-ribaudo]

In the last three years and a half, @babel/preset-env has shown its full potential in reducing bundle sizes not only by not transpiling supported syntax features, but also by not including unnecessary core-js polyfills.
Currently Babel has three different ways to inject core-js polyfills in the source code:

• By using @babel/preset-env's useBuiltIns: "entry" option, it is possible to inject polyfills for every ECMAScript functionality not natively supported by the target browsers;
• By using useBuiltIns: "usage", Babel will only inject polyfills for unsupported ECMAScript features but only if they are actually used in the input souce code;
• By using @babel/plugin-transform-runtime, Babel will inject ponyfills (which are "pure" and don't pollute the global scope) for every used ECMAScript feature supported by core-js. This is usually used by library authors.

ℹ️ INFO: Before continuing, I highly recommend reading "Annex B: The current relationship between Babel and core-js" to get a deeper understanding of the current situation!

Our position in the JavaScript ecosystem allows us to push these optimizations even further. @babel/plugin-transform-runtime has big advantages for some users over useBuiltIns, but it doesn't consider target environments: it's 2020 and probably very few people need to load an Array.prototype.forEach polyfill.
Additionally, why should we limit this ability to automatically inject only the necessary polyfill to core-js? There are also DOM polyfills, Intl polyfills, and polyfills for a myriad of other web platform APIs. Additionally, not everyone wants to use core-js: there are many other valid ECMAScript polyfills, which have different tradeoffs (e.g. source size vs spec compliancy) and may work better for some users.

What if the logic to inject them was not related to the actual data about the available or required polyfills, so that they can be used and developed independently?

Concepts

• Polyfill provider is a special kind of Babel plugin that injects is used to specify which JavaScript expressions need to be polyfilled, where to load that polyfill from and how to apply it. Multiple polyfill providers can be used at the same time, so that users can load, for example, both an ECMAScript polyfill provider and a DOM-related one.

  Polyfill providers can expose three different methods of injecting the polyfills:

  • entry-global, which reflects the current useBuiltIns: "entry" option of @babel/preset-env;
  • usage-global, which reflects the current useBuiltIns: "usage" option of @babel/preset-env;
  • usage-pure, which reflects the current polyfilling behavior of @babel/plugin-transform-runtime.

  Every interested project should have their own polyfill provider: for example, babel-plugin-polyfill-corejs3 or @ungap/babel-plugin-polyfill.

New user experience

Suppose a user is testing some ECMAScript proposals, they are localizing their application using Intl and they are using fetch. To avoid loading too many bytes of polyfills, they are ok with supporting only commonly used browsers, and they don't want to load unused polyfills.

How would their new config look like?

{
  "presets": [
    ["@babel/env", { "targets": [">1%"] }]
  ],
  "plugins": [
    "@babel/proposal-class-properties",
    ["polyfill-corejs3", {
      "targets": [">1%"],
      "method": "usage-global",
      "proposals": true
    }]
  ]
}

New developer experience

In order to provide consistent APIs and functionalities to our users, we will provide utilities to:

1. centralize the responsibility of handling the possible configuration options to a single shared package
2. abstract the AST structure from the polyfill information, so that new usage detection features will be added to all the different providers.
   We can provide those APIs in a new @babel/helper-define-polyfill-provider package.

These new APIs will look like this:

import definePolyfillProvider from "@babel/helper-define-polyfill-provider";

export default definePolyfillProvider((api, options) => {
  return {
    name: "object-entries-polyfill-provider",

    polyfills: {
      "Object/entries": { chrome: "54", firefox: "47" },
    },

    entryGlobal(meta, utils, path) {
      if (name !== "object-entries-polyfill") return false;
      if (api.shouldInjectPolyfill("Object/entries")) {
        utils.injectGlobalImport("object-entries-polyfill/global");
      }
      path.remove();
    },
    
    usageGlobal(meta, utils) {
      if (
        meta.kind === "property" &&
        meta.placement === "static" &&
        meta.object === "Object" &&
        meta.property === "entries" &&
        api.shouldInjectPolyfill("Object/entries")
      ) {
        utils.injectGlobalImport("object-entries-polyfill/global");
      }
    },
    
    usagePure(name, targets, path) {
      if (
        meta.kind === "property" &&
        meta.placement === "static" &&
        meta.object === "Object" &&
        meta.property === "entries" &&
        api.shouldInjectPolyfill("Object/entries")
      ) {
        path.replaceWith(
          utils.injectDefaultImport("object-entries-polyfill/pure")
        );
      }
    }
  }
});

The createPolyfillProvider function will take a polyfll plugin factory, and wrap it to create a proper Babel plugin. The factory function takes the same arguments as any other plugin: an instance of an API object and the polyfill options.
It's return value is different from the object returned by normal plugins: it will have an optional method for each of the possible polyfill implementations. We won't disallow other keys in that object, so that we will be able to easily introduce new kind of polyfills, like "inline".

Every polyfilling method will take three parameters:

• A meta object describing the built-in to be polyfilled:
  • Promise -> { kind: "global", name: "Promise" }
  • Promise.try -> { kind: "property", object: "Promise", key: "try", placement: "static" }
  • [].includes -> { kind: "property", object: "Array", key: "includes", placement: "prototype" }
  • foo().includes -> { kind: "property", object: null, key: "includes", placement: null }
• An utils object, exposing a few methods to inject imports in the current program.
• The NodePath of the expression which triggered the polyfill provider call. It could be an ImportDeclaration (for entry-global), an identifier (or computed expression) representing the method name, or a BinaryExpression for "foo" in Bar checks.

Polyfill providers will be able to specify custom visitors (like normal plugins): for exapmle, core-js needs to inject some polyfills for yield* expressions.

How does this affect the current plugins?

Implementing this RFC won't require changing any of the existing plugins.
We can start working on it as an experiment (like it was done for @babel/preset-env), and wait to see how the community reacts to it.

If it will then success, we should integrate it in the main Babel project. It should be possible to do this without breaking changes:

• Remove the custom useBuiltIns implementation from @babel/preset-env, and delegate to this plugin:
  • If useBuiltIns is enabled, @babel/preset-env will enable @babel/inject-polyfills
  • Depending on the value of the corejs option, it will inject the correct polyfill provider plugin.
• Deprecate the regenerator and corejs options from @babel/plugin-transform-runtime: both should be implemented in their own polyfill providers.

Open questions

• Should the polyfill injector be part of @babel/preset-env? No.
  • Pro: it's easier to share the targets. On the other hand, it would be more complex but feasible even if it was a separate plugin.
  • Con: it wouldn't be possible to inject polyfills without using @babel/preset-env. Currently @babel/plugin-transform-runtime has this capability.
  • If it will be only available in the preset, @babel/helper-polyfill-provider should be a separate package. Otherwise, we can just export the needed hepers from the plugin package.
  • It could be a separate plugin, but @babel/preset-env could include it.
• Who should maintain polyfill providers?
  • Probably not us, but we shouldn't completely ignore them. It's more important to know the details of the polyfill rather than Babel's internals.
  • Currently the core-js polyfilling logic in @babel/preset-env and @babel/plugin-transform-runtime has been mostly maintained by Denis (@zloirock).
  • It could be a way of attracting new contributors; both for us and for the polyfills maintainers.
  • Both the regenerator and core-js providers should probably be in the Babel org (or at least supported by us), since we have been officially supporting them so far.
• Are Babel helpers similar to a polyfill, or we should not reuse the same logic for both?

Annex A: The ideal evolution after this proposal

This proposal defines polyfill providers without requiring any change to the current Babel architecture. This is an important point, because will allow us to experiment more freely.

However, I think that to further improve the user experience we should:

1. Lift the targets option to the top-level configuration. By doing so, it can be shared by different plugins, presets or polyfill providers.
2. Add a new polyfills option, similar to the existing presets and plugins.

By doing so, the above configuration would become like this:

{
  "targets": [">1%"],

  "presets": ["@babel/env"],
  "plugins": ["@babel/proposal-class-properties"],
  "polyfills": [
    ["corejs3", {
      "method": "usage-global",
      "proposals": true
    }]
  ]
}

# Annex B: The current relationship between Babel and core-js

Babel is a compiler, core-js is a polyfill.

A compiler is used to make modern syntax work in old browsers; a polyfill is used to make modern native functions work in old browsers. You usually want to use both, so that you can write modern code and run it in old browsers without problems.

However, compilers and polyfills are two independent units. There are a lot of compiler you can choose (Babel, TypeScript, Traceur, swc, ...), and a lot of polyfills you can choose (core-js, es-shims, polyfill.io, ...).

You can choose them independently, but for historical reasons (and because core-js is a really good polyfill!) so far Babel has made it easier to use core-js.

What does the Babel compiler do with core-js?

Babel internally does not depend on core-js. What it does is providing a simple way of automatically generate imports to core-js in your code.

Babel provides a plugin to generate imports to core-js in your code. It's then your code that depends on core-js.

// input (your code):
Symbol();

// output (your compiled code):
import "core-js/modules/es.symbol";
Symbol();

In order to generate those imports, we don't need to depend on core-js: we handle your code as if it was a simple string, similarly to how this function does:

function addCoreJSImport(input) {
  return `import "core-js/modules/es.symbol";\n` + input;
}

(well, it's not that simple! 😛)

Be careful though: even if Babel doesn't depend on core-js, your code will do!

Is there any other way in which Babel directly depends on core-js?

Kind of. While the Babel compiler itself doesn't depend on core-js, we provide a few runtime packages that might be used at runtime by your application that depend on it.

• @babel/polyfill is a "proxy package": all what it does is importing regenerator-runtime (a runtime helper used for generators) and core-js 2. This package has been deprecated at least since the release of core-js 3 in favor of the direct inclusion of those two other packages.
  One of the reasons we deprecated it is that many users didn't understand that @babel/polyfill just imported core-js code, effectively not giving to the project the recognition it deserved.
• (NOTE: @babel/runtime contains all the Babel runtime helpers.)
• @babel/runtime-corejs2 is @babel/runtime + "proxy files" to core-js. Imports to this package are injected by @babel/plugin-transform-runtime, similarly to how @babel/preset-env injects imports to core-js.
• @babel/runtime-corejs3 is the same, but depending on core-js-pure 3 (which is mostly core-js but without attaching polyfills to the global scope).

With the polyfill providers proposed in this RFC, we will just generate imports to core-js-pure when using @babel/plugin-transform-runtime rather than using the @babel/runtime-corejs3 "proxy".

Related issues

• Element instanceof EventTarget #9289
• https://github.com/babel/babel/issues/9666
• Setting broserslist to "ie 9" lacks "MutationObserver" object #9842
• @babel/preset-env doesn't include fetch polyfill #9160
• IE: Object doesn't support property or method 'remove' #8406
• response.getHeaders babel polyfill #7052
• Fetch API feature request #2113
• Allow whitelist and blacklist options for runtime transformer definitions (T6904) #3934
• plugin-transform-runtime doesn't honor preset-env targets when corejs option is set #9363
• Move transform-runtime functionality into this preset #6629
• ...

[schmod]

Is there a short-term possibility of just creating a fork of transform-runtime that's exclusively used within preset-env?

In that scenario, we wouldn't need to rethink Babel's entire plugin architecture, but could offer significant bundle-size reductions to most library authors with a (considerably) lower level of effort.

Then, we could expose an API that looks something like:

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"], "corejs": 3, "transformRuntime": { 
      "helpers": true, "regenerator": false 
    } }]
  ]
}

---

I think this proposal has considerable long-term merits, but it also seems like we can achieve 95% of what people need in a much shorter timeframe.

[rjgotten]

Had pretty much the same idea as @schmod but he beat me to it.

Never quite understood why several parts of transform-runtime were not brought over to preset-env before. E.g. the deduplication of helpers like class construction into separate modules; that seems like a very beneficial general purpose optimization you would want people to take. And thus seems quite a sane default. There will be people that need to compile one self-contained module with the helpers inlined. And they can turn that off.

Now, with env's browserslist becoming a bigger thing with conditional polyfilling, it just seems like it makes more sense to integrate the transform-runtime plugin wholesale.

"useBuiltIns" : "pure" maybe?

And then when that's all fine and working -- and tested in practice, work on adding the layer of indirection and extensibility with polyfill providers.

[nicolo-ribaudo]

"useBuiltIns": "pure" would definetly work in the short term. Given our current release cadence (a minor version every about 2 months), I fear that the whole @babel/plugin-inject-polyfills would come in the minor release right after useBuiltIns: "pure"*, and I'd prefer to avoid introducing an option which will be soon be legacy.

@schmod If @babel/transform-runtime would magically get the targets from @babel/preset-env, how would that config be better than how it would be currently done? Personally, I feel that helpers and regenerator don't really fit in @babel/preset-env since they are about an optimization ortogonal to the one introduced by the preset.

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"] }],
  ],
  "plugins": [
    ["@babel/transform-runtime", {
      "corejs": 3,
      "helpers": true,
      "regenerator": false,
    }]
  ]
}

[devinus]

I'd love to see a simple useBuiltIns: 'pure' option while still retaining the functionality of @babel/transform-runtime for people using it today. I was honestly surprised when I learned that it wasn't a thing already.

It would be relatively easy to implement and provide an immediate upgrade path without worrying about deprecating anything people are already using.

That doesn't mean keeping it around forever, but it's a lot easier to steer people already using pure to the new hotness after it's ready rather than quietly introducing a whole new thing.

[devinus]

Personally, I feel that helpers and regenerator don't really fit in @babel/preset-env since they are about an optimization ortogonal to the one introduced by the preset.

Really, helpers is the only one that doesn't make sense in preset-env since there are environments that legitimately support generators today.

[rjgotten]

That doesn't mean keeping it around forever, but it's a lot easier to steer people already using pure to the new hotness after it's ready rather than quietly introducing a whole new thing.

Well put. For something as big as an entire injection architecture around polyfills, easing people in with an easy-to-adopt upgrade path first has some benefits that should be considered, imho.

Basically; it prevents people from running into a brick wall; or from slamming on the brakes full-stop because they think they're going to.

Speaking from my own experience, I think the 'full' solution would be kind of a hard sell for some of the development teams I'm involved with, whereas the outwardly deceptively simple API of just another config switch value can do a lot to mitigate concern.

[nicolo-ribaudo]

Thank you all for your feedback.

---

@devinus

Really, helpers is the only one that doesn't make sense in preset-env since there are environments that legitimately support generators today.

You are right, regenerator is just another polyfill. Actually, when I started working on this RFC babel-polyfill-provider-regenerator is the first one I implemented to test it 😅

---

@rjgotten
If useBuiltIns: "pure" and this RFC would land at the same time, do you think that useBuiltIns: "pure" would still be that valuable?
To ease the migration, adding it to the preset once this RFC is implemented would probably be trivial, but it wouldn't make it possible to add other polyfills (e.g. node APIs or web APIs) without using two parallel polyfill injectors.

[rjgotten]

@nicolo-ribaudo

If they would land simultaneously, I still think there's value in there being an easy adoption mode that gets users going and can ween them off of babel-runtime based polyfills. For the full package -- i.e. not just ES polyfills but also Node; Web platform; etc. -- you'd still need to go 'all in' afterwards, but that can still be a 'stage 2' of a migration process. (And one that'll be considerably easier to sell, also to a non-technical management tier, imho.)

[Conaclos]

Any news?

[askbeka]

Great! Would really love to have this feature.
But confsued about pure part.

Not sure how ponyfills are implemented right now.

Consider scenario when ponyfill is implemented using some of the language builtins, but if thise builtins are not avaialbe in target they are replaced by their ponyfills.

Is it the case? Or ponyfills always implemented based on other ponyfills without making assumptions on evironement?

[nicolo-ribaudo]

That's up to the ponyfill that would be used.

For example, an [].includes ponyfill could be implemented like this:

import _some from "./array.prototype.some";

export default function includes(array, item) {
  return _some(array, val => val === item);
}

Or it could assume that .some is available, and ask you to transform the polyfill source with another polyfill provider which gives you ES5 features:

export default function includes(array, item) {
  return array.some(val => val === item);
}

[askbeka]

Thanks for the example snippet.
I was hoping that it implemented like in 2nd example and that all ponyfills are passed through babel with defined target. Otherwise some feature ends up being fully poly filled even though 90% of it is available on the target.

Anyway. I'd love to contribute. Do you see any blockers?

[nicolo-ribaudo]

Anyway. I'd love to contribute. Do you see any blockers?

I'm currently working on this privately, because it's a really big feature and I need the ability to commit/refactor whatever I want until it's ready to be shared publicly. I'll let you know when it's ready to be made public!

[ssipos90]

Awesome, now we can have multiple packages for the same stuff in our bundles.

Sarcasm intended :)

[zloirock]

A long time ago I wrote that we should discuss it. I wrote that it's over-complicated. Ok, I'll write what I mean here.

Yes, we should simplify adding new plugins for polyfilling and we should extract core-js plugins from preset-env. This work I started on #7646.

But we don't need new conceptions like "polyfill provider" and "polyfill injector". Why add new entities? Moreover - entities based on preset-env.

A solution is very simple - we can allow defining targets at the top level of .babelrc / babel.config.js and pass (parsed or not) targets as an option to each plugin - it could be useful not only for polyfills - also for some userland plugins could optimize output for required targets. Theoretically, for some cases, it could even make unnecessary preset-env in the current form.

We will have

{
  "targets": [">1%"],
  "presets": [
    ["@babel/preset-env"]
  ],
  "plugins": [
    "intl-polyfill",
    "fetch-polyfill",
    ["corejs3", { "proposals": true }]
  ]
}

Instead of

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"] }]
  ],
  "plugins": [
    ["@babel/inject-polyfills", {
      "method": "usage-global",
      "providers": [
        "intl-polyfill",
        "fetch-polyfill",
        ["corejs3", { "proposals": true }] 
      ]
    }],
  ]
}

It's plain and, for me, - more simple and obvious.

We should not limit the plugin developer to only proposed modes: entry-global, usage-global, and usage-pure - someone could add something own.

"New developer experience"? The development of any serious plugin for polyfilling will go much further than suggested here. For the development of simple plugins, could be added pseudo-visitors or just helpers like proposed here. Serious polyfilling plugins should be written as usual plugins, the only moment that's required - as I wrote, passing targets as an option.

[zloirock]

@nicolo-ribaudo any feedback?

[askbeka]

@zloirock I can relate with your proposal, but does it require too big of a change from current implementation?
Looking at time this one is taking, I am afraid that this issue will never be resolved.

@nicolo-ribaudo Some time has passed and no visible progress on this issue.
Did you meet any dead ends? Some update will appreciated. Even better if you could publish your work.
I believe others would love to help.

[zloirock]

@askbeka top-level targets option? Nope, it's the simplest solution. More over - targets parser already extracted from preset-env in #10899.

[nicolo-ribaudo]

Yeah sorry for postponing this, I will try to work again on this feature in January/February.

Two quick comments:

1. I love the idea of a top-level targets option. It would not only be useful for polyfills and preset-env, but also for any other preset which wants to use the most modern features possible (like a minifier which could transform a normal function to an arrow).
   Regardless of the outcome, the first versions of the polyfilling plugins will still have a targets option: I want to release this feature as something that doesn't need any change in the existing Babel packages so that we can refine it with real-world feedback. Then, I would probably be in favor of the top-level targets.
2. My initial proposal used different plugins rather than a single @babel/inject-polyfills. You can click on "edited" in my original post and check the first version. I don't remember exactly why we changed the proposal, I have to check our past meeting notes to understand if we had valid concerns or was just an aesthetic preference.
   In the future, if people start putting many polyfills in their configs, we might even think about introducing a polyfills array similar to plugins and presets.

[zloirock]

the first versions of the polyfilling plugins will still have a targets option

My proposal is adding support of top-level targets, not removing this option for each plugin. I think that plugins / presets should have this option, but if it's not passed - they should inherit it from the top level.

[reactuser35]

"useBuiltIns" : "pure" option looks real solid.

I don't want to pollute the global scope and also I don't want to include so much and unnecessary polyfills while using babel-plugin-transform-runtime.

I think this future is the one of the needed one on next releases.Any update on that?

[nicolo-ribaudo]

I made the babel-polyfills repository public. Also, I have created an RFC to move targets to the top-level options (babel/rfcs#2).

I have not published the babel-polyfills packages on npm yet, I still have to polish some things and to test that it passes all the polyfills-related tests of @babel/preset-env.

If anyone has feedback, please open issues in that repository!

[reactuser35]

I made the babel-polyfills repository public. Also, I have created an RFC to move targets to the top-level options (babel/rfcs#2).

I have not published the babel-polyfills packages on npm yet, I still have to polish some things and to test that it passes all the polyfills-related tests of @babel/preset-env.

If anyone has feedback, please open issues in that repository!

Thanks for the answer.
That repository looks exiting and also be sure that lots of developer in the community are waiting for that "pure" related solutions. I hope we will able to use it soon.

My suggestion is that after you finish your job and open it to the usage of community please write detailed docs and maybe some medium.com articles about the past and future usage of that target/babel/transform runtime relation.

Because I see that there might be some misunderstandings about the usage of polyfill, ponyfills, core.js,transform runtime plugin and babel. Some developers really have some confusions about the true usage of babel. It will be good to have detailed and official docs which tells the past usages and future usages of that things.

[nicolo-ribaudo]

Sure, proper docs and a blog post is one of the things I need to finish before the first release!

[nicolo-ribaudo]

I'm closing this issue since we have released babel-polyfills!
We have decided not to implement @babel/runtime-like helpers support in @babel/preset-env since it wouldn't have reduced much the time needed to release it.

If there is anythig more to discuss, please open an issue there 😉