Allow plugins to indicate dependencies on random files

[loganfsmyth]

Feature Request

In contexts where Babel is running alongside a filesystem watcher, there are cases where a plugin may load an arbitrary file. Take a plugin like https://www.npmjs.com/package/babel-plugin-inline-import for example. Or some usecases of babel-plugin-macros.

It would be ideal if Babel had a way for plugins to indicate that these dependencies exist, so that if callers of Babel wanted to take that into account, they could. For example, Webpack has .addDependency, so that changes to that file will cause the loader to re-execute when the dependency is changed. We could easily use this API in babel-loader to handle this. Implementing a similar API in @babel/cli's watcher wouldn't be too bad either.

I don't know how other Babel wrappers would fair, but if someone wanted to help investigate that, it would certainly help. Off the top off my head we'd want to check, at the very least, if the following have ways to register additional files to watch:

• Meteor
• Jest
• Browserify
• Gulp

Edit

We may also want to include in this work logic to help us take dependency file content into account for the purposes of cache invalidation. For example, babel-loader's cacheDirectory option will not invalidate caches, if say a .browserlistrc file changes preset-env's plugin list: babel/babel-loader#690

[loganfsmyth]

cc @kentcdodds How often do macros end up depending on arbitrary content where automatic re-evaluation would be helpful? I feel like I remember an issue about this, but I can't seem to find it, so it might have only been tangentially related.

[kentcdodds]

This is the issue: kentcdodds/babel-plugin-preval#19

Yes, this would be extremely useful for many macros and babel plugins. I definitely would use this feature.

[loganfsmyth]

Ahh it was preval, that's why I couldn't find it when I searched. Thanks for digging it up!

[Andarist]

I would even say that this is quite a common thing for macros - all of those do that: preval.macro, codegen.macro, import-all.macro, svgr.macro, raw.macro, lqip.macro, data-uri.macro. So it's 7 out of 20 listed in the docs 😉

[kentcdodds]

I have several one-off macros that are not open source where this would be helpful as well. I definitely would love this feature 👌

[bebbi]

graphql.macro

[devthejo]

I published a lib for with this feature, it works by adding annotation to your js file:

/*!@compileDependencies([
  './my/js-dependencie1.js',
  '../dependencie2.js',
  '../a-directory/',
])*/

when a specified dependency change, the file containing annotation is automatically recompiled,
directories are watched recursively and must end with a trailing slash

babel-watch-extra is coupled with nodemon

here is the link:
https://github.com/di-ninja/babel-watch-extra

usage:
babel-watch-extra --src src --dist dist entry-point1.js entry-point2.js

It has also others advantages against official @babel/cli watcher:

• take into account deletion and others events, keeping everything synced
• fix a bug of watching that was caused by awaitWriteFinish option of chokidar used in official @babel/cli watcher (with my favorite editor, geany, when I save a file, it write a temp file first, and then move it to dest, and sometimes the recompile watching was just lost and never fired again until I restart babel watch)

see also: https://stackoverflow.com/a/53510227/5338073

[kentcdodds]

Thanks for building/posting @TakioN! That's a great workaround for people who are using the babel CLI. Hopefully we can get this issue resolved for those folks who are running babel via webpack/rollup/jest/etc.

[devthejo]

@kentcdodds I'm glad that you found my workaround relevant for babel usage as CLI.
I use webpack too for my web bundles, but for server side code, till now, I don't see enough benefit to use bundler against all problems and complexity that's adding with NodeJS, that's why I worked with babel CLI and encountered a lot of misadventures bringing me too develop babel-watch-extra.

[jescalan]

Hey is anyone currently working on this? I need to fix this for this plugin to work (if a new file is added, it will not be picked up because the expansion is cached), and am willing to contribute a fix. Just want to make sure I'm not duplicating efforts.

[kentcdodds]

I don't think anyone's working on it, but I know a lot of people would love it if it were fixed. Seems kinda non-trivial though 🤔

[jescalan]

Going to aim to get it fixed for babel-loader specifically, at least to start. I'll do a little writeup on my plan to fix and see if I can get some feedback here before moving forward.

[nicolo-ribaudo]

Thanks! If you need any help, you can ask in our Slack's #development channel

[devuxer]

@jescalan, Just wondering if you had an update on this. Thanks.

[jescalan]

Sadly, no update yet - I have this queued but it keeps sinking down the priorities for other things. In the meantime we are disabling the babel cache and getting by, albeit a bit more slowly.

[vedantroy]

Currently working on this. Is it sufficient to have plugins indicate dependencies on external files just once, or should this be on a per-file basis?

Also is there anything I should be wary of (e.g solutions the Babel team has considered that seem obvious, but in reality don't work for some subtle reason).

[goatandsheep]

@nicolo-ribaudo I'm testing this new version v7.17.0 and the function is present in my library but I'm getting some errors. I used the code sample provided in the PR:

function plugin(api) {
  const dotEnvFilename = `${process.cwd()}/.env`;

  const env = api.cache.using(() => fs.readFileSync(dotEnvFilename, "utf8"));
  api.addExternalDependency(dotEnvFilename);
  
  return {
    visitor: {}...
  }
}

the cache function gets Cannot change caching after evaluation has completed

and the addExternalDependency function gets Cannot add property 0, object is not extensible

Code attached: https://github.com/goatandsheep/react-native-dotenv/pull/288/files?diff=unified&w=1

[JLHwung]

@goatandsheep Thanks for the reproduction. It has been tracked in #14233.