fix: Enable cache with externalDependencies

[liuxingbaoyu]

Please Read the CONTRIBUTING Guidelines
In particular the portion on Commit Message Formatting

Please check if the PR fulfills these requirements

• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Other... Please describe:

What is the current behavior? (You can also link to an open issue here)
Fixes: #983

What is the new behavior?

Does this PR introduce a breaking change?

• Yes
• No

If this PR contains a breaking change, please describe the following...

• Impact:
• Migration path for existing applications:
• Github Issue(s) this is regarding:

Other information:
I'm not sure how to test it, I manually tested it successfully in the repro of #983.
This may need to be backported to 8.x as this is a regression.

[JLHwung]

Thanks for the PR. I am afraid this approach is not very performant: essentially we are querying fs for external dependencies stats during every webpack re-compilation, even though most of re-run may not be trigged by external dependencies.

We can obtain a set of changed files from the webpack watch file system:

this._compiler.watchFileSystem.watcher.getAggregated()
// returns { changes: Set<string>, removals: Set<string> }

and then we can see if there are changes in external dependencies, if so we can invalidate and purge the cache, otherwise we assume the cache is safe to be reused. In this approach all the extra works are done in memory, no extra IO is involved.

Of course this._compiler is a webpack specific property, the docs has warned that using this_compiler would "have a negative impact on your loaders compatibility." We can open an issue at the webpack repo to ask for their input.

[liuxingbaoyu]

We can open an issue at the webpack repo to ask for their input.

Maybe in webpack5 we don't need to implement our own caching.

[liuxingbaoyu]

Unfortunately this doesn't seem to work.

[JLHwung]

We can open an issue at the webpack repo to ask for their input.

Maybe in webpack5 we don't need to implement our own caching.

Well babel-loader@9 does support webpack 5 only. You can explore if we can get rid of our own cache and count on the webpack persistent cache.

[JLHwung]

Unfortunately this doesn't seem to work.

Does it work if we pass absolute file paths to this.addDependency?

[liuxingbaoyu]

Should already be an absolute path in this test.

[liuxingbaoyu]

Well babel-loader@9 does support webpack 5 only. You can explore if we can get rid of our own cache and count on the webpack persistent cache.

Yes, I plan to finish the backport first to fix the regression, then migrate the code to ts, and then try to refactor the cache.

[liuxingbaoyu]

My prettier-vscode doesn't seem to be working properly.

[JLHwung]

Did you try https://yarnpkg.com/getting-started/editor-sdks?

[liuxingbaoyu]

Unfortunately this still caches twice.