Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: avoid string copy when processing input source-map #10885

Merged
merged 3 commits into from Dec 18, 2019

Conversation

JLHwung
Copy link
Contributor

@JLHwung JLHwung commented Dec 17, 2019

Q                       A
Fixed Issues? Fixes #10875
Patch: Bug Fix? Yes
Tests Added + Pass? No test is added, it is covered by current test suite
License MIT

This PR fixes a performance regression introduced in #10623.

// fromMapFileComment requires the whole comment block
`//${lastComment}`,

The lastComment was copied into a new string, prefixed by // and later unwrapped by convert-source-map.
https://github.com/thlorenz/convert-source-map/blob/9e9a7a9b652c30878c9c9aa591d861a9fdf61a7e/index.js#L31

This approach introduced significant memory footprint when there are many files with input source-map, i.e. the test repo mentioned in #10875. Here we use convertSourceMap.fromJSON to avoid unnecessary string copy.

@artisb45 Thank you for the reproduction repo, it helps me a lot! You can test flight this fix by downloading and overwrite node_modules/@babel/core/lib/transformation/normalize-file.js with this file. We are expected to land it in a patch release if review is good.

@JLHwung JLHwung added PR: Bug Fix 🐛 A type of pull request used for our changelog categories Priority: High area: sourcemaps labels Dec 17, 2019
@@ -157,7 +160,7 @@ function parser(

// eslint-disable-next-line max-len
const INLINE_SOURCEMAP_REGEX = /^[@#]\s+sourceMappingURL=data:(?:application|text)\/json;(?:charset[:=]\S+?;)?base64,(?:.*)$/;
const EXTERNAL_SOURCEMAP_REGEX = /^[@#][ \t]+sourceMappingURL=(?:[^\s'"`]+?)[ \t]*$/;
const EXTERNAL_SOURCEMAP_REGEX = /^[@#][ \t]+sourceMappingURL=(?<url>[^\s'"`]+?)[ \t]*$/;
Copy link
Contributor Author

@JLHwung JLHwung Dec 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dogfooding named-capturing-regex here.

The file size is bloated by helpers (6678 B -> 11276 B) but I think it is okay since named-capturing-regex is supported on Node.js 10+, which means we can get rid of the helpers in v8.

I have changed my mind.

Copy link
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @jridgewell suggestion. Apart from that, ✔️

Co-Authored-By: Justin Ridgewell <justin@ridgewell.name>
@existentialism existentialism merged commit b3c7df9 into babel:master Dec 18, 2019
@existentialism
Copy link
Member

Derp, missed the flow errors... will push a PR fix in the morning if no one gets to it first!

@abonckus
Copy link

abonckus commented Dec 18, 2019

Hello, thanks for such a quick response to this issue @JLHwung ! However I tried to replace the file you provided and it did not seem to do anything. Not sure if I'm doing it wrong, but I'm just supposed to replace the file at the path right? I'm using the same repo I linked in the issue. Also tried rolling back before ejection and then replacing - same result.

@nicolo-ribaudo
Copy link
Member

#10885 (comment)

Ops, I didn't see your comment 😅

@JLHwung
Copy link
Contributor Author

JLHwung commented Dec 18, 2019

@artisb45 Oops, you may need clear the babel-loader cache first because it is an ad-hoc approach.

# Clear babel-loader cache
rm -rf node_modules/.cache/babel-loader
cp normalize-file.js node_modules/@babel/core/lib/transformation/normalize-file.js
npm run build

It went well on my local environment. You can also wait for the upcoming 7.7.7 release.

Update:

#10875 is not fixed: #10875 (comment)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area: sourcemaps outdated A closed issue/PR that is archived due to age. Recommended to make a new issue PR: Bug Fix 🐛 A type of pull request used for our changelog categories Priority: High
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Very high memory usage compiling mozilla/PDF.js
5 participants