Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix BytePos -> CharPos calculations #6574

Merged
merged 4 commits into from Dec 4, 2022
Merged

Fix BytePos -> CharPos calculations #6574

merged 4 commits into from Dec 4, 2022

Conversation

jridgewell
Copy link
Contributor

@jridgewell jridgewell commented Dec 3, 2022 •
edited

Description:

This fixes the BytePos -> CharPos calculation necessary for source maps. There were a few issues in the old code:

  1. UTF-8 maps 1-3 bytes into 1 UTF-16 char, but 4 bytes into 2 UTF-16 chars
  2. The starting offset was not recorded when we reach the end of the multibyte_chars iteration
  3. The mappings can be unordered, meaning we need to restart UTF-16 offset calculation

BREAKING CHANGE:

Related issue (if exists):

Fixes #6552

@jridgewell
Fix UTF-8 -> UTF-16 calculation
ef0c185 
There were a few issues in the old code:

1. UTF-8 maps 1-3 bytes into 1 UTF-16 char, but 4 bytes into 2 UTF-16 chars
2. The starting offset was not recorded when we end the `multibyte_chars` iteration
3. The `mappings` can be unordered, meaning we need to restart UTF-16 offset calculation
kdy1
kdy1 previously approved these changes Dec 4, 2022
View reviewed changes
Copy link
Member

@kdy1 kdy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much!

swc-bump:

  • swc_common

@jridgewell
Copy link
Contributor Author

Sorry, I just pushed a reverse conversion for when the mapping isn't ordered. This hopefully makes it a bit faster.

kdy1
kdy1 previously approved these changes Dec 4, 2022
View reviewed changes
Copy link
Member

@kdy1 kdy1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@kdy1 kdy1 enabled auto-merge (squash) December 4, 2022 04:03
@kdy1 kdy1 added this to the Planned milestone Dec 4, 2022
@jridgewell
Copy link
Contributor Author

Ah, one of my recent changes is causing a failure. Working to fix it now.

@jridgewell
Copy link
Contributor Author

Should be good now.

@jridgewell jridgewell mentioned this pull request Dec 4, 2022
Closed
@IWANABETHATGUY
Copy link
Contributor

It seems that this pr regressed performance again

PR

image

Main

image

@kdy1 kdy1 disabled auto-merge December 4, 2022 04:40
@IWANABETHATGUY
Copy link
Contributor

large.js https://gist.github.com/IWANABETHATGUY/8cbfa7246af46a26cd8480ce7e9a5f47

@kdy1 kdy1 enabled auto-merge (squash) December 4, 2022 05:50
@kdy1 kdy1 disabled auto-merge December 4, 2022 05:50
@jridgewell
Copy link
Contributor Author

Can you explain how you're doing the benchmark? I'm actually seeing an ~2% improvement with that file running time ./target/release/swc compile --source-maps true large.js > /dev/null

@kdy1 kdy1 merged commit a203fdb into swc-project:main Dec 4, 2022
@kdy1
Copy link
Member

kdy1 commented Dec 4, 2022

Let's fix performance regression with another PR..

@jridgewell jridgewell deleted the multibyte-fix branch December 4, 2022 05:52
@IWANABETHATGUY
Copy link
Contributor

Sorry, you are right, I just using the wrong branch. No performance regression.

@kdy1 kdy1 modified the milestones: Planned, v1.3.22 Dec 9, 2022
@kdy1 kdy1 added this to the v1.3.22 milestone Dec 9, 2022
@swc-project swc-project locked as resolved and limited conversation to collaborators Jan 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@kdy1 kdy1 kdy1 approved these changes

Assignees
No one assigned
Labels
None yet
Milestone
v1.3.22
Development

Successfully merging this pull request may close these issues.

SWC generate wrong sourcemap when have mutibyte character after 1.3.20
3 participants
@jridgewell @IWANABETHATGUY @kdy1