Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(tokenizer): Use explicit offsets for locations #402

Merged
merged 4 commits into from Feb 24, 2022

Commits on Feb 11, 2022

  1. refactor(tokenizer): Use explicit offsets for locations

    Supersedes inikulin#397
    
    Previously, `ctLoc` was overwritten on every new character. inikulin#397 attempted to improve this by maintaining the locations in an updated object, and creating copies when needed.
    
    With this PR, we instead use explicit offsets when creating tokens. There is no more need to update locations; instead, we exploit the fact that a *DATA state is entered only after emitting the previous token.
    
    This change should lead to a great reduction in allocations when using location information.
    fb55 committed Feb 11, 2022
    Configuration menu
    Copy the full SHA
    65cc12c View commit details
    Browse the repository at this point in the history
  2. Remove doctypeStartLoc property

    fb55 committed Feb 11, 2022
    Configuration menu
    Copy the full SHA
    48a04d0 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2022

  1. Configuration menu
    Copy the full SHA
    cb22db8 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2022

  1. Remove unnecessary location property

    Merge `ctLoc` and `currentAttrLocation` as `currentLocation`
    fb55 committed Feb 21, 2022
    Configuration menu
    Copy the full SHA
    f6c21b4 View commit details
    Browse the repository at this point in the history