Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor Lexer #3115

Merged
merged 2 commits into from Jun 3, 2021
Merged

Refactor Lexer #3115

merged 2 commits into from Jun 3, 2021

Commits on Jun 3, 2021

  1. Refactor Lexer 

    The lexer needed some cleanup, I found myself doing this as part of a Unicode RFC, but factoring all that out to make the Unicode RFC PR easier to follow.

* Always use hexadecimal form for code values.
* Remove use of `isNaN` for checking source over-reads.
* Defines `isSourceCharacter`
* Add more documentation and comments, also replaces regex with lexical grammar
* Simplifies error messages
* Adds additional tests
    @leebyron
    leebyron committed Jun 3, 2021
    Copy the full SHA
    540c59e View commit details
    Browse the repository at this point in the history

  2. Code review 

    Co-authored-by: Ivan Goncharov <ivan.goncharov.ua@gmail.com>
    @leebyron @IvanGoncharov
    leebyron and IvanGoncharov committed Jun 3, 2021
    Copy the full SHA
    b6864ac View commit details
    Browse the repository at this point in the history