Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RegExp syntax checking performance #58339

Merged
merged 8 commits into from Apr 29, 2024
Merged

RegExp syntax checking performance #58339

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a4db189
Extract scanRegularExpressionWorker from reScanSlashToken
rbuckton Apr 26, 2024
5333c47
Fix spelling
rbuckton Apr 26, 2024
2af307e
Reduce array/set allocation count
rbuckton Apr 26, 2024
673d843
Use existing text/end from closure
rbuckton Apr 26, 2024
83379be
Make unchecked reads of 'text' explicit
rbuckton Apr 26, 2024
0785328
Use charCodeChecked where possible in regex scanner
rbuckton Apr 26, 2024
d413625
Use charCodeChecked in a few more places
rbuckton Apr 27, 2024
de73b32
Update src/compiler/scanner.ts
rbuckton Apr 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/compiler/checker.ts
View file
Open in desktop
Expand Up @@ -31360,7 +31360,7 @@ export function createTypeChecker(host: TypeCheckerHost): TypeChecker {

function checkGrammarRegularExpressionLiteral(node: RegularExpressionLiteral) {
const sourceFile = getSourceFileOfNode(node);
if (!hasParseDiagnostics(sourceFile)) {
if (!hasParseDiagnostics(sourceFile) && !node.isUnterminated) {
let lastError: DiagnosticWithLocation | undefined;
scanner ??= createScanner(ScriptTarget.ESNext, /*skipTrivia*/ true);
scanner.setScriptTarget(sourceFile.languageVersion);
Expand Down
2 changes: 1 addition & 1 deletion src/compiler/diagnosticMessages.json
View file
Open in desktop
Expand Up @@ -1733,7 +1733,7 @@
"category": "Error",
"code": 1519
},
"Expected a class set oprand.": {
"Expected a class set operand.": {
"category": "Error",
"code": 1520
},
Expand Down
1,947 changes: 1,002 additions & 945 deletions src/compiler/scanner.ts
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions src/compiler/types.ts
View file
Open in desktop
Expand Up @@ -7612,6 +7612,7 @@ export type CommandLineOption = CommandLineOptionOfCustomType | CommandLineOptio
// dprint-ignore
/** @internal */
export const enum CharacterCodes {
EOF = -1,
nullCharacter = 0,
maxAsciiCharacter = 0x7F,

Expand Down
20 changes: 10 additions & 10 deletions tests/baselines/reference/regularExpressionScanning(target=es2015).errors.txt
View file
Open in desktop
Expand Up @@ -157,24 +157,24 @@ regularExpressionScanning.ts(37,61): error TS1508: Unexpected '}'. Did you mean
regularExpressionScanning.ts(37,63): error TS1517: Range out of order in character class.
regularExpressionScanning.ts(37,76): error TS1535: This character cannot be escaped in a regular expression.
regularExpressionScanning.ts(38,8): error TS1005: '--' expected.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,12): error TS1005: '--' expected.
regularExpressionScanning.ts(38,15): error TS1522: A character class must not contain a reserved double punctuator. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,20): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,28): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,40): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,47): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,49): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,55): error TS1511: '\q' is only available inside character class.
regularExpressionScanning.ts(38,57): error TS1508: Unexpected '{'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,61): error TS1508: Unexpected '}'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,66): error TS1508: Unexpected '-'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,67): error TS1005: '--' expected.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,75): error TS1508: Unexpected '&'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,87): error TS1501: This regular expression flag is only available when targeting 'esnext' or later.
regularExpressionScanning.ts(39,56): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(39,67): error TS1005: '&&' expected.
Expand Down Expand Up @@ -561,9 +561,9 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1005: '--' expected.
~~
Expand All @@ -579,7 +579,7 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag
~
!!! error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~~
!!! error TS1511: '\q' is only available inside character class.
~
Expand All @@ -591,11 +591,11 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~
!!! error TS1508: Unexpected '&'. Did you mean to escape it with backslash?

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~
!!! error TS1501: This regular expression flag is only available when targeting 'esnext' or later.
/[[^\P{Decimal_Number}&&[0-9]]&&\p{L}&&\p{ID_Continue}--\p{ASCII}\p{CWCF}]/v,
Expand Down
20 changes: 10 additions & 10 deletions tests/baselines/reference/regularExpressionScanning(target=es5).errors.txt
View file
Open in desktop
Expand Up @@ -164,24 +164,24 @@ regularExpressionScanning.ts(37,63): error TS1517: Range out of order in charact
regularExpressionScanning.ts(37,76): error TS1535: This character cannot be escaped in a regular expression.
regularExpressionScanning.ts(37,87): error TS1501: This regular expression flag is only available when targeting 'es6' or later.
regularExpressionScanning.ts(38,8): error TS1005: '--' expected.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,12): error TS1005: '--' expected.
regularExpressionScanning.ts(38,15): error TS1522: A character class must not contain a reserved double punctuator. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,20): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,28): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,40): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,47): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,49): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,55): error TS1511: '\q' is only available inside character class.
regularExpressionScanning.ts(38,57): error TS1508: Unexpected '{'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,61): error TS1508: Unexpected '}'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,66): error TS1508: Unexpected '-'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,67): error TS1005: '--' expected.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,75): error TS1508: Unexpected '&'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,87): error TS1501: This regular expression flag is only available when targeting 'esnext' or later.
regularExpressionScanning.ts(39,56): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(39,67): error TS1005: '&&' expected.
Expand Down Expand Up @@ -582,9 +582,9 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1005: '--' expected.
~~
Expand All @@ -600,7 +600,7 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag
~
!!! error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~~
!!! error TS1511: '\q' is only available inside character class.
~
Expand All @@ -612,11 +612,11 @@ regularExpressionScanning.ts(47,101): error TS1501: This regular expression flag

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~
!!! error TS1508: Unexpected '&'. Did you mean to escape it with backslash?

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~
!!! error TS1501: This regular expression flag is only available when targeting 'esnext' or later.
/[[^\P{Decimal_Number}&&[0-9]]&&\p{L}&&\p{ID_Continue}--\p{ASCII}\p{CWCF}]/v,
Expand Down
20 changes: 10 additions & 10 deletions tests/baselines/reference/regularExpressionScanning(target=esnext).errors.txt
View file
Open in desktop
Expand Up @@ -141,24 +141,24 @@ regularExpressionScanning.ts(37,61): error TS1508: Unexpected '}'. Did you mean
regularExpressionScanning.ts(37,63): error TS1517: Range out of order in character class.
regularExpressionScanning.ts(37,76): error TS1535: This character cannot be escaped in a regular expression.
regularExpressionScanning.ts(38,8): error TS1005: '--' expected.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,9): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,11): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,12): error TS1005: '--' expected.
regularExpressionScanning.ts(38,15): error TS1522: A character class must not contain a reserved double punctuator. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,20): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,28): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,40): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,47): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,49): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,50): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,55): error TS1511: '\q' is only available inside character class.
regularExpressionScanning.ts(38,57): error TS1508: Unexpected '{'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,61): error TS1508: Unexpected '}'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,66): error TS1508: Unexpected '-'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,67): error TS1005: '--' expected.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,70): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(38,75): error TS1508: Unexpected '&'. Did you mean to escape it with backslash?
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set oprand.
regularExpressionScanning.ts(38,85): error TS1520: Expected a class set operand.
regularExpressionScanning.ts(39,56): error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
regularExpressionScanning.ts(39,67): error TS1005: '&&' expected.
regularExpressionScanning.ts(41,5): error TS1518: Anything that would possibly match more than a single character is invalid inside a negated character class.
Expand Down Expand Up @@ -503,9 +503,9 @@ regularExpressionScanning.ts(47,89): error TS1518: Anything that would possibly

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.

!!! error TS1005: '--' expected.
~~
Expand All @@ -521,7 +521,7 @@ regularExpressionScanning.ts(47,89): error TS1518: Anything that would possibly
~
!!! error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~~
!!! error TS1511: '\q' is only available inside character class.
~
Expand All @@ -533,11 +533,11 @@ regularExpressionScanning.ts(47,89): error TS1518: Anything that would possibly

!!! error TS1005: '--' expected.

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
~
!!! error TS1508: Unexpected '&'. Did you mean to escape it with backslash?

!!! error TS1520: Expected a class set oprand.
!!! error TS1520: Expected a class set operand.
/[[^\P{Decimal_Number}&&[0-9]]&&\p{L}&&\p{ID_Continue}--\p{ASCII}\p{CWCF}]/v,
~~
!!! error TS1519: Operators must not be mixed within a character class. Wrap it in a nested class instead.
Expand Down