Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ts): parse multiline declarations correctly #12776

Merged
merged 10 commits into from Feb 16, 2021

Conversation

fedeci
Copy link
Member

@fedeci fedeci commented Feb 6, 2021

Q                       A
Fixed Issues? Fixes #12773
Patch: Bug Fix? Yes
Major: Breaking Change?
Minor: New Feature?
Tests Added + Pass? Yes
Documentation PR Link
Any Dependency Changes?
License MIT

We check that the next token does not have a previous line break (only if we want to run next()). This is necessary to avoid skipping a token that will not be parsed.

@codesandbox-ci
Copy link

codesandbox-ci bot commented Feb 6, 2021

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit cb2c0ec:

Sandbox Source
babel-repl-custom-plugin Configuration
babel-plugin-multi-config Configuration

@babel-bot
Copy link
Collaborator

babel-bot commented Feb 6, 2021

Build successful! You can test your changes in the REPL here: https://babeljs.io/repl/build/40507/

@nicolo-ribaudo nicolo-ribaudo added area: typescript pkg: parser PR: Bug Fix 🐛 A type of pull request used for our changelog categories labels Feb 6, 2021
Copy link
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@armano2
Copy link
Contributor

armano2 commented Feb 7, 2021

is this going to also work with namespaces?

this seem to be parsed as namespace and should not

declare namespace 
oo 
{ }

https://www.typescriptlang.org/play?ssl=3&ssc=4&pln=1&pc=1#code/CYUwxgNghgTiAEA7KBbEBnADlMCCwAUAPZHyEDe8AvkA

@fedeci
Copy link
Member Author

fedeci commented Feb 7, 2021

@armano2 yes it is parsed as a declaration, it needs to be fixed

@armano2
Copy link
Contributor

armano2 commented Feb 7, 2021

should i raise new ticket for this?

@nicolo-ribaudo
Copy link
Member

nicolo-ribaudo commented Feb 7, 2021

It's not needed, since those are practically two instances of the same bug @fedeci can fix it in this PR if he wants.

return this.tsParseDeclaration(nany, value, /* next */ true);
const oldState = this.state.clone(true);
Copy link
Member Author

@fedeci fedeci Feb 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 100% happy with this, if there was a way to check that the next token is a line terminator it would be way better.


edit: maybe the oldState should be saved and applied directly in tsParseDeclaration

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can clone state in tsTryParseDeclare. Since it is a try parse method, if any error is thrown in its subroutine, it should be ignored and the parser state should be restored.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it already cloned in tsTryParseDeclare subroutine? Or you just mean tsTryParse should be used?

return this.tsTryParse(() =>
  this.tsParseDeclaration(nany, value, /* next */ true),
);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although it was named tsTryParseDeclare, it does not restore state when error is thrown:

tsTryParseDeclare(nany: any): ?N.Declaration {
if (this.isLineTerminator()) {
return;
}
let starttype = this.state.type;
let kind;
if (this.isContextual("let")) {
starttype = tt._var;
kind = "let";
}
return this.tsInDeclareContext(() => {
switch (starttype) {
case tt._function:
nany.declare = true;
return this.parseFunctionStatement(
nany,
/* async */ false,
/* declarationPosition */ true,
);
case tt._class:
// While this is also set by tsParseExpressionStatement, we need to set it
// before parsing the class declaration to now how to register it in the scope.
nany.declare = true;
return this.parseClass(
nany,
/* isStatement */ true,
/* optionalId */ false,
);
case tt._const:
if (this.match(tt._const) && this.isLookaheadContextual("enum")) {
// `const enum = 0;` not allowed because "enum" is a strict mode reserved word.
this.expect(tt._const);
this.expectContextual("enum");
return this.tsParseEnumDeclaration(nany, /* isConst */ true);
}
// falls through
case tt._var:
kind = kind || this.state.value;
return this.parseVarStatement(nany, kind);
case tt.name: {
const value = this.state.value;
if (value === "global") {
return this.tsParseAmbientExternalModuleDeclaration(nany);
} else {
return this.tsParseDeclaration(nany, value, /* next */ true);
}
}
}
});
}

you just mean tsTryParse should be used?

That could work, just note that tsTryParseDeclare does not equal to tryParse(() => tsParseDeclaration): The former addresses declare ... parsing, the latter handles general declaration, e.g.type X = ....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, but I am not sure if the entire tsTryParseDeclare method should be wrapped with tsTryParse because it is quite expensive. For the moment I will just update the current fix which is working fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#12771 (comment) might be useful for this, if the goal is to handle the [no LineTerminator here].

@fedeci fedeci force-pushed the fix-multiline-module-declaration branch 2 times, most recently from 874e203 to 953bda2 Compare February 9, 2021 15:02
@fedeci fedeci changed the title fix(ts): parse multiline module declaration correctly fix(ts): parse multiline module and namespace declarations correctly Feb 9, 2021
Copy link
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a comment on the coding style, otherwise the PR looks good!

packages/babel-parser/src/plugins/typescript/index.js Outdated Show resolved Hide resolved
packages/babel-parser/src/plugins/typescript/index.js Outdated Show resolved Hide resolved
@fedeci fedeci force-pushed the fix-multiline-module-declaration branch from ee05451 to aca0a8d Compare February 10, 2021 21:29
Copy link
Contributor

@JLHwung JLHwung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the following cases? All of them should throw

declare interface
i
{}

declare enum
e
{}

declare abstract
class A {
}

declare type
A = number

The idea is that the line break check should be applied universally to the keyword after declare.

@fedeci
Copy link
Member Author

fedeci commented Feb 11, 2021

// tsc converts this to an enum declaration according to AST explorer, but all of the others require a fix :)
declare enum
e
{}

@fedeci fedeci changed the title fix(ts): parse multiline module and namespace declarations correctly fix(ts): parse multiline declarations correctly Feb 12, 2021
@fedeci fedeci force-pushed the fix-multiline-module-declaration branch from 267be70 to df2f32c Compare February 12, 2021 16:53
@fedeci fedeci force-pushed the fix-multiline-module-declaration branch from df2f32c to f8e3472 Compare February 12, 2021 17:00
@fedeci fedeci force-pushed the fix-multiline-module-declaration branch from ff7af7b to cb2c0ec Compare February 12, 2021 19:46
Copy link
Contributor

@armano2 armano2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i executed alignment tests between typescript and your pr, and it looks like its parsed correctly now, thanks 👍

@nicolo-ribaudo nicolo-ribaudo merged commit 8e9143f into babel:main Feb 16, 2021
@nicolo-ribaudo
Copy link
Member

Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area: typescript outdated A closed issue/PR that is archived due to age. Recommended to make a new issue pkg: parser PR: Bug Fix 🐛 A type of pull request used for our changelog categories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

@babel/parser(ts): incorrect multiline parsing of module declaration
5 participants