New: Recoverable Error Handling

[mysticatea]

Link to Rendered RFC.

Summary

ESLint cannot verify source code if the code has a syntax error. However, we can make valid AST even if several kinds of syntax errors existed. For example, conflict of variable names doesn't break AST. This RFC calls such a syntax error as "Recoverable Errors".

This RFC adds handling of Recoverable Errors into ESLint.

Related Issues

• Provide a way to run Espree in tolerant mode eslint#3815
• Proposal: errors option espree#368
• Breaking: make no-redeclare stricter (fixes #11370, fixes #11405) eslint#11509 (review)

[btmills]

I like the idea! It seems fairly straightforward. On possible addition - since rules might not handle an invalid AST in the presence of recoverable errors, and an incorrect fix could make things worse, should ESLint skip applying fixes if there are recoverable errors?

[not-an-aardvark]

I generally like the idea, but I'm also concerned about how this would affect rules. In addition to what @btmills mentioned about autofixing, a rule might simply crash if certain invariants about the AST are violated. It seems like it could be confusing for separation of responsibility if a rule needs to fix bugs that pertain to invalid ASTs.

For example, if a parser started raising "recoverable" errors for missing closing parentheses rather than "unrecoverable" errors, I suspect a significant number of rules would start to crash because they assume all parentheses have a match. Would those rules need to be updated to address the new case from the parser? With some rules, it's not clear that there's a well-defined reasonable behavior when invalid syntax occurs.

Maybe one possibility would be to simply swallow rule crashes when a recoverable parser error occurs (i.e. effectively providing linting on a "best-effort" basis for that case).

[mysticatea]

Thank you for the great feedback. I added some description.

• If the parser returned any recoverable errors, the Linter disables autofix as making disableFixes option true internally because it's considered not safe.
• If the parser returned any recoverable errors, the Linter catches exceptions which were thrown from rules and reports the exceptions as regular messages (e.g., "'a-rule' failed to lint the code because of syntax error(s)") rather than crash. If a rule has both an exception and regular messages, the Linter drops the regular messages to avoid confusion of wrong messages.

[ilyavolodin]

On one hand, I like the idea, and I advocated for it before. On the other, I agree with others that it might cause confusion with rules. Also disabling autofix on the whole file just because of one small error in the beginning of the file, might confuse the end user.

[kaicataldo]

Thanks for writing this up!

While I understand why this could improve the user experience in some cases, I’m not convinced the value this feature adds outweighs the extra complexity introduced. I think we can eat the maintenance cost, but I am concerned with how this might introduce confusion and make it harder for end users to understand why something may or may not be working (i.e. understanding what a “recoverable error” is requires understanding some of the underlying implementation of what an AST is and what makes it valid or usable).

Additionally, I think end users will have to immediately fix the “recoverable error” anyway to ensure their linting results are 100% correct, so while it might surface some linting errors sooner, I’m not sure it would end up saving time in the end.

[mysticatea]

Hi. Thank you for your feedback.

I have updated this RFC to follow the feedback on the TSC meeting.

• By default, ESLint doesn't run rules if recoverable errors existed. Instead, ESLint shows those recoverable errors. Practically, this is the support for multiple syntax errors.
• verifyOnRecoverableParsingErrors option was added. This option is a core option rather than a parser option because it doesn't change the parser behavior. It changes how the Linter handles recoverable errors.
  If this option was given, ESLint runs rules even if recoverable errors existed. Then it shows both results.

[platinumazure]

This looks good to me, thanks! Maybe others will have objections, though.

[btmills]

Nice work @mysticatea! LGTM. As a direction for future work, your suggestion about treating ES version errors as recoverable is intriguing. My most popular answer on Stack Overflow is explaining how to resolve a parsing error on async/await by setting the correct version. I’d be willing to sacrifice a source of recurring Internet points for a better user experience!

[btmills]

@kaicataldo is there anything in this design that would preclude compatibility with Babel?

[ilyavolodin]

I think in worst case, babel will just continue to work exactly as it does now, by throwing unrecoverable error.

[kaicataldo]

I’ll have to dig deeper for a more definitive answer, but if I’m reading this RFC correctly it would continue to work as it does now (as @ilyavolodin mentions above). I’ll try to take a look sometime over the next few days.

[kaicataldo]

Sorry, a bit late to the party, but I looked over babel-parser's code and confirmed with the team that babel-parser does not currently support recoverable errors, however a member of the team did mention that they would be supportive of adding that feature if it's needed for babel-eslint. As of right now, I think we should move forward assuming babel-eslint will continue working as it does now.

[ilyavolodin]

With the new changes, this RFC looks good to me. Thanks.

[kaicataldo]

Here’s an example of how an incomplete AST can affect rules: typescript-eslint/typescript-eslint#441

[kaicataldo]

After the TSC discussion, I was under the impression that we weren’t going to pursue linting with recoverable errors at all, but rather we were going to list all syntax errors until a file was no longer parseable. I still have the same concerns about being able to lint with recoverable errors as before.

Edit: Are we committing to updating all core rules to handle recoverable errors if we accept this RFC?

[mysticatea]

Multiple members had mentioned the option to configure how ESLint handle recoverable errors.

I believe we don't fix rules for invalid AST. But fixes rules for valid AST.

[mysticatea]

About typescript-eslint/typescript-eslint#441, that looks like a valid AST, so we should fix our rules. ESTree spec doesn't guarantee that VariableDeclaration#declarations has 1 declarator at least.

[kaicataldo]

Thanks for working on this!

[mysticatea]

I'll merge this RFC. Thank you!