From 7a83ca067cf49a2228d26bf2d05fd0c9431f4767 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter Date: Sun, 12 Feb 2023 16:43:27 -0500 Subject: [PATCH 1/8] docs: Custom Parsers cleanup/expansion --- docs/src/extend/custom-parsers.md | 98 +++++++++++++++++++------------ 1 file changed, 62 insertions(+), 36 deletions(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index a341da5088a..176d6ac5047 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -8,63 +8,52 @@ eleventyNavigation: --- -If you want to use your own parser and provide additional capabilities for your rules, you can specify your own custom parser. If a `parseForESLint` method is exposed on the parser, this method will be used to parse the code. Otherwise, the `parse` method will be used. Both methods should take in the source code as the first argument, and an optional configuration object as the second argument (provided as `parserOptions` in a config file). The `parse` method should simply return the AST. The `parseForESLint` method should return an object that contains the required property `ast` and optional properties `services`, `scopeManager`, and `visitorKeys`. +ESLint custom parsers let you extend ESLint to support linting new non-standard JavaScript language features or custom syntax in your code. A parser is responsible for taking your code and transforming it into an abstract syntax tree (AST) that ESLint can then analyze and lint. -* `ast` should contain the AST. +## Creating a Custom Parser + +If a `parseForESLint` method is exposed on the parser, this method will be used to parse the code. Otherwise, the `parse` method will be used. Both methods should take in the source code as the first argument, and an optional configuration object as the second argument (provided as `parserOptions` in a config file). + +TODO: have a simple example here w parse and parserOptions + +## `parse` Return Object + +The `parse` method should simply return the [AST](#ast-specification) object. + +### `parseForESLint` Return Object + +The `parseForESLint` method should return an object that contains the required property `ast` and optional properties `services`, `scopeManager`, and `visitorKeys`. + +* `ast` should contain the [AST](#ast-specification) object. * `services` can contain any parser-dependent services (such as type checkers for nodes). The value of the `services` property is available to rules as `context.parserServices`. Default is an empty object. * `scopeManager` can be a [ScopeManager](./scope-manager-interface) object. Custom parsers can use customized scope analysis for experimental/enhancement syntaxes. Default is the `ScopeManager` object which is created by [eslint-scope](https://github.com/eslint/eslint-scope). * Support for `scopeManager` was added in ESLint v4.14.0. ESLint versions which support `scopeManager` will provide an `eslintScopeManager: true` property in `parserOptions`, which can be used for feature detection. * `visitorKeys` can be an object to customize AST traversal. The keys of the object are the type of AST nodes. Each value is an array of the property names which should be traversed. Default is [KEYS of `eslint-visitor-keys`](https://github.com/eslint/eslint-visitor-keys#evkkeys). * Support for `visitorKeys` was added in ESLint v4.14.0. ESLint versions which support `visitorKeys` will provide an `eslintVisitorKeys: true` property in `parserOptions`, which can be used for feature detection. -You can find an ESLint parser project [here](https://github.com/typescript-eslint/typescript-eslint). - -```json -{ - "parser": "./path/to/awesome-custom-parser.js" -} -``` - -```javascript -var espree = require("espree"); -// awesome-custom-parser.js -exports.parseForESLint = function(code, options) { - return { - ast: espree.parse(code, options), - services: { - foo: function() { - console.log("foo"); - } - }, - scopeManager: null, - visitorKeys: null - }; -}; - -``` - -## The AST specification +## AST Specification The AST that custom parsers should create is based on [ESTree](https://github.com/estree/estree). The AST requires some additional properties about detail information of the source code. -### All nodes: +### All Nodes All nodes must have `range` property. * `range` (`number[]`) is an array of two numbers. Both numbers are a 0-based index which is the position in the array of source code characters. The first is the start position of the node, the second is the end position of the node. `code.slice(node.range[0], node.range[1])` must be the text of the node. This range does not include spaces/parentheses which are around the node. -* `loc` (`SourceLocation`) must not be `null`. [The `loc` property is defined as nullable by ESTree](https://github.com/estree/estree/blob/25834f7247d44d3156030f8e8a2d07644d771fdb/es5.md#node-objects), but ESLint requires this property. On the other hand, `SourceLocation#source` property can be `undefined`. ESLint does not use the `SourceLocation#source` property. +* `loc` (`SourceLocation`) must not be `null`. [The `loc` property is defined as nullable by ESTree](https://github.com/estree/estree/blob/25834f7247d44d3156030f8e8a2d07644d771fdb/es5.md#node-objects), but ESLint requires this property. `SourceLocation#source` property can be `undefined`. ESLint does not use the `SourceLocation#source` property. -The `parent` property of all nodes must be rewritable. ESLint sets each node's `parent` property to its parent node while traversing, before any rules have access to the AST. +The `parent` property of all nodes must be rewritable. Before any rules have access to the AST, ESLint sets each node's `parent` property to its parent node while traversing. -### The `Program` node: +### The `Program` Node -The `Program` node must have `tokens` and `comments` properties. Both properties are an array of the below Token interface. +The `Program` node must have `tokens` and `comments` properties. Both properties are an array of the below `Token` interface. ```ts interface Token { type: string; loc: SourceLocation; - range: [number, number]; // See "All nodes:" section for details of `range` property. + // See the "All Nodes" section for details of the `range` property. + range: [number, number]; value: string; } ``` @@ -74,8 +63,45 @@ interface Token { The range indexes of all tokens and comments must not overlap with the range of other tokens and comments. -### The `Literal` node: +### The `Literal` Node The `Literal` node must have `raw` property. * `raw` (`string`) is the source code of this literal. This is the same as `code.slice(node.range[0], node.range[1])`. + +## Packaging a Custom Parser + +TODO: Add info on turning into a package + +## Example + +For a complex example of a custom parser, refer to the [`@typescript-eslint/parser`](https://github.com/typescript-eslint/typescript-eslint/tree/main/packages/parser) source code. + +A simple custom parser that logs `"foo"` to the console when it processes a node: + +```javascript +// awesome-custom-parser.js +var espree = require("espree"); +exports.parseForESLint = function(code, options) { + return { + ast: espree.parse(code, options), + services: { + foo: function() { + console.log("foo"); + } + }, + scopeManager: null, + visitorKeys: null + }; +}; + +``` + +Include the custom parser in an ESLint configuration file: + +```js +// .eslintrc.json +{ + "parser": "./path/to/awesome-custom-parser.js" +} +``` From e6f47cdb2baa5c5153abf16ead00379583003492 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter Date: Sat, 18 Feb 2023 12:38:02 -0500 Subject: [PATCH 2/8] clarify TODOs --- docs/src/extend/custom-parsers.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index 176d6ac5047..e15aeee6052 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -14,7 +14,9 @@ ESLint custom parsers let you extend ESLint to support linting new non-standard If a `parseForESLint` method is exposed on the parser, this method will be used to parse the code. Otherwise, the `parse` method will be used. Both methods should take in the source code as the first argument, and an optional configuration object as the second argument (provided as `parserOptions` in a config file). -TODO: have a simple example here w parse and parserOptions +```javascript +// TODO: have a simple example here w parse and parserOptions +``` ## `parse` Return Object From 0c81e6b5e45f7f22c19af6ff029cad4bc3e43078 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter Date: Sun, 26 Feb 2023 12:29:14 -0500 Subject: [PATCH 3/8] copy edits based on feedback --- docs/src/extend/custom-parsers.md | 52 ++++++++++++++++++++++++++++--- 1 file changed, 47 insertions(+), 5 deletions(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index e15aeee6052..f5fe7b99c75 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -12,17 +12,32 @@ ESLint custom parsers let you extend ESLint to support linting new non-standard ## Creating a Custom Parser -If a `parseForESLint` method is exposed on the parser, this method will be used to parse the code. Otherwise, the `parse` method will be used. Both methods should take in the source code as the first argument, and an optional configuration object as the second argument (provided as `parserOptions` in a config file). +A custom parser is a JavaScript object with either a `parse` or `parseForESLint` method. The `parse` method only returns the AST, whereas `parseForESLint` also returns additional values that let you customize the behavior of the parser even more. + +Both methods should take in the source code as the first argument, and an optional configuration object as the second argument, which is provided as [`parserOptions`](../use/configure/language-options#specifying-parser-options) in a configuration file. ```javascript -// TODO: have a simple example here w parse and parserOptions +// customParser.js + +const espree = require("espree"); + +// Logs the time before and after each node traversal. +function parse(code, options) { + const label = `Parsing file "${options.filePath}"`; + console.time(label); + const ast = espree.parse(code, options); + console.timeEnd(label); + return ast; // Only the AST is returned. +}; + +module.exports = { parse }; ``` ## `parse` Return Object The `parse` method should simply return the [AST](#ast-specification) object. -### `parseForESLint` Return Object +## `parseForESLint` Return Object The `parseForESLint` method should return an object that contains the required property `ast` and optional properties `services`, `scopeManager`, and `visitorKeys`. @@ -73,7 +88,33 @@ The `Literal` node must have `raw` property. ## Packaging a Custom Parser -TODO: Add info on turning into a package +To publish your custom parser to npm, perform the following: + +1. Create a custom parser following the [Creating a Custom Parser](#creating-a-custom-parser) section above. +1. Create an npm package for the custom parser. +1. In your `package.json` file, set the [`main`](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#main) field as the file that exports your custom parser. +1. Publish the npm package. + +For more information on publishing an npm package, refer to the [npm documentation](https://docs.npmjs.com/). + +Once you've published the npm package, you can use it by adding the package to your project. For example: + +```shell +npm install eslint-parser-myParser +``` + +Then add the custom parser to your ESLint configuration file with the `parser` property. For example: + +```js +// .eslintrc.js + +module.exports = { + parser: 'eslint-parser-myParser', + // ... rest of configuration +}; +``` + +To learn more about using ESLint parsers in your project, refer to [Configure a Parser](../use/configure/parser). ## Example @@ -84,7 +125,7 @@ A simple custom parser that logs `"foo"` to the console when it processes a node ```javascript // awesome-custom-parser.js var espree = require("espree"); -exports.parseForESLint = function(code, options) { +function parseForESLint(code, options) { return { ast: espree.parse(code, options), services: { @@ -97,6 +138,7 @@ exports.parseForESLint = function(code, options) { }; }; +module.exports = { parseForESLint }; ``` Include the custom parser in an ESLint configuration file: From 8ed7cf69d5d6276399fb9e3e222f54c34ce01ae0 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter Date: Sun, 26 Feb 2023 12:51:56 -0500 Subject: [PATCH 4/8] update comment copy --- docs/src/extend/custom-parsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index f5fe7b99c75..21b8729be82 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -21,7 +21,7 @@ Both methods should take in the source code as the first argument, and an option const espree = require("espree"); -// Logs the time before and after each node traversal. +// Logs the time before and after parsing each file. function parse(code, options) { const label = `Parsing file "${options.filePath}"`; console.time(label); From fc140278a7247282288709ce2f8b6538cf880d2e Mon Sep 17 00:00:00 2001 From: Ben Perlmutter <57849986+bpmutter@users.noreply.github.com> Date: Tue, 28 Feb 2023 21:16:41 -0500 Subject: [PATCH 5/8] Apply suggestions from code review Co-authored-by: Nicholas C. Zakas Co-authored-by: Nitin Kumar --- docs/src/extend/custom-parsers.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index 21b8729be82..0c3829f6552 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -12,7 +12,7 @@ ESLint custom parsers let you extend ESLint to support linting new non-standard ## Creating a Custom Parser -A custom parser is a JavaScript object with either a `parse` or `parseForESLint` method. The `parse` method only returns the AST, whereas `parseForESLint` also returns additional values that let you customize the behavior of the parser even more. +A custom parser is a JavaScript object with either a `parse` or `parseForESLint` method. The `parse` method only returns the AST, whereas `parseForESLint` also returns additional values that let the parser customize the behavior of ESLint even more. Both methods should take in the source code as the first argument, and an optional configuration object as the second argument, which is provided as [`parserOptions`](../use/configure/language-options#specifying-parser-options) in a configuration file. @@ -21,7 +21,7 @@ Both methods should take in the source code as the first argument, and an option const espree = require("espree"); -// Logs the time before and after parsing each file. +// Logs the duration it takes to parse each file. function parse(code, options) { const label = `Parsing file "${options.filePath}"`; console.time(label); @@ -43,10 +43,10 @@ The `parseForESLint` method should return an object that contains the required p * `ast` should contain the [AST](#ast-specification) object. * `services` can contain any parser-dependent services (such as type checkers for nodes). The value of the `services` property is available to rules as `context.parserServices`. Default is an empty object. -* `scopeManager` can be a [ScopeManager](./scope-manager-interface) object. Custom parsers can use customized scope analysis for experimental/enhancement syntaxes. Default is the `ScopeManager` object which is created by [eslint-scope](https://github.com/eslint/eslint-scope). - * Support for `scopeManager` was added in ESLint v4.14.0. ESLint versions which support `scopeManager` will provide an `eslintScopeManager: true` property in `parserOptions`, which can be used for feature detection. -* `visitorKeys` can be an object to customize AST traversal. The keys of the object are the type of AST nodes. Each value is an array of the property names which should be traversed. Default is [KEYS of `eslint-visitor-keys`](https://github.com/eslint/eslint-visitor-keys#evkkeys). - * Support for `visitorKeys` was added in ESLint v4.14.0. ESLint versions which support `visitorKeys` will provide an `eslintVisitorKeys: true` property in `parserOptions`, which can be used for feature detection. +* `scopeManager` can be a [ScopeManager](./scope-manager-interface) object. Custom parsers can use customized scope analysis for experimental/enhancement syntaxes. The default is the `ScopeManager` object which is created by [eslint-scope](https://github.com/eslint/eslint-scope). + * Support for `scopeManager` was added in ESLint v4.14.0. ESLint versions that support `scopeManager` will provide an `eslintScopeManager: true` property in `parserOptions`, which can be used for feature detection. +* `visitorKeys` can be an object to customize AST traversal. The keys of the object are the type of AST nodes. Each value is an array of the property names which should be traversed. The default is [KEYS of `eslint-visitor-keys`](https://github.com/eslint/eslint-visitor-keys#evkkeys). + * Support for `visitorKeys` was added in ESLint v4.14.0. ESLint versions that support `visitorKeys` will provide an `eslintVisitorKeys: true` property in `parserOptions`, which can be used for feature detection. ## AST Specification @@ -57,7 +57,7 @@ The AST that custom parsers should create is based on [ESTree](https://github.co All nodes must have `range` property. * `range` (`number[]`) is an array of two numbers. Both numbers are a 0-based index which is the position in the array of source code characters. The first is the start position of the node, the second is the end position of the node. `code.slice(node.range[0], node.range[1])` must be the text of the node. This range does not include spaces/parentheses which are around the node. -* `loc` (`SourceLocation`) must not be `null`. [The `loc` property is defined as nullable by ESTree](https://github.com/estree/estree/blob/25834f7247d44d3156030f8e8a2d07644d771fdb/es5.md#node-objects), but ESLint requires this property. `SourceLocation#source` property can be `undefined`. ESLint does not use the `SourceLocation#source` property. +* `loc` (`SourceLocation`) must not be `null`. [The `loc` property is defined as nullable by ESTree](https://github.com/estree/estree/blob/25834f7247d44d3156030f8e8a2d07644d771fdb/es5.md#node-objects), but ESLint requires this property. The `SourceLocation#source` property can be `undefined`. ESLint does not use the `SourceLocation#source` property. The `parent` property of all nodes must be rewritable. Before any rules have access to the AST, ESLint sets each node's `parent` property to its parent node while traversing. From b9026abd4fbe0b1c18c5dc5e05c525bdee22ba1c Mon Sep 17 00:00:00 2001 From: Ben Perlmutter <57849986+bpmutter@users.noreply.github.com> Date: Tue, 28 Feb 2023 21:19:07 -0500 Subject: [PATCH 6/8] Apply suggestions from code review --- docs/src/extend/custom-parsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index 0c3829f6552..de8342d456e 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -120,7 +120,7 @@ To learn more about using ESLint parsers in your project, refer to [Configure a For a complex example of a custom parser, refer to the [`@typescript-eslint/parser`](https://github.com/typescript-eslint/typescript-eslint/tree/main/packages/parser) source code. -A simple custom parser that logs `"foo"` to the console when it processes a node: +A simple custom parser that provides a `context.parserServices.foo()` method to rules. ```javascript // awesome-custom-parser.js From 6d3ae7d11c0b493707d8481a693f9cadda762a17 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter <57849986+bpmutter@users.noreply.github.com> Date: Tue, 28 Feb 2023 21:22:21 -0500 Subject: [PATCH 7/8] Apply suggestions from code review --- docs/src/extend/custom-parsers.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index de8342d456e..47c8ebe87f3 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -91,16 +91,16 @@ The `Literal` node must have `raw` property. To publish your custom parser to npm, perform the following: 1. Create a custom parser following the [Creating a Custom Parser](#creating-a-custom-parser) section above. -1. Create an npm package for the custom parser. +1. [Create an npm package](https://docs.npmjs.com/creating-node-js-modules) for the custom parser. 1. In your `package.json` file, set the [`main`](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#main) field as the file that exports your custom parser. -1. Publish the npm package. +1. [Publish the npm package.](https://docs.npmjs.com/creating-and-publishing-unscoped-public-packages) For more information on publishing an npm package, refer to the [npm documentation](https://docs.npmjs.com/). Once you've published the npm package, you can use it by adding the package to your project. For example: ```shell -npm install eslint-parser-myParser +npm install eslint-parser-myparser --save-dev ``` Then add the custom parser to your ESLint configuration file with the `parser` property. For example: From 3f1dc78e35b1eaddfc2ecf103af9c0d43c20fee6 Mon Sep 17 00:00:00 2001 From: Ben Perlmutter <57849986+bpmutter@users.noreply.github.com> Date: Sun, 5 Mar 2023 13:58:01 -0500 Subject: [PATCH 8/8] Update docs/src/extend/custom-parsers.md Co-authored-by: Francesco Trotta --- docs/src/extend/custom-parsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/extend/custom-parsers.md b/docs/src/extend/custom-parsers.md index 47c8ebe87f3..388f54b726b 100644 --- a/docs/src/extend/custom-parsers.md +++ b/docs/src/extend/custom-parsers.md @@ -109,7 +109,7 @@ Then add the custom parser to your ESLint configuration file with the `parser` p // .eslintrc.js module.exports = { - parser: 'eslint-parser-myParser', + parser: 'eslint-parser-myparser', // ... rest of configuration }; ```