Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Custom Processors cleanup and expansion #16838

Merged
merged 11 commits into from Mar 23, 2023
93 changes: 76 additions & 17 deletions docs/src/extend/custom-processors.md
Expand Up @@ -7,11 +7,12 @@ eleventyNavigation:
order: 5

---
You can also create custom processors that tell ESLint how to process files other than JavaScript.

You can also create custom processors that tell ESLint how to process files other than standard JavaScript. For example, you could write a custom processor to extract and process JavaScript from Markdown files ([eslint-plugin-markdown](https://www.npmjs.com/package/eslint-plugin-markdown) includes a custom processor for this).

## Custom Processor Specification

In order to create a processor, the object that is exported from your module has to conform to the following interface:
In order to create a custom processor, the object exported from your module has to conform to the following interface:

```js
module.exports = {
Expand Down Expand Up @@ -46,29 +47,66 @@ module.exports = {

**The `preprocess` method** takes the file contents and filename as arguments, and returns an array of code blocks to lint. The code blocks will be linted separately but still be registered to the filename.

A code block has two properties `text` and `filename`; the `text` property is the content of the block and the `filename` property is the name of the block. Name of the block can be anything, but should include the file extension, that would tell the linter how to process the current block. The linter will check [`--ext` CLI option](../use/command-line-interface#--ext) to see if the current block should be linted, and resolve `overrides` configs to check how to process the current block.
A code block has two properties `text` and `filename`. The `text` property is the content of the block and the `filename` property is the name of the block. The name of the block can be anything, but should include the file extension, which tells the linter how to process the current block. The linter checks the [`--ext` CLI option](../use/command-line-interface#--ext) to see if the current block should be linted and resolves `overrides` configs to check how to process the current block.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last sentence here about checking --ext describes the pre-ESLint v6 behavior. That still happens, but it also checks extensions in overrides patterns now. That's described in https://eslint.org/docs/latest/use/configure/plugins#specify-a-processor.

If updating this is out of scope for the change you're intending here, feel free to disregard.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! I couldn't remember if we had removed that functionality or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't fully understand. would one of you mind providing a bit more info or a draft of what the copy should say here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you pass ESLint a folder on the command line, like eslint src/, ESLint walks the file system to find files to lint. By default, it finds .js, but you can override that in two different ways:

  1. By passing the --ext flag with different suffixes to find
  2. By specifying them in your config file through overrides patterns. Basically, each object in overrides has a files key that specifies glob patterns to match. These will be used to match inside of src/ in the previous example.

So --ext and overrides.files determines which blocks will be linted, and the other settings in overrides determine which configuration to apply to the blocks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for clarifying!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To expand on what nzakas said and give an example, with the config below, to use the Markdown plugin prior to ESLint v6, one had to pass --ext .js,.md, or ESLint would only look at .js files. Now, ESLint will automatically see there's an overrides pattern for .md files and include them in the file traversal. One can still pass --ext to override that automatic behavior, but it in normal cases ESLint will do the right thing automatically.

// .eslintrc.js
module.exports = {
    plugins: ["markdown"],
    overrides: [
        {
            files: ["**/*.md"],
            processor: "markdown/markdown"
        }
    ]
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we think any further documentation is needed on this beyond what's in this PR?

my inclination is no, we do not. but maybe i'm not fullying grokking this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need anything further.


It's up to the plugin to decide if it needs to return just one part, or multiple pieces. For example in the case of processing `.html` files, you might want to return just one item in the array by combining all scripts, but for `.md` file where each JavaScript block might be independent, you can return multiple items.
It's up to the plugin to decide if it needs to return just one part of the non-JavaScript file or multiple pieces. For example in the case of processing `.html` files, you might want to return just one item in the array by combining all scripts. However, for `.md` files, you can return multiple items because each JavaScript block might be independent.

**The `postprocess` method** takes a two-dimensional array of arrays of lint messages and the filename. Each item in the input array corresponds to the part that was returned from the `preprocess` method. The `postprocess` method must adjust the locations of all errors to correspond to locations in the original, unprocessed code, and aggregate them into a single flat array and return it.

Reported problems have the following location information:
Reported problems have the following location information in each lint message:

```typescript
{
line: number,
column: number,
type LintMessage = {

/// The 1-based line number where the message occurs.
line: number;

/// The 1-based column number where the message occurs.
column: number;
bpmutter marked this conversation as resolved.
Show resolved Hide resolved

/// The 1-based line number of the end location.
endLine: number;

/// The 1-based column number of the end location.
endColumn: number;

/// If `true`, this is a fatal error.
fatal: boolean;

/// Information for an autofix.
fix: Fix;

/// The error message.
message: string;

/// The ID of the rule which generated the message, or `null` if not applicable.
ruleId: string | null;

/// The severity of the message.
severity: 0 | 1 | 2;

/// Information for suggestions.
suggestions?: Suggestion[];
};

type Fix = {
range: [number, number];
text: string;
}

endLine?: number,
endColumn?: number
type Suggestion = {
desc?: string;
messageId?: string;
fix: Fix;
}

```

By default, ESLint will not perform autofixes when a processor is used, even when the `--fix` flag is enabled on the command line. To allow ESLint to autofix code when using your processor, you should take the following additional steps:
By default, ESLint does not perform autofixes when a custom processor is used, even when the `--fix` flag is enabled on the command line. To allow ESLint to autofix code when using your processor, you should take the following additional steps:

1. Update the `postprocess` method to additionally transform the `fix` property of reported problems. All autofixable problems will have a `fix` property, which is an object with the following schema:
1. Update the `postprocess` method to additionally transform the `fix` property of reported problems. All autofixable problems have a `fix` property, which is an object with the following schema:

```js
```typescript
{
range: [number, number],
text: string
Expand All @@ -81,8 +119,7 @@ By default, ESLint will not perform autofixes when a processor is used, even whe

2. Add a `supportsAutofix: true` property to the processor.

You can have both rules and processors in a single plugin. You can also have multiple processors in one plugin.
To support multiple extensions, add each one to the `processors` element and point them to the same object.
You can have both rules and custom processors in a single plugin. You can also have multiple processors in one plugin. To support multiple extensions, add each one to the `processors` element and point them to the same object.

## Specifying Processor in Config Files

Expand All @@ -102,19 +139,41 @@ See [Specify a Processor](../use/configure/plugins#specify-a-processor) in the P

## File Extension-named Processor

If a processor name starts with `.`, ESLint handles the processor as a **file extension-named processor** especially and applies the processor to the kind of files automatically. People don't need to specify the file extension-named processors in their config files.
If a custom processor name starts with `.`, ESLint handles the processor as a **file extension-named processor**. ESLint applies the processor to files with that filename extension automatically. Users don't need to specify the file extension-named processors in their config files.
bpmutter marked this conversation as resolved.
Show resolved Hide resolved

For example:

```js
module.exports = {
processors: {
// This processor will be applied to `*.md` files automatically.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@btmills just double-checking: is this still true? I seem to recall we might have removed automatically applied processors? Or maybe I'm just imagining it?

Copy link
Member

@btmills btmills Mar 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's at least still true under .eslintrc (implementation, tests). I don't know for sure that it's not part of flat config, but the tests all explicitly specify processor, and I can't find evidence of the back compat logic being carried over.

Should these docs discourage building file extension-named processors?

// Also, people can use this processor as "plugin-id/.md" explicitly.
// Also, you can use this processor as "plugin-id/.md" explicitly.
".md": {
preprocess(text, filename) { /* ... */ },
postprocess(messageLists, filename) { /* ... */ }
}
// This processor will not be applied to any files automatically.
// To use this processor, you must explicitly specify it
// in your configuration as "plugin-id/markdown".
"markdown": {
preprocess(text, filename) { /* ... */ },
postprocess(messageLists, filename) { /* ... */ }
}
}
}
```

You can also use the same custom processor with multiple filename extensions. The following example shows using the same processor for both `.md` and `.mdx` files:

```js
bpmutter marked this conversation as resolved.
Show resolved Hide resolved
const myCustomProcessor = { /* processor methods */ };

module.exports = {
bpmutter marked this conversation as resolved.
Show resolved Hide resolved
// The same custom processor is applied to both
// `.md` and `.mdx` files.
processors: {
".md": myCustomProcessor,
".mdx": myCustomProcessor
}
}
```