Change Request: Ensure that examples in rule docs are parsable

[fasttime]

ESLint version

v8.50.0

What problem do you want to solve?

Some of the correct/incorrect example code blocks in rule docs fail to parse because of various reasons.
Other code blocks expect certain parser options (e.g. sourceType: "module") in order to work as intended, but these options are not always explicitly encoded along with the correct/incorrect tag.
In both cases, the outcome is the same: when such a code block is opened in the Playground from the rule docs page, a visitor ends up seeing a syntax error instead of the expected rule validation.

I run a script to find these unparsable code blocks, and it found 186. The output also includes other problems like rules without a correct/incorrect code block:

File docs/src/rules/arrow-body-style.md:
  - code block #2 on line 42 cannot be parsed: Identifier 'foo' has already been declared
  - code block #3 on line 60 cannot be parsed: Identifier 'foo' has already been declared
  - code block #4 on line 83 cannot be parsed: Identifier 'foo' has already been declared
  - code block #5 on line 117 cannot be parsed: Identifier 'foo' has already been declared
  - code block #6 on line 130 cannot be parsed: Identifier 'foo' has already been declared
  - code block #7 on line 146 cannot be parsed: Identifier 'foo' has already been declared
  - code block #8 on line 165 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/camelcase.md:
  - code block #1 on line 31 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 75 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #10 on line 238 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #11 on line 252 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #12 on line 266 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/class-methods-use-this.md:
  - code block #2 on line 78 cannot be parsed: Identifier 'A' has already been declared

File docs/src/rules/comma-spacing.md:
  - code block #3 on line 91 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/constructor-super.md:
  - code block #1 on line 19 cannot be parsed: super() call outside constructor of a subclass
  - code block #2 on line 51 cannot be parsed: Identifier 'A' has already been declared

File docs/src/rules/eol-last.md:
  - code block #2 on line 38 cannot be parsed: Expecting Unicode escape sequence \uXXXX

File docs/src/rules/func-names.md:
  - code block #1 on line 45 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 67 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 93 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 111 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/generator-star.md:
  - no correct code blocks
  - no incorrect code blocks

File docs/src/rules/global-strict.md:
  - no correct code blocks
  - no incorrect code blocks

File docs/src/rules/id-blacklist.md:
  - no correct code blocks
  - no incorrect code blocks

File docs/src/rules/id-denylist.md:
  - code block #1 on line 44 cannot be parsed: Unexpected token }
  - code block #2 on line 84 cannot be parsed: Unexpected token }

File docs/src/rules/id-length.md:
  - code block #1 on line 28 cannot be parsed: Identifier 'x' has already been declared
  - code block #2 on line 63 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #3 on line 116 cannot be parsed: Identifier 'x' has already been declared
  - code block #8 on line 247 cannot be parsed: Identifier 'x' has already been declared

File docs/src/rules/id-match.md:
  - code block #1 on line 32 cannot be parsed: Identifier 'myClass' has already been declared
  - code block #2 on line 64 cannot be parsed: Identifier 'myClass' has already been declared
  - code block #4 on line 121 cannot be parsed: Identifier 'myClass' has already been declared

File docs/src/rules/indent-legacy.md:
  - code block #7 on line 198 cannot be parsed: Identifier 'a' has already been declared
  - code block #8 on line 219 cannot be parsed: Identifier 'a' has already been declared
  - code block #9 on line 240 cannot be parsed: Identifier 'a' has already been declared
  - code block #10 on line 261 cannot be parsed: Identifier 'a' has already been declared

File docs/src/rules/indent.md:
  - code block #9 on line 242 cannot be parsed: Identifier 'a' has already been declared
  - code block #10 on line 263 cannot be parsed: Identifier 'a' has already been declared
  - code block #11 on line 284 cannot be parsed: Identifier 'a' has already been declared
  - code block #12 on line 305 cannot be parsed: Identifier 'a' has already been declared
  - code block #13 on line 326 cannot be parsed: Identifier 'a' has already been declared
  - code block #14 on line 347 cannot be parsed: Identifier 'a' has already been declared
  - code block #44 on line 852 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #45 on line 873 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #46 on line 888 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/jsx-quotes.md:
  - code block #2 on line 52 cannot be parsed: Adjacent JSX elements must be wrapped in an enclosing tag
  - code block #4 on line 79 cannot be parsed: Adjacent JSX elements must be wrapped in an enclosing tag

File docs/src/rules/keyword-spacing.md:
  - code block #2 on line 61 cannot be parsed: Identifier 'a' has already been declared
  - code block #6 on line 176 cannot be parsed: Identifier 'a' has already been declared
  - code block #9 on line 285 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/lines-around-comment.md:
  - code block #29 on line 660 cannot be parsed: Unexpected token ,

File docs/src/rules/lines-between-class-members.md:
  - code block #4 on line 91 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #5 on line 115 cannot be parsed: Identifier 'Foo' has already been declared

File docs/src/rules/max-len.md:
  - code block #3 on line 72 cannot be parsed: Expecting Unicode escape sequence \uXXXX
  - code block #4 on line 84 cannot be parsed: Expecting Unicode escape sequence \uXXXX

File docs/src/rules/max-params.md:
  - code block #1 on line 39 cannot be parsed: Identifier 'foo' has already been declared
  - code block #2 on line 58 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/max-statements.md:
  - code block #1 on line 45 cannot be parsed: Identifier 'foo' has already been declared
  - code block #2 on line 86 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/newline-after-var.md:
  - code block #1 on line 45 cannot be parsed: Identifier 'greet' has already been declared
  - code block #2 on line 73 cannot be parsed: Identifier 'greet' has already been declared
  - code block #3 on line 107 cannot be parsed: Identifier 'greet' has already been declared
  - code block #4 on line 139 cannot be parsed: Identifier 'greet' has already been declared

File docs/src/rules/no-arrow-condition.md:
  - no correct code blocks

File docs/src/rules/no-delete-var.md:
  - no correct code blocks

File docs/src/rules/no-dupe-class-members.md:
  - code block #1 on line 32 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #2 on line 67 cannot be parsed: Identifier 'Foo' has already been declared

File docs/src/rules/no-duplicate-imports.md:
  - code block #1 on line 23 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 37 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 50 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 70 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #5 on line 84 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #6 on line 98 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-empty-static-block.md:
  - code block #2 on line 34 cannot be parsed: Identifier 'Foo' has already been declared

File docs/src/rules/no-extra-parens.md:
  - code block #7 on line 217 cannot be parsed: Identifier 'Component' has already been declared
  - code block #8 on line 233 cannot be parsed: Identifier 'Component' has already been declared
  - code block #9 on line 245 cannot be parsed: Identifier 'Component' has already been declared
  - code block #10 on line 265 cannot be parsed: Identifier 'Component' has already been declared
  - code block #11 on line 285 cannot be parsed: Identifier 'Component' has already been declared

File docs/src/rules/no-import-assign.md:
  - code block #1 on line 18 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 38 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-invalid-this.md:
  - code block #2 on line 100 cannot be parsed: Identifier 'Foo' has already been declared

File docs/src/rules/no-irregular-whitespace.md:
  - code block #1 on line 72 cannot be parsed: Unexpected character '᠎'
  - code block #6 on line 200 cannot be parsed: Unexpected token `}`. Did you mean `}` or `{"}"}`?

File docs/src/rules/no-loss-of-precision.md:
  - code block #1 on line 16 cannot be parsed: Identifier 'x' has already been declared
  - code block #2 on line 33 cannot be parsed: Identifier 'x' has already been declared

File docs/src/rules/no-misleading-character-class.md:
  - code block #1 on line 58 cannot be parsed: Unexpected token ^
  - code block #2 on line 75 cannot be parsed: Unexpected token ^

File docs/src/rules/no-multi-assign.md:
  - code block #1 on line 25 cannot be parsed: Identifier 'a' has already been declared
  - code block #2 on line 49 cannot be parsed: Identifier 'a' has already been declared

File docs/src/rules/no-multi-spaces.md:
  - code block #10 on line 201 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-multiple-empty-lines.md:
  - code block #5 on line 93 cannot be parsed: Unexpected character '⏎'
  - code block #6 on line 110 cannot be parsed: Unexpected character '⏎'

File docs/src/rules/no-plusplus.md:
  - code block #1 on line 34 cannot be parsed: 'return' outside of function
  - code block #2 on line 54 cannot be parsed: 'return' outside of function

File docs/src/rules/no-restricted-exports.md:
  - code block #1 on line 31 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 63 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 99 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 111 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #5 on line 123 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #6 on line 141 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #7 on line 157 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #8 on line 173 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #9 on line 188 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #10 on line 202 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-restricted-globals.md:
  - code block #2 on line 73 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-restricted-imports.md:
  - code block #7 on line 197 cannot be parsed: Identifier 'AllowedObject' has already been declared

File docs/src/rules/no-script-url.md:
  - no correct code blocks

File docs/src/rules/no-self-compare.md:
  - no correct code blocks

File docs/src/rules/no-sequences.md:
  - code block #3 on line 84 cannot be parsed: Identifier 'foo' has already been declared
  - code block #4 on line 99 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/no-shadow.md:
  - code block #7 on line 182 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-tabs.md:
  - code block #1 on line 15 cannot be parsed: Expecting Unicode escape sequence \uXXXX
  - code block #3 on line 57 cannot be parsed: Expecting Unicode escape sequence \uXXXX

File docs/src/rules/no-this-before-super.md:
  - code block #1 on line 21 cannot be parsed: Identifier 'A' has already been declared
  - code block #2 on line 59 cannot be parsed: Identifier 'A' has already been declared

File docs/src/rules/no-underscore-dangle.md:
  - code block #2 on line 38 cannot be parsed: Identifier 'foo' has already been declared
  - code block #5 on line 102 cannot be parsed: 'super' keyword outside a method
  - code block #7 on line 132 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #8 on line 160 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #9 on line 192 cannot be parsed: Identifier '_bar' has already been declared
  - code block #10 on line 208 cannot be parsed: Identifier 'foo' has already been declared
  - code block #11 on line 221 cannot be parsed: Identifier 'foo' has already been declared
  - code block #12 on line 236 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/no-unexpected-multiline.md:
  - code block #1 on line 29 cannot be parsed: Identifier 'x' has already been declared

File docs/src/rules/no-unused-private-class-members.md:
  - code block #1 on line 18 cannot be parsed: Identifier 'Foo' has already been declared
  - code block #2 on line 55 cannot be parsed: Identifier 'Foo' has already been declared

File docs/src/rules/no-use-before-define.md:
  - code block #1 on line 17 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 71 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #8 on line 315 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #9 on line 335 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/no-useless-constructor.md:
  - code block #2 on line 52 cannot be parsed: Identifier 'A' has already been declared

File docs/src/rules/no-useless-rename.md:
  - code block #1 on line 58 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 79 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 110 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 122 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #5 on line 135 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/object-curly-newline.md:
  - code block #13 on line 550 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #14 on line 575 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/object-curly-spacing.md:
  - code block #1 on line 55 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 72 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 97 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 118 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/one-var-declaration-per-line.md:
  - code block #1 on line 41 cannot be parsed: Identifier 'a' has already been declared
  - code block #2 on line 57 cannot be parsed: Identifier 'a' has already been declared
  - code block #3 on line 78 cannot be parsed: Identifier 'a' has already been declared
  - code block #4 on line 95 cannot be parsed: Identifier 'a' has already been declared

File docs/src/rules/one-var.md:
  - code block #3 on line 190 cannot be parsed: Identifier 'bar' has already been declared
  - code block #10 on line 412 cannot be parsed: Identifier 'bar' has already been declared

File docs/src/rules/prefer-const.md:
  - code block #1 on line 21 cannot be parsed: Identifier 'a' has already been declared
  - code block #2 on line 57 cannot be parsed: Identifier 'a' has already been declared
  - code block #4 on line 162 cannot be parsed: Identifier 'a' has already been declared

File docs/src/rules/prefer-destructuring.md:
  - code block #2 on line 52 cannot be parsed: Identifier 'foo' has already been declared

File docs/src/rules/require-unicode-regexp.md:
  - code block #2 on line 78 cannot be parsed: Identifier 'f' has already been declared

File docs/src/rules/rest-spread-spacing.md:
  - code block #1 on line 83 cannot be parsed: Unexpected token ...
  - code block #2 on line 100 cannot be parsed: Unexpected token ...
  - code block #3 on line 125 cannot be parsed: Unexpected token ...
  - code block #4 on line 142 cannot be parsed: Unexpected token ...

File docs/src/rules/semi.md:
  - code block #4 on line 164 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #6 on line 206 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #7 on line 244 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #8 on line 259 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/sort-imports.md:
  - code block #1 on line 74 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #2 on line 106 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #3 on line 145 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #4 on line 158 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #5 on line 178 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #6 on line 190 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #7 on line 200 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #8 on line 218 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #9 on line 229 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #10 on line 253 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #11 on line 265 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #12 on line 278 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #13 on line 300 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #14 on line 314 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #15 on line 327 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'
  - code block #16 on line 340 cannot be parsed: 'import' and 'export' may appear only with 'sourceType: module'

File docs/src/rules/sort-keys.md:
  - code block #1 on line 18 cannot be parsed: Identifier 'obj' has already been declared
  - code block #2 on line 44 cannot be parsed: Identifier 'obj' has already been declared
  - code block #3 on line 115 cannot be parsed: Identifier 'obj' has already been declared
  - code block #4 on line 135 cannot be parsed: Identifier 'obj' has already been declared
  - code block #5 on line 157 cannot be parsed: Identifier 'obj' has already been declared
  - code block #6 on line 171 cannot be parsed: Identifier 'obj' has already been declared
  - code block #9 on line 215 cannot be parsed: Identifier 'obj' has already been declared
  - code block #10 on line 243 cannot be parsed: Identifier 'obj' has already been declared
  - code block #12 on line 315 cannot be parsed: Identifier 'obj' has already been declared

File docs/src/rules/space-before-keywords.md:
  - code block #1 on line 46 cannot be parsed: Identifier 'foo' has already been declared
  - code block #2 on line 69 cannot be parsed: Unexpected token onClick

File docs/src/rules/space-infix-ops.md:
  - code block #1 on line 44 cannot be parsed: Identifier 'a' has already been declared
  - code block #2 on line 69 cannot be parsed: Identifier 'a' has already been declared

File docs/src/rules/strict.md:
  - code block #7 on line 173 cannot be parsed: Illegal 'use strict' directive in function with non-simple parameter list

File docs/src/rules/yield-star-spacing.md:
  - no incorrect code blocks

Note that at most one syntax error per code block is reported.

For reference, this is what my script looks like:

check-rule-examples.js

// Put this file into the eslint project folder and run it with `node`.

"use strict";

const { parse } = require("espree");
const { promises: { readFile } } = require("fs");
const glob = require("glob");
const { promisify } = require("util");

/**
 * Tries to parse a specified JavaScript code.
 * @param {string} code The JavaScript code to parse.
 * @param {ParserOptions} parserOptions Explicitly specified parser options.
 * @returns {string|undefined} An error message if the code cannot be parsed, or undefined.
 */
function tryParse(code, parserOptions) {
    try {
        parse(code, { ecmaVersion: "latest", ...parserOptions });
    } catch ({ message }) {
        return message;
    }
    return void 0;
}

/**
 * Checks a rule docs file for possible problems.
 * @param {string} file The path of a rule docs file to be checked.
 * @returns {Promise<string[]>} A list of warnings found for the file.
 */
async function checkFile(file) {
    const text = await readFile(file, "utf-8");
    const warnings = [];
    const blockMatches = text.matchAll(/^:::(?<heading>.*?)$\s*^(?<codeBlock>.*?)$\s*^:::$/gmsu);
    let anyCorrectMatches = false;
    let anyIncorrectMatches = false;
    let count = 0;

    for (const blockMatch of blockMatches) {
        const blockNumber = ++count;
        let currentBlock = {
            toString() {
                const lineNumber = text.slice(0, blockMatch.index).match(/\n/gu).length + 1;

                return (currentBlock = `code block #${blockNumber} on line ${lineNumber}`);
            }
        };
        const { heading, codeBlock } = blockMatch.groups;
        const { typeTag, parserOptionsJSON } =
        /^\s*(?<typeTag>\S*)(?:\s+(?<parserOptionsJSON>.*?)\s*)?$/u.exec(heading).groups;

        switch (typeTag) {
            case "correct":
                anyCorrectMatches = true;
                break;
            case "incorrect":
                anyIncorrectMatches = true;
                break;
            default:
                warnings.push(`${currentBlock} has wrong type tag '${typeTag}'`);
                break;
        }
        const codeBlockMatch = /^```(?<langTag>.*?)\n(?<code>.*)\n```$/su.exec(codeBlock);

        if (codeBlockMatch) {
            const { code, langTag } = codeBlockMatch.groups;

            if (!["javascript", "js", "jsx"].includes(langTag)) {
                warnings.push(`${currentBlock} has unexpected language tag '${langTag}'`);
            }
            const parserOptions = parserOptionsJSON ? JSON.parse(parserOptionsJSON) : { };
            const message = tryParse(code, parserOptions);

            if (message) {
                warnings.push(`${currentBlock} cannot be parsed: ${message}`);
            }
        } else {
            warnings.push(`${currentBlock} has no fenced code block`);
        }
    }
    if (!anyCorrectMatches) {
        warnings.push("no correct code blocks");
    }
    if (!anyIncorrectMatches) {
        warnings.push("no incorrect code blocks");
    }
    return warnings;
}

(async function main() {
    const files = await promisify(glob)("docs/src/rules/*.md");
    const warningsList = await Promise.all(files.map(checkFile));

    warningsList.forEach((warnings, index) => {
        if (warnings.length) {
            console.log( // eslint-disable-line no-console -- not for release
                `File ${files[index]}:\n${warnings.map(warning => `  - ${warning}\n`).join("")}`
            );
        }
    });
}());

What do you think is the correct solution?

I would suggest adding a check at build time to make sure that code blocks in rule docs contain no syntax errors.
Before doing so, unparsable code blocks should be fixed manually.

Most of the times, the appropriate fix is inferred directly from the error message. Here is a summary of error messages and their possible fixes without warranty:

• Identifier 'x' has already been declared
  Rename the identifier to make it unique. Other options are splitting the code blocks or inserting each individual example into a block statement { ... }.
• 'import' and 'export' may appear only with 'sourceType: module'
  Add parser option "sourceType": "module".
• Adjacent JSX elements must be wrapped in an enclosing tag
  Add a semicolon after each example. Other good solutions exist.
• super() call outside constructor of a subclass
  Move the call into a subclass constructor.
• 'super' keyword outside a method
  Move the call into a subclass method.
• Expecting Unicode escape sequence \uXXXX
  This typically happens when an unsyntactic escape sequence is inserted in place of a valid character for readability, for example \t for a tab.
• 'return' outside of function
  Move the return statement into a function or add parser option "sourceType": "commonjs".
• Illegal 'use strict' directive in function with non-simple parameter list
  Add parser option "ecmaVersion": 6
• Unexpected character/Unexpected token
  This requires looking at the code. The fix could be adding a parser option, inserting a missing semicolon, fixing a typo, replacing unsyntactic pseudocode with real code, etc.

Participation

• I am willing to submit a pull request for this change.

Additional comments

Some of the reported problems concern removed or deprecated rules, and the "Open in Playground" button works for those rules as well. I was thinking about excluding removed and deprecated rules from code block validation and disabling the Playground buttons, but that could be actually more effort than just fixing all code blocks, so not sure.

[nzakas]

Wow, very interesting. Thanks for doing this. I agree, having a check for parseable code would be a good idea. I'm curious, when you say "at build time", are you thinking that would be a lint check on PRs? Run during lint-staged? Literally run as part of the Eleventy build?

And how are you envisioning setting sourceType or ecmaVersion for such a check?

[fasttime]

I'm curious, when you say "at build time", are you thinking that would be a lint check on PRs? Run during lint-staged? Literally run as part of the Eleventy build?

I was thinking of a new step in the CI workflow, in the Verify Files job, something like node Makefile checkRuleExamples, but I'm open to other ideas. An additional check in lint-staged would be also nice to have. With that, contributors would notice problems in their examples and would be required to fix them before a PR is submitted.

And how are you envisioning setting sourceType or ecmaVersion for such a check?

To pass parser options to the playground we are using JSON after the ::: correct or ::: incorrect tag. I think this format was introduced in #17306. For example in no-restricted-import, where code examples contain import statements, we have:

eslint/docs/src/rules/no-restricted-imports.md

Line 271 in 89f3225

::: correct { "sourceType": "module" }

I think we should use the same information to determine parser options for the syntax check. If a code example cannot be parsed in the Playground because it's missing some parser options, adding the options after the tag would fix both the Playground link and the syntax check.

[nzakas]

Sounds good to me. Let's do it. 👍

[fasttime]

Thanks for accepting @nzakas! Any suggestion about how to handle examples in removed rules, e.g. space-before-keywords? They can't be tested in the Playground anyway, so I was thinking to add a check in the Eleventy build to remove the "Open in Playground" button for those examples, and the same kind of check to exclude them from syntax validation. Or if we want to keep the buttons we could just fix the examples to enure that they are parsable like the other ones, this would be a bit less maintenance burden.

[nzakas]

I think it makes sense to just omit the removed rules from "Open in Playground", although it would still be nice to make sure they are parseable. I'd prefer not to have syntax errors anywhere in the docs.

And another thing to consider: What if we made sourceType: module the default for all code examples and then we could override when necessary? Once we switch to flat config, sourceType: module will be the default, so it might be nice to prepare for that.

[mdjermanovic]

And another thing to consider: What if we made sourceType: module the default for all code examples and then we could override when necessary? Once we switch to flat config, sourceType: module will be the default, so it might be nice to prepare for that.

👍

I was going to suggest the same. Since the current default in the playground is script, we can set module here if source type wasn't specified on the example (I think we have some examples where we'll need sourceType: script).

[fasttime]

Thanks guys! Great feedback, and I like all suggestions very much. To summarize the discussion, here is the list of todos:

• Use "sourceType": "module" as a default in rule examples.
• Manually fix rule examples that cannot be parsed.
• Check that all rule examples can be parsed in the CI workflow.
• Check that rule examples in modified docs files can be parsed in lint-staged.
• Remove the "Open in Playground" button from examples of removed rules.

If this is correct, I'm ready to work on it. I'm planning to split the changes into multiple PRs so they'll be easier to review. In case someone else would like to do part of the work, just let me know here so we can coordinate.