Bug: config files and ignores elements are not being validated

[fasttime]

Environment

Node version: v20.5.0
npm version: v9.8.0
Local ESLint version: v8.46.0 (Currently used)
Global ESLint version: Not found
Operating System: darwin 22.5.0

What parser are you using?

Default (Espree)

What did you do?

I run eslint . with a config that contains an invalid element in the files array, e.g.:

Configuration

export default [
    {
        files: [
            "**/*.js",
            undefined
        ]
    }
];

What did you expect to happen?

There should be an error message indicating that files contains an invalid value.

What actually happened?

A generic error message:

Oops! Something went wrong! :(

ESLint: 8.46.0

TypeError: Unexpected matcher type undefined.

For comparison, here is the error message printed by an eslintrc config with overrides: [{ files: ["**/*.js", undefined] }]:

Oops! Something went wrong! :(

ESLint: 8.46.0

Error: ESLint configuration in .eslintrc.js is invalid:
        - Property "overrides[0].files" is the wrong type (expected string but got `["**/*.js",null]`).
        - Property "overrides[0].files[1]" is the wrong type (expected string but got `undefined`).
        - "overrides[0].files" should match exactly one schema in oneOf. Value: ["**/*.js",null].

Link to Minimal Reproducible Example

https://stackblitz.com/edit/stackblitz-starters-vmehy7?file=eslint.config.js&view=editor

Participation

• I am willing to submit a pull request for this issue.

Additional comments

A similar problem occurs when an ignores pattern is invalid. For example, when running eslint . with this config:

export default [
    {
        files: ["**/*.js"],
        ignores: [1],
    }
];

The error message looks like this:

Oops! Something went wrong! :(

ESLint: 8.46.0

TypeError: matcher.startsWith is not a function

[nzakas]

Huh, that is very strange. Both files and ignores are validated by the default ConfigArray, and it looks like the schema should catch this and throw a useful error message:
https://github.com/humanwhocodes/config-array/blob/main/src/base-schema.js

But for some reason the error is being thrown here instead:
https://github.com/humanwhocodes/config-array/blob/main/src/config-array.js#L285

So, something about the order of operations looks off here.

@fasttime if you'd like to explore this, please feel free. Otherwise, let me know and I can dig into it.

[fasttime]

Thanks for linking to the relevant code @nzakas. I can look into this.

[fasttime]

Okay, after digging for a few hours into the code of config-array, I can confirm that the config is being validated too late in the program flow. The files and ignores of the single config elements are being accessed in different ways before the config is validated for the first time.

The problematic code seems to be in the getConfig method in config-array. In this method, the first access to the elements of the ignores arrays occurs when isDirectoryIngored is called in line 704: isDirectoryIngored invokes the this.ignores getter to pick the global ignores out of the config array, but at this time, no validation of any kind has been done yet.

The config elements are validated for the first time when they are merged (line 835). That is when the merge implementation calls validate on the config objects to be merged.

So the fix would probably consist in moving the validation up in the getConfig code, not after the call to isDirectoryIngored. Another option is to prevalidate just the files and ignores (the only properties used in config-array itself), and leave the rest of the validation where it is. I will shortly create a PR so we can better discuss the details and find the best approach.