Enhancement: Add the ability to skip valid or invalid tests

[azat-io]

Before You File a Proposal Please Confirm You Have Done The Following...

• I have searched for related issues and found none that match my proposal.
• I have searched the current rule list and found no rules that match my proposal.
• I have read the FAQ and my problem is not listed.

Relevant Package

rule-tester

My proposal is suitable for this project

• I believe my proposal would be useful to the broader TypeScript community (meaning it is not a niche proposal).

Description

I would like to be able to skip valid or invalid scenarios in some tests. In some cases, such examples are simply not needed.

Fail

import noLetRule from '../rule/no-let'

ruleTester.run(
  `disallow to use let`,
  rule,
  {
    valid: [],
    invalid: [`let a = 1`],
  },
)

This test is currently crashing with an error: Error: No test found in suite valid.

[JoshuaKGoldberg]

Looking through https://eslint.org/docs/latest/integrate/nodejs-api, it looks like the core ESLint RuleTester does support valid: []. And when I try it out locally, it passes with 0 passing.

In fact, ours does too! At least with Mocha. Repro here: https://github.com/JoshuaKGoldberg/repros/tree/eslint-rule-tester-empty-arrays

npx mocha *.test.js

0 passing

So I'm betting it's your specific test runner that's giving you grief. Both the core RuleTester and our version are pretty thin wrappers over test runner APIs.

To accept this issue, I think we'd want to see a reproduction that shows where the core ESLint RuleTester supports something we don't. I.e. a case where you can run with valid: [] in them but not us.

[bradzacher]

Yeah I wouldn't want us to break from eslint here - the intent with RuleTester is to keep it as close to core as possible - just adding new features (eg dependency filtering and snapshot testing) rather than changing the internal logic.

I believe that the logic I used was exactly the same as the underlying RuleTester.
So I'd agree that if this doesn't work upstream, then we'd want to fix it upstream first so that the two remain aligned.

[JoshuaKGoldberg]

Maybe relevant: eslint/eslint#17455

[snitin315]

We have accepted this as an enhancement in ESLint and it will be implemented soon.

[JoshuaKGoldberg]

Great, thanks! Once the ESLint PR is approved we can take one in here. Just making sure our implementation doesn't differ too much.

Ref: eslint/eslint#17455 -> eslint/eslint#17475

[azat-io]

Looks like this PR was merged: eslint/eslint#17475