Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: use uvu to run rule tests for better reports #257

Merged
merged 6 commits into from Oct 20, 2021

Conversation

timdeschryver
Copy link
Owner

@timdeschryver timdeschryver commented Oct 5, 2021

Currently, running the rule tests is not friendly because it abruptly stops when a rule test fails.
It also doesn't show why it fails.

In this PR, we're going to resolve this because it can waste some time and it was on my todo list 😅

The trick is to run the tests in an uvu context, with it uvu will:

  • gracefully handle exceptions and continue to run other tests
  • show the exception message (formatted) so we can clearly see what's wrong

As a fix, we got two options and I would like to hear your opinion @rafaelss95 before I migrate the other rules.

We could either do it like avoid-combining-selectors, and export the tests (we should then also rename them to fixtures to make it clear) and then iterate over them, and create a test for each fixture. This gives us a good overview (name of the fixture, and the formatted exception).

  FAIL  "avoid-combining-selectors.fixture"
    Error endColumn should be 89 (90 strictEqual 89)  (strictEqual)

        ++89    (Expected)
        --90    (Actual)

The other option is to keep the rules test in place but wrap them in an uvu test, like did for avoid-cyclic-effects

rules\avoid-cyclic-effects.test.ts
✘   (0 / 1)

   FAIL  "\tests\rules\avoid-cyclic-effects.test.ts"
    Error endColumn should be 30 (29 strictEqual 30)  (strictEqual)

        ++30    (Expected)
        --29    (Actual)

@rafaelss95
Copy link
Collaborator

Cool, this is much better and will definitely save a considerable debugging time. Thank you for that! 🥇

I liked the second option better... but I have a suggestion, to avoid too much nesting we could move the valid and invalid arrays to outside of any callbacks... so we'd have:

const valid = [
  // ...
]
const invalid = [
  // ...
]

test(__filename, () => {
  ruleTester().run(path.parse(__filename).name, rule, { valid, invalid })
}
test.run()

@timdeschryver
Copy link
Owner Author

timdeschryver commented Oct 7, 2021

That's a great idea!
I'm afraid that if I convert all rules, that this will conflict with #221 though.
If it's OK I prefer to wait until that one is merged and then I'll update all tests.
If the "refactored" version is helpful for #221, feel free to update the rules that you refactor in #221, and then I will do the rest of the rules in this PR.

@github-actions github-actions bot added the conflicts There is a merge conflict label Oct 12, 2021
@github-actions github-actions bot removed the conflicts There is a merge conflict label Oct 12, 2021
@timdeschryver
Copy link
Owner Author

Something like this @rafaelss95 ?

Copy link
Collaborator

@rafaelss95 rafaelss95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, but overall it looks pretty good. 🥳

tests/rules/avoid-combining-selectors.test.ts Outdated Show resolved Hide resolved
tests/rules/avoid-duplicate-actions-in-reducer.test.ts Outdated Show resolved Hide resolved
{},
on(),
)`,
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add this where possible:

Suggested change
]
] as const

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where would this be needed?

tests/rules/avoid-duplicate-actions-in-reducer.test.ts Outdated Show resolved Hide resolved
path.parse(__filename).name,
rule,
type MessageIds = ESLintUtils.InferMessageIdsTypeFromRule<typeof rule>
type Options = ESLintUtils.InferOptionsTypeFromRule<typeof rule>[0][]
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still had to do [0][] here to get the types correct, [option] vs option[]

@timdeschryver
Copy link
Owner Author

Merging this in so other tests can be changed without affecting this merge.
If there are comments, I'll resolve them in a separate PR.

@timdeschryver timdeschryver merged commit 44e0829 into main Oct 20, 2021
@timdeschryver timdeschryver deleted the test/improve-dx branch October 20, 2021 16:39
@github-actions
Copy link

🎉 This PR is included in version 1.46.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants