Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repo: add e2e test(s) for the website playground #5579

Closed
bradzacher opened this issue Aug 31, 2022 · 6 comments · Fixed by #6080
Closed

Repo: add e2e test(s) for the website playground #5579

bradzacher opened this issue Aug 31, 2022 · 6 comments · Fixed by #6080
Assignees
Labels
accepting prs Go ahead, send a pull request that resolves this issue repo maintenance things to do with maintenance of the repo, and not with code/docs tests anything to do with testing

Comments

@bradzacher
Copy link
Member

bradzacher commented Aug 31, 2022

Suggestion

Reference: #5572

#5551 introduced a breakage into the website playground.
We don't have checks for this, so we didn't discover this until a few days after it had been merged and released.

The playground is the only part of the website that can easily and silently break due to dependency changes or changes in our packages because all the code is lazy-loaded.
It's too cumbersome to expect every relevant change to be manually tested against the website - so we should have automated testing to ensure that the page works correctly.

The MVP for this would be a test that ensures the playground can load without issue. Given the architecture this will likely need to be some form of webdriver (or whatever the best framework is) based e2e browser test, but we'd be more than open to some other solution if there's a better way to do it (even if it means some re-architecting of the tooling).

Full coverage would include a test that is an e2e browser test that enters code and config into the playground and asserts that the errors are shown.

@bradzacher bradzacher added tests anything to do with testing repo maintenance things to do with maintenance of the repo, and not with code/docs accepting prs Go ahead, send a pull request that resolves this issue labels Aug 31, 2022
@santoshyadavdev
Copy link
Contributor

As we are using NX, are you open to try cypress? I can do the initial PR to setup the cypress and run the tests on CI and local,

@JoshuaKGoldberg
Copy link
Member

#4362 - we already have Cypress set up 🙂

@bradzacher
Copy link
Member Author

Though, it is super flaky in the CI.
We're not sure why.

Right now we don't really pay attention to that signal cos it's often wrong.

@JoshuaKGoldberg
Copy link
Member

Aside: we've since switched to Playwright, because we couldn't get Cypress to stop being flaky in CI. #5814

@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Nov 16, 2022

My current plan is to do them on my 10:30am EST Twitch stream on November 24th. 🙌

Roughly:

  1. Adding text to the playground
  2. Enabling the AST viewer to inspect its ESTree and TypeScript ASTs
  3. Enabling a lint rule with the GUI
  4. Enabling a lint rule in the config text editor
  5. Seeing lint rule failures when violating text is added

@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Nov 25, 2022

Enabling the AST viewer to inspect its ESTree and TypeScript ASTs

oh oop I never added this... next!

Edit: actually, this is a good opportunity to make a good first issue. Open source maintenance!

Edit 2: #6086

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
accepting prs Go ahead, send a pull request that resolves this issue repo maintenance things to do with maintenance of the repo, and not with code/docs tests anything to do with testing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants