New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(ui): refactor e2e tests to use playwright cli #4721
Conversation
✅ Deploy Preview for fastidious-cascaron-4ded94 canceled.
|
.github/workflows/ci.yml
Outdated
@@ -109,6 +109,32 @@ jobs: | |||
- name: Test UI | |||
run: pnpm run ui:test | |||
|
|||
test-ui-e2e: | |||
runs-on: ubuntu-latest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this run on multiple os/browsers (like test
and test-browser
jobs)?
I just tested on macos and windows 02c77f7
Macos is okay but Windows seems to have problems when killing processes.
https://github.com/vitest-dev/vitest/actions/runs/7162660207/job/19499945258?pr=4721#step:7:24
I think this is a typical Windows issue, so probably I can find tips somewhere (runVitestCli
helper?).
Windows seems to be fine after using the same kill/exit await technique as runVitestCli
5f58331.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. This simplifies the testing setup a lot. Also happy to see execa
not used as it used to randomly cause issues in these tests.
Description
While thinking about expanding e2e tests for coverage preview fix PR #4717, I noticed that current e2e test in
test/ui
usesplaywright-chromium
package directly (introduced in #2710) and it doesn't benefit from usual playwright test authoring goodies (e.g. debug mode, trace viewer, multiple browsers, etc...) which are provided only forplaywright test
CLI usage and@playwright/test
package.Here, I'm proposing to rewrite current e2e tests into standard playwright test for the ease of maintenance.
I feel this is worth it (and overall change is very simple at this point), but I would like to know what maintainers think about this direction.
I also noticed that people from cypress was suggesting to use cypress at some point #590, so I'm not sure what is the overall opinion on vitest ui testing.
here is a quick rundown of potential testing workflow:
Previously the test was run during
pnpm test:ci
, but I think it makes sense to move it as a separate category, so I added a new CI jobtest-ui-e2e
to just runpnpm -C test/ui test-e2e
. I'll continue to think about what's the ideal structure to manage this on CI.Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
pnpm-lock.yaml
unless you introduce a new test example.Tests
pnpm test:ci
.Documentation
pnpm run docs
command.Changesets
feat:
,fix:
,perf:
,docs:
, orchore:
.