test: 5449 shard tests #5492

tihuan · 2023-08-14T20:54:29Z

Reason for Change

Shard tests to support multiple browser tests and combine test report #5449

WARNING ⚠️ :
~~We had to upgrade to Playwright 1.37.0 to use the merge report function, however, .only() is temporarily broken at the moment. See issue~~ UPDATE: Upgraded to 1.37.1, which fixes the .only() issue!

Changes

Upgrade Playwright to the latest (1.37.0)
Update e2e test to run with 30 workers
NOT running firefox and edge yet, because that's too many workers for GHA to handle. We will revisit testing FF and Edge down the road
Add GHA steps to upload and merge sharded Playwright reports

Testing steps

e2e tests should now run with 30 works and roughly ~10 mins
You should still see the test report in the GHA summary artifacts section

Notes for Reviewer

codecov · 2023-08-14T21:35:54Z

Codecov Report

Merging #5492 (5046ee4) into main (055c131) will decrease coverage by 0.02%.
Report is 1 commits behind head on main.
The diff coverage is n/a.

❗ Current head 5046ee4 differs from pull request most recent head 085f000. Consider uploading reports for the commit 085f000 to get more accurate results

@@            Coverage Diff             @@
##             main    #5492      +/-   ##
==========================================
- Coverage   85.61%   85.59%   -0.02%     
==========================================
  Files         195      194       -1     
  Lines       14929    14915      -14     
==========================================
- Hits        12781    12767      -14     
  Misses       2148     2148

Flag	Coverage Δ
unittests	`85.59% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

tihuan · 2023-08-15T23:03:32Z

.github/workflows/push-tests.yml

+      - name: Merge Playwright sharded reports
+        run: |
+          npx playwright --version
+          npx playwright merge-reports --reporter html ./all-blob-reports


"unknown command" for some reason

Asking Playwright in Discord here: https://discord.com/channels/807756831384403968/1141146168299094107

tihuan · 2023-08-16T17:42:20Z

Parking this PR until Playwright 1.38 is out and hopefully .only() is fixed! microsoft/playwright#26492

tihuan · 2023-08-21T15:17:50Z

Good news! Just upgraded to Playwright 1.37.1, so .only() works again 👍

I also temporarily skip logged in tests in this PR until I implement this ticket this week

tihuan · 2023-08-21T15:48:30Z

fixing npm commands a little

This reverts commit 56e3bc5.

* test: remove broken, but no longer relevant test related to removed Add Tissue functionality (#5512) * test: remove manage tissue tests (#5509) * chore(cellguide): add types to tree builder pipeline (#5504) * chore(cellguide): add types to tree builder pipeline * update test * updated * update * first round of refactoring * fix * add docstrings * move module * add arg to remove nones * set default to false (#5519) * refactor * test: fix rightSideBar.test.ts (#5508) * fix(wmg-v2): Fixing Dynamic X-Axis Height Logic (#5415) * feat(cellguide): implement pipeline for the cellguide cards, synonyms, and descriptions data artifacts (#5501) * implement pipeline for cellguide metadata * update type * start writing tests * create test fixture * test top level pipeline function instead * address review * update json file to exclude nones * update to output json * update tests to test generator to avoid needing to mock json upload * ran precommit --------- Co-authored-by: atarashansky <atarashansky@chanzuckerberg.com> * feat: Update privacy (#5498) * feat(cellguide): Replace Source Data Scroll with Pagination (#5432) * fix(schema-migration): Pass collection_version to publish_collection_version (#5518) * Pass collection_version to publish_collection_version Signed-off-by: Trent Smith <trent.smith@chanzuckerberg.com> * collection_version.version_id Signed-off-by: Trent Smith <trent.smith@chanzuckerberg.com> --------- Signed-off-by: Trent Smith <trent.smith@chanzuckerberg.com> Co-authored-by: Nayib Gloria <55710092+nayib-jose-gloria@users.noreply.github.com> * fix(wmg): revert FMG pipeline simplification (#5522) * fix(wmg): revert FMG pipeline simplification * revert test changes * update fixtures --------- Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com> * chore: Update gitignore to account for unzipped fixtures (#5528) * test: Remove tsx imports (#5517) * feat(cellguide): compile all cell type marker genes from ASCTB (#5520) * feat(cellguide): compile all cell type marker genes from ASCTB (#5520) * chore(cellguide): ChatGPT Help Icon Tooltip Styles (#5507) * chore(cellguide): small fixes to typing and added safety (#5535) * feat: Add a Github Action that automatically closes stale PRs. (#5474) * fix: add codecov reports on merges to main (#5544) * fix: add codecov reports on merges to main * overhaul * merge -> push * chore(cellguide): Minor UI fixes – #5410 (#5431) * chore(cellguide): Minor UI fixes – #5410 * header xl lineHeight to 28px * removing unused import * fixing autocomplete label text overlapping top navbar * pr changes * typo * feat(cellguide): Move HuBMAP and CZI Tooltip Label (#5513) Co-authored-by: atarashansky <atarashansky@chanzuckerberg.com> * feat(cellguide): Data Table "overflow" Tags #5405 (#5503) * feat(cellguide): Data Table "overflow" Tags #5405 * removing leave delay for tissue/disease tooltip --------- Co-authored-by: atarashansky <atarashansky@chanzuckerberg.com> * test: Update max failure (#5545) * feat(cellguide): add tissue metadata pipeline (#5553) * reduce memory allocated to prod and staging for upload jobs. (#5510) Signed-off-by: Trent Smith <trent.smith@chanzuckerberg.com> Co-authored-by: Nayib Gloria <55710092+nayib-jose-gloria@users.noreply.github.com> * feat(cellguide): computational marker genes (#5530) * feat(cellguide): computational marker gene pipeline * feat(cellguide): implement pipeline for the source collections data artifact (#5542) * This is a combination of 4 commits. pipeline code * test: Update max failure again (#5557) * fix: Staging hotfix 08/19/2023 (#5558) * fix(schema-migration): Pass collection_version to publish_collection_version (#5518) * reduce memory allocated to prod and staging for upload jobs. (#5510) * fix: 5358 share link tissue accept id and name (#5531) * feat(cellguide): pipeline polish to get it to run locally as a script and add parallelism (#5560) feat(cellguide): pipeline polish to get it to run locally as a script and add parallelism (#5560) * fix: staging batch container memory limit + file cleanup (#5565) * fix: staging batch container memory limit * fix(schema-migration): migrated.h5ad file cleanup * log the error message for each dataset during publish * fix(wmg): add correct tooltip cell type for tissue rows and add test coverage (#5567) * fix(wmg): add correct tooltip cell type for tissue rows and add test coverage * remove incorrect testing * rename symbols and add explanation comments for UBERON IDs * fix: merge hotfixes back into dev (#5580) * fix: staging batch container memory limit + file cleanup (#5565) * fix: staging batch container memory limit * fix(schema-migration): migrated.h5ad file cleanup * log the error message for each dataset during publish * fix: correctly handle setting filtered cell types to 0 (#5568) * fix: correctly handle setting filtered cell types to 0 * Update frontend/src/views/WheresMyGeneV2/components/HeatMap/index.tsx Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com> --------- Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com> * chore: use functest@example.com for functional testing (#5541) * fix: pull ontology mappings in PR to appease branch protections (#5385) * fix: pull ontology mappings in PR to appease branch protections * remove test-only trigger * fix: functional test user (#5584) * test: 5449 shard tests (#5492) * test: e2e shard tests * temporarily not run logged in tests * use npm command * Revert "test: 5449 shard tests (#5492)" (#5587) This reverts commit 56e3bc5. --------- Signed-off-by: Trent Smith <trent.smith@chanzuckerberg.com> Co-authored-by: Severiano Badajoz <sbadajoz@chanzuckerberg.com> Co-authored-by: atarashansky <atarashansky@chanzuckerberg.com> Co-authored-by: Andrew Shin <109984998+ashin-czi@users.noreply.github.com> Co-authored-by: Joyce Yan <5653616+joyceyan@users.noreply.github.com> Co-authored-by: Timmy Huang <tihuan@users.noreply.github.com> Co-authored-by: Nayib Gloria <55710092+nayib-jose-gloria@users.noreply.github.com> Co-authored-by: maniarathi <arathi.mani@chanzuckerberg.com> Co-authored-by: Emanuele Bezzi <ebezzi@chanzuckerberg.com>

* test: e2e shard tests * temporarily not run logged in tests * use npm command

This reverts commit 56e3bc5.

tihuan changed the title ~~Thuang 5449 shard tests~~ test: 5449 shard tests Aug 14, 2023

tihuan force-pushed the thuang-5449-shard-tests branch 7 times, most recently from e15fc2d to 247b1c7 Compare August 14, 2023 21:35

tihuan force-pushed the thuang-5449-shard-tests branch 13 times, most recently from 949876f to 8d40602 Compare August 15, 2023 22:53

tihuan commented Aug 15, 2023

View reviewed changes

tihuan force-pushed the thuang-5449-shard-tests branch 5 times, most recently from 56c8bb7 to ece44e6 Compare August 16, 2023 16:23

tihuan requested review from atarashansky and seve August 16, 2023 17:31

tihuan marked this pull request as draft August 16, 2023 17:41

tihuan mentioned this pull request Aug 16, 2023

test(e2e): only run chrome until GHA sharding is done #4640

Merged

tihuan force-pushed the thuang-5449-shard-tests branch 2 times, most recently from c47090f to 9d13b29 Compare August 21, 2023 15:06

tihuan marked this pull request as ready for review August 21, 2023 15:12

tihuan force-pushed the thuang-5449-shard-tests branch 2 times, most recently from 5d9a423 to dcdf7d6 Compare August 21, 2023 15:48

tihuan marked this pull request as draft August 21, 2023 15:48

tihuan force-pushed the thuang-5449-shard-tests branch from dcdf7d6 to f1e30c1 Compare August 21, 2023 15:57

tihuan marked this pull request as ready for review August 21, 2023 18:10

tihuan enabled auto-merge (squash) August 21, 2023 19:29

atarashansky approved these changes Aug 22, 2023

View reviewed changes

tihuan added 3 commits August 22, 2023 08:40

test: e2e shard tests

501d0bb

temporarily not run logged in tests

4320fb6

use npm command

bc57fad

tihuan force-pushed the thuang-5449-shard-tests branch from 7496f75 to bc57fad Compare August 22, 2023 15:41

tihuan and others added 2 commits August 22, 2023 08:45

fix cell guide test

5046ee4

Merge branch 'main' into thuang-5449-shard-tests

085f000

tihuan merged commit 56e3bc5 into main Aug 22, 2023
43 checks passed

tihuan deleted the thuang-5449-shard-tests branch August 22, 2023 16:36

nayib-jose-gloria added a commit that referenced this pull request Aug 22, 2023

Revert "test: 5449 shard tests (#5492)"

a76a723

This reverts commit 56e3bc5.

nayib-jose-gloria mentioned this pull request Aug 22, 2023

fix: Revert "test: 5449 shard tests" #5587

Merged

nayib-jose-gloria added a commit that referenced this pull request Aug 22, 2023

Revert "test: 5449 shard tests (#5492)" (#5587)

9df5fdd

This reverts commit 56e3bc5.

tihuan mentioned this pull request Aug 23, 2023

test: e2e shard tests #5596

Merged

Bento007 pushed a commit that referenced this pull request Oct 17, 2023

test: 5449 shard tests (#5492)

ff007a1

* test: e2e shard tests * temporarily not run logged in tests * use npm command

Bento007 pushed a commit that referenced this pull request Oct 17, 2023

Revert "test: 5449 shard tests (#5492)" (#5587)

173bcc5

This reverts commit 56e3bc5.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: 5449 shard tests #5492

test: 5449 shard tests #5492

tihuan commented Aug 14, 2023 •

edited

codecov bot commented Aug 14, 2023 •

edited

tihuan Aug 15, 2023 •

edited

tihuan Aug 15, 2023

tihuan commented Aug 16, 2023

tihuan commented Aug 21, 2023

tihuan commented Aug 21, 2023

test: 5449 shard tests #5492

test: 5449 shard tests #5492

Conversation

tihuan commented Aug 14, 2023 • edited

Reason for Change

Changes

Testing steps

Notes for Reviewer

codecov bot commented Aug 14, 2023 • edited

Codecov Report

tihuan Aug 15, 2023 • edited

Choose a reason for hiding this comment

tihuan Aug 15, 2023

Choose a reason for hiding this comment

tihuan commented Aug 16, 2023

tihuan commented Aug 21, 2023

tihuan commented Aug 21, 2023

tihuan commented Aug 14, 2023 •

edited

codecov bot commented Aug 14, 2023 •

edited

tihuan Aug 15, 2023 •

edited