Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix occasional test failures #25855

Merged
merged 1 commit into from Jun 7, 2021
Merged

Fix occasional test failures #25855

merged 1 commit into from Jun 7, 2021

Conversation

ijjk
Copy link
Member

@ijjk ijjk commented Jun 7, 2021 •
edited

This separates out the basic test suite into a non-concurrent group since it seems to fail when run concurrently with other tests at the same time, this also fixes another test item that has been flaking recently from us not waiting for the page to transition completely before a check.

Bug

  • Related issues linked using fixes #number
  • Integration tests added

Feature

  • Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
  • Related issues linked using fixes #number
  • Integration tests added
  • Documentation added
  • Telemetry added. In case of a feature if it's used or not.

Documentation / Examples

  • Make sure the linting passes

@ijjk
Copy link
Member Author

ijjk commented Jun 7, 2021

Stats from current PR

Default Server Mode (Decrease detected ✓)
General
vercel/next.js canary ijjk/next.js fix/flakey-test Change
buildDuration 14.3s 13.9s -439ms
buildDurationCached 3.5s 3.6s ⚠️ +127ms
nodeModulesSize 46.7 MB 46.7 MB ✓
Page Load Tests Overall decrease ⚠️
vercel/next.js canary ijjk/next.js fix/flakey-test Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.82 2.755 -0.06
/ avg req/sec 886.44 907.31 +20.87
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.46 1.515 ⚠️ +0.05
/error-in-render avg req/sec 1712.55 1650.24 ⚠️ -62.31
Client Bundles (main, webpack, commons)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
framework-HASH.js gzip 39.3 kB 39.3 kB ✓
main-HASH.js gzip 20.2 kB 20.2 kB ✓
webpack-HASH.js gzip 804 B 804 B ✓
Overall change 60.3 kB 60.3 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB ✓
Overall change 31.1 kB 31.1 kB ✓
Client Pages
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_app-HASH.js gzip 801 B 801 B ✓
_error-HASH.js gzip 3.07 kB 3.07 kB ✓
amp-HASH.js gzip 527 B 527 B ✓
css-HASH.js gzip 334 B 334 B ✓
hooks-HASH.js gzip 890 B 890 B ✓
index-HASH.js gzip 263 B 263 B ✓
link-HASH.js gzip 1.64 kB 1.64 kB ✓
routerDirect..HASH.js gzip 333 B 333 B ✓
withRouter-HASH.js gzip 330 B 330 B ✓
bb14e60e810b..30f.css gzip 125 B 125 B ✓
Overall change 8.31 kB 8.31 kB ✓
Client Build Manifests
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_buildManifest.js gzip 391 B 391 B ✓
Overall change 391 B 391 B ✓
Rendered Page Sizes
vercel/next.js canary ijjk/next.js fix/flakey-test Change
index.html gzip 572 B 572 B ✓
link.html gzip 579 B 579 B ✓
withRouter.html gzip 567 B 567 B ✓
Overall change 1.72 kB 1.72 kB ✓
Serverless Mode
General
vercel/next.js canary ijjk/next.js fix/flakey-test Change
buildDuration 15.8s 16.1s ⚠️ +344ms
buildDurationCached 4.7s 4.6s -51ms
nodeModulesSize 46.7 MB 46.7 MB ✓
Client Bundles (main, webpack, commons)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
framework-HASH.js gzip 39.3 kB 39.3 kB ✓
main-HASH.js gzip 20.2 kB 20.2 kB ✓
webpack-HASH.js gzip 804 B 804 B ✓
Overall change 60.3 kB 60.3 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB ✓
Overall change 31.1 kB 31.1 kB ✓
Client Pages
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_app-HASH.js gzip 801 B 801 B ✓
_error-HASH.js gzip 3.07 kB 3.07 kB ✓
amp-HASH.js gzip 527 B 527 B ✓
css-HASH.js gzip 334 B 334 B ✓
hooks-HASH.js gzip 890 B 890 B ✓
index-HASH.js gzip 263 B 263 B ✓
link-HASH.js gzip 1.64 kB 1.64 kB ✓
routerDirect..HASH.js gzip 333 B 333 B ✓
withRouter-HASH.js gzip 330 B 330 B ✓
bb14e60e810b..30f.css gzip 125 B 125 B ✓
Overall change 8.31 kB 8.31 kB ✓
Client Build Manifests
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_buildManifest.js gzip 391 B 391 B ✓
Overall change 391 B 391 B ✓
Serverless bundles
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_error.js 16.9 kB 16.9 kB ✓
404.html 2.44 kB 2.44 kB ✓
500.html 2.43 kB 2.43 kB ✓
amp.amp.html 10.8 kB 10.8 kB ✓
amp.html 1.63 kB 1.63 kB ✓
css.html 1.81 kB 1.81 kB ✓
hooks.html 1.69 kB 1.69 kB ✓
index.js 17.2 kB 17.2 kB -2 B
link.js 17.4 kB 17.4 kB ⚠️ +2 B
routerDirect.js 17.4 kB 17.4 kB ✓
withRouter.js 17.4 kB 17.4 kB ✓
Overall change 107 kB 107 kB ✓
Webpack 4 Mode (Increase detected ⚠️)
General
vercel/next.js canary ijjk/next.js fix/flakey-test Change
buildDuration 12.8s 12.8s -61ms
buildDurationCached 5.1s 5.2s ⚠️ +72ms
nodeModulesSize 46.7 MB 46.7 MB ✓
Page Load Tests Overall increase ✓
vercel/next.js canary ijjk/next.js fix/flakey-test Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.984 2.93 -0.05
/ avg req/sec 837.83 853.14 +15.31
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.493 1.489 0
/error-in-render avg req/sec 1674.03 1678.89 +4.86
Client Bundles (main, webpack, commons)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
677f882d2ed8..HASH.js gzip 13.3 kB 13.3 kB ✓
framework.HASH.js gzip 39 kB 39 kB ✓
main-HASH.js gzip 7.99 kB 7.99 kB ✓
webpack-HASH.js gzip 751 B 751 B ✓
Overall change 61.1 kB 61.1 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js fix/flakey-test Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB ✓
Overall change 31.3 kB 31.3 kB ✓
Client Pages
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_app-HASH.js gzip 1.07 kB 1.07 kB ✓
_error-HASH.js gzip 3.74 kB 3.74 kB ✓
amp-HASH.js gzip 536 B 536 B ✓
css-HASH.js gzip 339 B 339 B ✓
hooks-HASH.js gzip 887 B 887 B ✓
index-HASH.js gzip 227 B 227 B ✓
link-HASH.js gzip 1.63 kB 1.63 kB ✓
routerDirect..HASH.js gzip 303 B 303 B ✓
withRouter-HASH.js gzip 302 B 302 B ✓
e025d2764813..52f.css gzip 125 B 125 B ✓
Overall change 9.16 kB 9.16 kB ✓
Client Build Manifests
vercel/next.js canary ijjk/next.js fix/flakey-test Change
_buildManifest.js gzip 420 B 420 B ✓
Overall change 420 B 420 B ✓
Rendered Page Sizes
vercel/next.js canary ijjk/next.js fix/flakey-test Change
index.html gzip 627 B 627 B ✓
link.html gzip 633 B 633 B ✓
withRouter.html gzip 620 B 620 B ✓
Overall change 1.88 kB 1.88 kB ✓
Commit: 4b7c580

@ijjk ijjk marked this pull request as ready for review June 7, 2021 15:56
@ijjk
Copy link
Member Author

ijjk commented Jun 7, 2021

Failing test suites

Commit: 4b7c580

test/acceptance/ReactRefreshLogBox.dev.test.js

  • server-side only compilation errors

Expand output

● server-side only compilation errors

ScriptTimeoutError: script timeout
  (Session info: headless chrome=91.0.4472.77)

  72 |
  73 |             // Wait for application to re-hydrate:
> 74 |             await browser.executeAsyncScript(function () {
     |             ^
  75 |               var callback = arguments[arguments.length - 1]
  76 |               if (window.__NEXT_HYDRATED) {
  77 |                 callback()

  at Object.throwDecodedError (../node_modules/selenium-webdriver/lib/error.js:550:15)
  at parseHttpResponse (../node_modules/selenium-webdriver/lib/http.js:565:13)
  at Executor.execute (../node_modules/selenium-webdriver/lib/http.js:491:26)
      at runMicrotasks (<anonymous>)
  at Proxy.execute (../node_modules/selenium-webdriver/lib/webdriver.js:700:17)
  at Object.patch (acceptance/helpers.js:74:13)
  at Object.<anonymous> (acceptance/ReactRefreshLogBox.dev.test.js:1415:3)

@timneutkens timneutkens merged commit 7038b62 into vercel:canary Jun 7, 2021
@timneutkens timneutkens deleted the fix/flakey-test branch June 7, 2021 16:17
flybayer pushed a commit to blitz-js/next.js that referenced this pull request Jun 16, 2021
@ijjk
Fix occasional test failures (vercel#25855)
99d0997
@vercel vercel locked as resolved and limited conversation to collaborators Jan 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@divmain divmain Awaiting requested review from divmain

@lfades lfades Awaiting requested review from lfades

@shuding shuding Awaiting requested review from shuding shuding is a code owner

@timneutkens timneutkens Awaiting requested review from timneutkens timneutkens is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ijjk @timneutkens