Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🐛] Failing API / Frisby Tests on Node.js 20 #2068

Closed
J12934 opened this issue Aug 26, 2023 · 4 comments
Closed

[🐛] Failing API / Frisby Tests on Node.js 20 #2068

J12934 opened this issue Aug 26, 2023 · 4 comments

Comments

@J12934
Copy link
Member

J12934 commented Aug 26, 2023

🐛 Bug report

Description

Our API / Frisby Tests have been failing regularly on Node.js 20.
Reproducible both locally and in CI environments on all operating systems.

Is this a regression?

Yes, before ~21.8.23 they have been working API / Frisby, e.g. for this commit 68ef6d3

First commit on develop where it stopped working:
1fb0f12

(Also broke around the same time on other unrelated branches, so might rather be a change in a dependency than a code change on JuiceShop side)

Currently unclear what has changed at this point

🔬 Minimal Reproduction

nvm use 20
rm -rf node_modules/
npm install
npm run frisby

🔥 Exception or Error

FAIL test/api/feedbackApiSpec.ts
  ● /api/Feedbacks › POST feedback is associated with current user

    FetchError: request to http://localhost:3000/rest/user/login failed, reason: socket hang up

      at ClientRequest.<anonymous> (node_modules/node-fetch/lib/index.js:1501:11)

The POST feedback is associated with current user always appears to be failing.
Unfortunately other tests are sometimes failing as well.

🌳 Your Environment

$ node -v && npm -v
v20.5.1
9.8.0

Additional Information

Related slack discussion: https://owasp.slack.com/archives/C255XSY04/p1692548270574669

Potentially related to:

bkimminich added a commit that referenced this issue Oct 11, 2023
bkimminich added a commit that referenced this issue Oct 11, 2023
bkimminich added a commit that referenced this issue Oct 11, 2023
@bkimminich
Copy link
Member

Just tested that the issue is entirely generic and not related to any particular test. xiting failing test cases will then make a different test case in the spec file fail. Removing the entire spec file will then make another spec file fail. Tested this for three iterations before giving up.

@bkimminich
Copy link
Member

Tried these, also with no success: https://connectreport.com/blog/tuning-http-keep-alive-in-node-js/

@bkimminich
Copy link
Member

I ran the test suite with network debugging enabled (NODE_DEBUG=net npm run frisby) but I can't spot any difference between the log output right before a test case fails from ECONNRESET vs. where test cases just pass without issue.

@bkimminich
Copy link
Member

I tried to narrow things down a bit more by removing the server-launch via the test script, i.e. removing

    "globalSetup": "./test/apiTestsSetup.ts",
    "globalTeardown": "./test/apiTestsTeardown.ts",

and instead starting the server manually a minute before launching the test suite. The error behavior remains the same, so I am quite confident that the problem has nothing to do with the server "not being ready" in any way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants