Unit Tests Sporadically Disconnect on our CI:CD builds

[eberhak]

Hi Karma team,

I'm part of an Angular project for my organization with a CI:CD build running approx. 1260 unit tests. We run these unit tests as part of a nightly build.

Our builds run the following command to launch the unit tests:
ng test [project_name] --browsers=ChromeHeadless --watch=false --source-map=false

After 921 passing tests, the unit test portion of our build fails with the following error:
Disconnected (0 times) reconnect failed before timeout of 2000ms (ping timeout)

This issue happens only on CI:CD, I never get it locally when running unit tests on my machine. Even then, we don't get this for every build. It happens randomly.

We're using Angular version 13.2.6 and the following Karma versions:
"karma": "~6.3.17",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage": "~2.0.3",
"karma-jasmine": "~4.0.1",
"karma-jasmine-html-reporter": "^1.7.0",
"karma-scss-preprocessor": "^4.0.0",

What are some potential root causes we can troubleshoot to better understand why the unit tests randomly disconnect. Please advise. Thank you!

[matttm]

I am experiencing this issue as-well in Angular 13 and we saw this issue in Angular 10 too

    "karma": "^6.4.1",
    "karma-coverage": "^2.2.0",
    "karma-chrome-launcher": "~3.1.0",
    "karma-coverage-istanbul-reporter": "~3.0.2",
    "karma-jasmine": "~4.0.0",
    "karma-jasmine-html-reporter": "^1.5.0",
    "karma-parallel": "^0.3.1",

From the debugging output

Chrome Headless 106.0.5249.119 (Mac OS 10.15.7): Executed 3097 of 3229 (6 FAILED) (skipped 132) (10 mins 19.298 secs / 2 mins 56.816 secs)
20 10 2022 13:02:55.441:DEBUG [launcher]: CAPTURED -> BEING_KILLED
20 10 2022 13:02:55.441:DEBUG [launcher]: BEING_KILLED -> BEING_FORCE_KILLED
20 10 2022 13:02:55.441:DEBUG [Chrome Headless 106.0.5249.119 (Mac OS 10.15.7)]: CONNECTED -> DISCONNECTED
TOTAL: 6 FAILED, 3091 SUCCESS

@eberhak though I'm facing a similar issue, I've heard people have had success by modifying the following in the karma config

    browserSocketTimeout: 210000,
    browserDisconnectTimeout: 210000,
    browserDisconnectTolerance: 3,
    browserNoActivityTimeout: 210000,

[eberhak]

Hi @matttm Thanks for the advice on the config file. I can see this mitigating the issue, but the heart of the problem remains. Why does the browser even feel the need to disconnect in the first place? Is it just a network issue or is there a known bug with Karma? Do you have any insight on this or do you know anyone who does? Thanks!

[matttm]

@eberhak I am not sure of what's causing the issue in karma. Frankly, I became frustrated with karma and started a migration to jest using jest-jasmine2 test runner, so we could keep our tests, but there are some hiccups with that. Doing it though, I have had to modify some tests because they were failing under jest, but the some of the errors seem like things that should've errored under karma.

For example, I had to fix a number of tests that were missing imported modules in the harness. Maybe this was some misconfiguration on my part as for why I'm seeing it now. I also caught a couple mem leaks from subscribers never unsubscribing. I think these may have played a role.

[sonali395]

Do we have a solution for this issue ? I have tried out the timeout configuration in karma config file as well, as advised in other issue threads but none of it resolves the issue for me.

[netnic0]

Hi Karma Team,
I'm facing to the same issue.
Disconnected (0 times) reconnect failed before timeout of 2000ms (ping timeout)
It's really painful.
How we can update the timeout to have more than 2 sec?
Here is the part of our karma.conf.js file
We try to use browserDisconnectTimeout with different place without success

customLaunchers.Piper_ChromeRemote = {
          base: "WebDriver",
          browserName: "chrome",
          name: "Karma",
          pseudoActivityInterval: 30000,
          browserDisconnectTimeout: 30000,
          config: {
            hostname: "localhost",
            port: "4444",
          },
        };
        config.customLaunchers = customLaunchers;
        config.hostname = "localhost";
        config.browsers = ["Piper_ChromeRemote"];
        config.customLaunchers.Piper_ChromeRemote.browserDisconnectTimeout = 30000;
        config.reporters.push("junit");

Thanks for your help!

[achrysanthakopoulou]

Any updates? I have the same issue!

[Saipa1UI]

Hi Karma Team,
I have the same issue and adding the time-out properties in karma.config file did not fix the issue. This happens on my local too.