[CI] SauceLabs Firefox tests failing

[nolanlawson]

We're getting a 500 error from SauceLabs for our Firefox tests starting yesterday. We did not change our config.

ERROR webdriver: Request failed with status 500 due to
  Error: Infrastructure Error -- The Sauce VMs failed to start the browser or device.

click to see

05 10 2022 02:39:35.474:WARN [launcher]: firefox latest  on SauceLabs has not captured in 60000 ms, killing.
05 10 2022 02:39:35.474:ERROR [SaucelabsLauncher]: Could not quit the Saucelabs selenium connection. Failure message:
05 10 2022 02:39:35.475:ERROR [SaucelabsLauncher]: TypeError: Cannot read properties of null (reading 'deleteSession')
    at SaucelabsLauncher.<anonymous> (/home/circleci/lwc/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:149:26)
    at Generator.next (<anonymous>)
    at /home/circleci/lwc/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/circleci/lwc/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:4:12)
    at SaucelabsLauncher.<anonymous> (/home/circleci/lwc/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:141:31)
    at SaucelabsLauncher.emit (node:events:520:28)
    at SaucelabsLauncher.emit (node:domain:475:12)
    at SaucelabsLauncher.emitAsync (/home/circleci/lwc/node_modules/karma/lib/events.js:52:10)
    at SaucelabsLauncher.BaseLauncher.kill (/home/circleci/lwc/node_modules/karma/lib/launchers/base.js:61:27)
    at Timeout._onTimeout (/home/circleci/lwc/node_modules/karma/lib/launchers/capture_timeout.js:22:12)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
05 10 2022 02:39:35.476:INFO [launcher]: Trying to start firefox latest  on SauceLabs again (1/2).
2022-10-05T02:40:01.100Z ERROR webdriver: Request failed with status 500 due to Error: Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
2022-10-05T02:40:01.101Z ERROR webdriver: Error: Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
    at Object.getErrorFromResponseBody (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:94:12)
    at WebDriverRequest._request (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/request.js:134:31)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.startWebDriverSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:29:20)
    at async Function.newSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/index.js:35:45)
    at async Object.exports.remote (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriverio/build/index.js:53:22)
05 10 2022 02:40:01.102:ERROR [SaucelabsLauncher]: Error: Failed to create session.
Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
    at Object.startWebDriverSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:34:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Function.newSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/index.js:35:45)
    at async Object.exports.remote (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriverio/build/index.js:53:22)
05 10 2022 02:40:01.102:INFO [launcher]: Trying to start firefox latest  on SauceLabs again (2/2).
2022-10-05T02:40:30.234Z ERROR webdriver: Request failed with status 500 due to Error: Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
2022-10-05T02:40:30.234Z ERROR webdriver: Error: Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
    at Object.getErrorFromResponseBody (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:94:12)
    at WebDriverRequest._request (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/request.js:134:31)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.startWebDriverSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:29:20)
    at async Function.newSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/index.js:35:45)
    at async Object.exports.remote (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriverio/build/index.js:53:22)
05 10 2022 02:40:30.234:ERROR [SaucelabsLauncher]: Error: Failed to create session.
Infrastructure Error -- The Sauce VMs failed to start the browser or device.
For more info, please check https://docs.saucelabs.com/dev/error-messages
    at Object.startWebDriverSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/utils.js:34:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Function.newSession (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriver/build/index.js:35:45)
    at async Object.exports.remote (/home/circleci/lwc/node_modules/karma-sauce-launcher/node_modules/webdriverio/build/index.js:53:22)
05 10 2022 02:40:30.234:ERROR [launcher]: firefox latest  on SauceLabs failed 2 times (failure). Giving up.
TOTAL: 3378 SUCCESS
error Command failed with exit code 1.

[nolanlawson]

Tried a bunch of stuff – different versions, different platforms, even screen resolution – nothing is working. Need to get in touch with Sauce support.

[jmsjtu]

The cause of this issue was from a change in Saucelabs, details can be found here.

A workaround, for now, is to use geckodriverVersion = 0.30.0.

The cause of the incompatible version could be from karma-sauce-launcher, but currently, we are not sure.

[nolanlawson]

There's a workaround involving yarn "resolutions", but it doesn't work yet for us because we already use a newer version of Webdriverio in another sub-package: karma-runner/karma-sauce-launcher#275 (comment)