Configure default timeout via flag or environment variable

[dgp1130]

Current Behavior

The only ways to set test timeout currently are to explicitly set jasmine.DEFAULT_TIMEOUT_INTERVAL = // ... in JavaScript, or to add a timeout to the specific test callback. When trying to change the timeout for a large number of tests, it requires either modifying every test individually, or having some piece of shared code where the default value can be set. If the jasmine binary is sharded and executed multiple times, there may not be a single place to set this value.

Possible Solution

It should be possible to configure jasmine.DEFAULT_TIMEOUT_INTERVAL via an environment variable or command line flag. Running jasmine --default-timeout 10000 # ... or JASMINE_DEFAULT_TIMEOUT="10000" jasmine # ... should use the provided value as the default timeout.

If any tests override this value directly in code, those should probably take precedence. This flag merely sets the default timeout value for all tests.

Context

This is a particular challenge with Bazel, where tests are often sharded and executed across multiple jasmine executions for higher parallelism and to reduce dependencies on each execution for better cacheability. In this system, a single command like bazel test //... may run many jasmine binaries to run many tests. With this system, it is very difficult to modify the timeout for all the executed tests. Answering a question like "Are my tests running too slowly, or are they stuck altogether?" is much harder without being able to easily increase the test timeout.

Another example I'm encountering right now when debugging flaky tests, I would like to re-run them many times over to reproduce rare bugs. Bazel supports this with --runs_per_test 100, which will run all tests 100 times, helping to reproduce rare bugs. There is also a --jobs flag to run many of these targets in parallel. However in practice, running so many tests in parallel means there is more resource competition and the tests run slower as a result. This causes a number of timeout errors that I can't easily ignore. If I could just set the default timeout via environment variable or command line, I would be able to configure Bazel to pass that through and, then these erroneous failures could be easily ignored for local development via some kind of --define JASMINE_TIMEOUT="99999". Unfortunately, Jasmine provides no such mechanism, and there is no single location where I could add jasmine.DEFAULT_TIMEOUT_INTERVAL = 99999, which would apply to all tests.

Your Environment

• Version used: 3.6.3

• Environment name and version (e.g. Chrome 39, node.js 5.4): Node v12.10

• Operating System and version (desktop or mobile): Bazel runs Jasmine in a Linux container. My setup is running Bazel inside WSL2 on Windows 10, though I think it should work the same on most Linux distros.

• Link to your project: https://github.com/dgp1130/rules_prerender/

• Steps to reproduce:

  npm install
  npm run -s -- bazel test //examples/... --runs_per_test 100 --jobs 5
  # A more powerful machine than I have might need a higher jobs value to start reproducing errors.
  

  Some tests fail with, and I want to be able to raise the timeout to allow them to run a little slower:

  Message:
    Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
  Stack:
    Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
        at <Jasmine>
        at listOnTimeout (internal/timers.js:531:17)
        at processTimers (internal/timers.js:475:7)
  

[slackersoft]

This sounds like a great feature to have added to Jasmine. I would be happy to review a pull request to add configuration in the node.js library as well as here to allow the current setting to be configured from the external environment.