Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite is not re-runnable using --watch due to keeping state #2502

Closed
fatso83 opened this issue Mar 24, 2023 · 1 comment
Closed

Test suite is not re-runnable using --watch due to keeping state #2502

fatso83 opened this issue Mar 24, 2023 · 1 comment

Comments

@fatso83
Copy link
Contributor

fatso83 commented Mar 24, 2023

Describe the bug
When using the --watch option of Mocha the tests start failing on any change

To Reproduce
Steps to reproduce the behavior:

  1. npx mocha --watch test
  2. Change any test file
  3. See the test suite break with 4 failing tests
 1259 passing (3s)
  2 pending
  4 failing

  1) assert
       message
         assert.calledWith match.defined exception message:

      AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n \x1B[31m\x1B[31mdefined\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n \x1B[31mdefined\x1B[0m'
      + expected - actual

       expected doSomething to be called with arguments 
      - defined
      + defined
      
      at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21)
      at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17)
      at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11)
      at Function.referee.<computed>.<computed> [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22)
      at Context.<anonymous> (test/assert-test.js:1847:20)
      at processImmediate (node:internal/timers:466:21)

  2) assert
       message
         assert.calledWith match.truthy exception message:

      AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n \x1B[31m\x1B[31mtruthy\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n \x1B[31mtruthy\x1B[0m'
      + expected - actual

       expected doSomething to be called with arguments 
      - truthy
      + truthy
      
      at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21)
      at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17)
      at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11)
      at Function.referee.<computed>.<computed> [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22)
      at Context.<anonymous> (test/assert-test.js:1858:20)
      at processImmediate (node:internal/timers:466:21)

  3) assert
       message
         assert.calledWith match.falsy exception message:

      AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n' +
  '\x1B[32mtrue\x1B[0m \x1B[31m\x1B[31mfalsy\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n' +
  '\x1B[32mtrue\x1B[0m \x1B[31mfalsy\x1B[0m'
      + expected - actual

       expected doSomething to be called with arguments 
      -true falsy
      +true falsy
      
      at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21)
      at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17)
      at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11)
      at Function.referee.<computed>.<computed> [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22)
      at Context.<anonymous> (test/assert-test.js:1869:20)
      at processImmediate (node:internal/timers:466:21)

  4) stub
       non enumerable properties
         create stub from function on object:

      AssertionError: [assert.equals] [ 'aProp', 'func2' ] expected to be equal to [ 'aProp' ]
      + expected - actual

       [
         "aProp"
      -  "func2"
       ]
      
      at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21)
      at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17)
      at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11)
      at Function.referee.<computed>.<computed> [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22)
      at Context.<anonymous> (test/stub-test.js:180:20)
      at processImmediate (node:internal/timers:466:21)

Expected behavior
Immutable, non-failing test runs with no shared global state.

Context (please complete the following information):

  • Library version: 15.0.2

Additional context

This is due to two things that I could see

  1. We change the prototype of Function in https://github.com/sinonjs/sinon/pull/1968/files#diff-ebffa5a039be75d22925d266e16de1f0dbf14a625e5998bb0982e8523ce791c9R1332-R1337
  2. Some tests seem to keep state that makes re-runs of tests fail
fatso83 added a commit to sinonjs/samsam that referenced this issue Mar 26, 2023
fatso83 added a commit that referenced this issue Mar 26, 2023
* fix: do not modify the Function prototype when running tests

Made a single test fail as the Function prototype had been mutated by a later test in the test suite.

refs #2502

* fix: do not mutate matcher object message

This made 3 of the assertion tests fail when re-running as Sinon's spy formatter changed the exports of the samsam module

* Use latest version of samsam to get immutable messages

Ensure we cannot do the same mistake again
@fatso83
Copy link
Contributor Author

fatso83 commented Apr 19, 2023

Was closed in #2504

@fatso83 fatso83 closed this as completed Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant