Jest not showing skipped tests and throws if testNamePattern does not match any test

[karanjitsingh]

💥 Regression Report

I need to discover the test case names before executing them. I use the configuration option testNamePattern with value ^$a which will not match any string and hence cause jest to skip all tests. I use a custom reporter to check the test case names. After updating jest to latest version, using this test pattern and running jest I get this error.

 FAIL  __test__/test.js
  ● Test suite failed to run

    Your test suite must contain at least one test.

      at node_modules/jest-cli/build/test_scheduler.js:246:22

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        2.68s
Ran all test suites with tests matching "^$a".

Another regression, if a testNamePattern matches some tests in one suite and no tests in another suite, the unmatched tests are not reported as skipped. Consider 4 test cases in the same file suite a > test 1, suite a > test 2, suite b > test 1 and suite b > test 2

 FAIL  __test__/test.js
  suite a
    √ test 1 (1ms)
    × test 2 (9ms)

  ● suite a › test 2

    assert.(received, expected)

    Expected value   undefined
    Received:
      ""

    Difference:

      Comparing two different types of values. Expected undefined but received string.

       7 |
       8 |     it('test 2', () => {
    >  9 |         assert.fail('');
         |                ^
      10 |     });
      11 | });
      12 |

      at Object.it (__test__/test.js:9:16)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 passed, 2 total
Snapshots:   0 total
Time:        2.981s
Ran all test suites with tests matching "suite a".

The two test cases in suite a match and are executed. Test cases for suite b don't match and are not shown as skipped.

Last working version

Worked up to version: 22.4.3
Stopped working in version: 23.2.0

To Reproduce

const assert = require('assert')

describe('suite a', () => {
    it('test 1', () => {

    });

    it('test 2', () => {
        assert.fail('');
    });
});

describe('suite b', () => {
    it('test 1', () => {

    });

    it('test 2', () => {
        assert.fail('');
    });
});

Run jest with config

{
    "testNamePattern": "^$a"
}

and

{
    "testNamePattern": "suite a"
}

Expected behavior

For "testNamePattern": "^$a"

Jest v22.1.2 node v7.4.0 linux/amd64
   
Test Suites: 1 skipped, 0 of 1 total
Tests:       4 skipped, 4 total
Snapshots:   0 total
Time:        0.915s, estimated 1s
Ran all test suites with tests matching "^$a".

For "testNamePattern": "suite a"

Jest v22.1.2 node v7.4.0 linux/amd64
   
 FAIL  ./add-test.js
  suite a
    ✓ test 1 (3ms)
    ✕ test 2 (9ms)
  suite b
    ○ skipped 2 tests

  ● suite a › test 2

    assert.(received, expected)
    
    Expected value   undefined
    Received:
      ""
    
    Difference:
    
      Comparing two different types of values. Expected undefined but received string.

       7 | 
       8 |     it('test 2', () => {
    >  9 |         assert.fail('');
      10 |     });
      11 | });
      12 | 
      
      at Object.it (add-test.js:9:16)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 2 skipped, 1 passed, 4 total
Snapshots:   0 total
Time:        1.002s
Ran all test suites with tests matching "suite a".
exit status 1

Link to repl or repo (highly encouraged)

I see that repl.it is running jest v22.1.2 and the output is as expected, but here it is anyway for the sake of this issue not being stalled: https://repl.it/repls/WideeyedFrayedTerabyte

Run npx envinfo --preset jest

npx: installed 1 in 2.58s
Path must be a string. Received undefined
npx: installed 1 in 2.651s
C:\Users\karsin\AppData\Roaming\npm-cache\_npx\8504\node_modules\envinfo\dist\cli.js

  System:
    OS: Windows 10
    CPU: x64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  Binaries:
    npm: 5.6.0 - C:\Program Files\nodejs\npm.CMD

[karanjitsingh]

Any update on this?

[SimenB]

Can reproduce on Jest master. Somebody:tm: should bisect

[wyze]

This looks pretty similar to #6415. Also, I fixed testNamePattern in regards to interactive snapshot updates. I wonder if there was some root cause that is bringing up regression reports related to testNamePattern?

[rogeliog]

I did a bisect, it was introduced by this PR #6234.

The regression is only present in jest-jasmine2, it works fine with JEST_CIRCUS=1

cc: @rickhanlonii @SimenB

[SimenB]

@jbdemonte thoughts?

[jbdemonte]

need to dig...

[jbdemonte]

I guess this pull request will fix it

[thymikee]

Closing via #6657

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.