Crashes when run as a GitHub Action

[neilco]

Version(if relevant):

type-coverage 2.18.0
typescript 4.3.5

Environment(if relevant):

• GitHub Action running ubuntu-latest

Code(if relevant):

Type Coverage config in package.json

  "typeCoverage": {
    "atLeast": 90,
    "cache": true,
    "debug": false,
    "detail": false,
    "ignoreCatch": true,
    "ignoreFiles": [
        [REDACTED]
    ],
    "project": "tsconfig.json",
    "strict": true,
    "ignoreUnread": true,
    "ignoreNested": true,
    "ignoreAsAssertion": true,
    "ignoreTypeAssertion": true,
    "ignoreNonNullAssertion": true,
    "showRelativePath": true
  }

Expected:

Running type-coverage will only output the ratio used to calculate the coverage percentage and the result of the "at least" condition. If the condition is not met, exit with exit code 1. For example:

101706 / 119972 84.77%
The type coverage rate(84.77%) is lower than the target(90%).
error Command failed with exit code 1.

Actual:

Running type-coverage dumps thousands of log lines and exits abruptly. For example:

[neilco]

I've tried switching the runner to the macos-11 (same OS as I'm running locally) and the same behaviour persists when run as a GitHub Action.

[neilco]

I've been digging into this a little more and found it's related to the --at-least flag and the sheer size of the project. The project in question has on the order of 120,000 identifiers but only about 85% of those are not any. This means this enumeration produces on the order of 20,000 console.log messages. The GitHub Action runner does not like this and bails before completing the enumeration.

For a CI/CD perspective, we're not interested in the detail in the 20K log lines. We're only interested in whether the type coverage meets our threshold. So we can use a shell pipeline to limit the output to just the last two lines:

type-coverage | tail -n 2

The problem with this is: even if type-coverage exits with 1 since atLeastFailed is true, $? will be 0 because that's the exit code for tail -n 2. Thankfully, Bash helpfully maintains an array of exit codes for each segment of a pipeline and we can capture the exit code of type-coverage using ${PIPESTATUS[0]}.

Now my GitHub Action step to run type-coverage looks like this:

- name: Check TypeScript Coverage
  shell: bash
  run: |
    type-coverage | tail -n 2
    exit ${PIPESTATUS[0]}

This correctly fails when --at-least is higher than the actual type coverage, but succeeds when it is lower or equal.

A better solution than this workaround would be to modify this predicate: if (detail || atLeastFailed || isFailed). If it were if (detail && (atLeastFailed || isFailed)), it would only print out the anys if --detail was passed. That's probably how most users would expect this tool to work.

ASIDE: In my original report, I said:

"Running type-coverage will only output the ratio used to calculate the coverage percentage and the result of the 'at least' condition."

Having now looked at the code here, I know this is not correct. It turns out the anys array returned by lint is empty when I run this tool on macOS. That explains why I wasn't seeing the log lines the GitHub Action was producing 🤦🏻‍♂️

[plantain-00]

v2.19.0-alpha.0 support --no-detail-when-failed and noDetailWhenFailed, it will not show detail message when the CLI failed, it should fix this issue and not breaking anything.