Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add GitHub Actions Reporter #11320

Merged
merged 23 commits into from Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -26,6 +26,7 @@
- `[jest-mock]` [**BREAKING**] Rename exported utility types `ConstructorLike`, `MethodLike`, `ConstructorLikeKeys`, `MethodLikeKeys`, `PropertyLikeKeys`; remove exports of utility types `ArgumentsOf`, `ArgsType`, `ConstructorArgumentsOf` - TS builtin utility types `ConstructorParameters` and `Parameters` should be used instead ([#12435](https://github.com/facebook/jest/pull/12435))
- `[jest-mock]` Improve `isMockFunction` to infer types of passed function ([#12442](https://github.com/facebook/jest/pull/12442))
- `[jest-mock]` Add support for auto-mocking async generator functions ([#11080](https://github.com/facebook/jest/pull/11080))
- `[jest-reporters]` Add GitHub Actions reporter ([#11320](https://github.com/facebook/jest/pull/11320))
- `[jest-resolve]` [**BREAKING**] Add support for `package.json` `exports` ([#11961](https://github.com/facebook/jest/pull/11961), [#12373](https://github.com/facebook/jest/pull/12373))
- `[jest-resolve, jest-runtime]` Add support for `data:` URI import and mock ([#12392](https://github.com/facebook/jest/pull/12392))
- `[jest-resolve, jest-runtime]` Add support for async resolver ([#11540](https://github.com/facebook/jest/pull/11540))
Expand Down
1 change: 1 addition & 0 deletions packages/jest-reporters/package.json
Expand Up @@ -19,6 +19,7 @@
"@jest/types": "^28.0.0-alpha.5",
"@types/node": "*",
"chalk": "^4.0.0",
"ci-info": "^3.2.0",
"collect-v8-coverage": "^1.0.0",
"exit": "^0.1.2",
"glob": "^7.1.2",
Expand Down
66 changes: 66 additions & 0 deletions packages/jest-reporters/src/GithubActionsReporter.ts
@@ -0,0 +1,66 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

import {GITHUB_ACTIONS} from 'ci-info';
import type {AggregatedResult, TestResult} from '@jest/test-result';
import BaseReporter from './BaseReporter';
import type {Context} from './types';

const lineAndColumnInStackTrace = /^.*?:([0-9]+):([0-9]+).*$/;

function replaceEntities(s: string): string {
const substitutions: Array<[RegExp, string]> = [
[/%/, '%25'],
ockham marked this conversation as resolved.
Show resolved Hide resolved
[/\r/g, '%0D'],
[/\n/g, '%0A'],
];
return substitutions.reduce((acc, sub) => acc.replace(...sub), s);
}

export default class GithubActionsReporter extends BaseReporter {
onRunComplete(
_contexts?: Set<Context>,
aggregatedResults?: AggregatedResult,
): void {
if (!GITHUB_ACTIONS) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question - should the reporter care? I'm thinking it makes more sense for the reporter to print when it's active, regardless of env. Then it's up to some other part of the code whether to activate it or not

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so not even having a more generic isCI check? 🤔

I was kinda thinking the reporter could be enabled by default, since GH is so pervasive -- adding a bit of OOTB delight to everyone's developer experience 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, but that's separate. We only enable coverage reporter of coverage is active (https://github.com/facebook/jest/blob/a20bd2c31e126fc998c2407cfc6c1ecf39ead709/packages/jest-core/src/TestScheduler.ts#L350-L381) we should do the same for a GH reporter. The reporter itself should always print if it's active, and the env detection should deicide to enable the reporter or not

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha! I'll remove the GHA check altogether then 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the defaults, should I add something like this to the PR?

Patch
diff --git a/packages/jest-core/src/TestScheduler.ts b/packages/jest-core/src/TestScheduler.ts
index 17fdff1935..7a9397e4e3 100644
--- a/packages/jest-core/src/TestScheduler.ts
+++ b/packages/jest-core/src/TestScheduler.ts
@@ -8,10 +8,12 @@
 /* eslint-disable local/ban-types-eventually */
 
 import chalk = require('chalk');
+import {GITHUB_ACTIONS} from 'ci-info';
 import exit = require('exit');
 import {
   CoverageReporter,
   DefaultReporter,
+  GitHubActionsReporter,
   NotifyReporter,
   Reporter,
   SummaryReporter,
@@ -347,11 +349,15 @@ class TestScheduler {
   }
 
   async _setupReporters() {
-    const {collectCoverage, notify, reporters} = this._globalConfig;
+    const {annotateGHA, collectCoverage, notify, reporters} =
+      this._globalConfig;
     const isDefault = this._shouldAddDefaultReporters(reporters);
 
     if (isDefault) {
-      this._setupDefaultReporters(collectCoverage);
+      this._setupDefaultReporters(
+        collectCoverage,
+        annotateGHA !== false && GITHUB_ACTIONS,
+      );
     }
 
     if (!isDefault && collectCoverage) {
@@ -364,6 +370,10 @@ class TestScheduler {
       );
     }
 
+    if (!isDefault && annotateGHA) {
+      this.addReporter(new GitHubActionsReporter());
+    }
+
     if (notify) {
       this.addReporter(
         new NotifyReporter(
@@ -379,7 +389,10 @@ class TestScheduler {
     }
   }
 
-  private _setupDefaultReporters(collectCoverage: boolean) {
+  private _setupDefaultReporters(
+    collectCoverage: boolean,
+    annotateGHA: boolean,
+  ) {
     this.addReporter(
       this._globalConfig.verbose
         ? new VerboseReporter(this._globalConfig)
@@ -396,6 +409,10 @@ class TestScheduler {
       );
     }
 
+    if (annotateGHA) {
+      this.addReporter(new GitHubActionsReporter());
+    }
+
     this.addReporter(new SummaryReporter(this._globalConfig));
   }

Or should that be done in a follow-up?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's do that in a follow-up!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to open up a PR when ready, btw. My main concern is if somebody wants to turn it off, that might be weird. But let's see! 🙂

Regardless, we should add 'github-actions' as a recognized string in the reporters array (like default is)

return;
}

const messages = getMessages(aggregatedResults?.testResults);

for (const message of messages) {
this.log(message);
}
}
}

function getMessages(results: Array<TestResult> | undefined) {
if (!results) return [];

return results.reduce(
flatMap(({testFilePath, testResults}) =>
testResults
.filter(r => r.status === 'failed')
.reduce(
flatMap(r => r.failureMessages),
[],
)
.map(m => replaceEntities(m))
.map(m => lineAndColumnInStackTrace.exec(m))
.filter((m): m is RegExpExecArray => m !== null)
.map(
([message, line, col]) =>
`::error file=${testFilePath},line=${line},col=${col}::${message}`,
),
),
[],
);
}

function flatMap<In, Out>(map: (x: In) => Array<Out>) {
ockham marked this conversation as resolved.
Show resolved Hide resolved
return (out: Array<Out>, entry: In) => out.concat(...map(entry));
}
127 changes: 127 additions & 0 deletions packages/jest-reporters/src/__tests__/GithubActionsReporter.test.js
@@ -0,0 +1,127 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
'use strict';

let GithubActionsReporter;

const write = process.stderr.write;
const globalConfig = {
rootDir: 'root',
watch: false,
};

let results = [];

function requireReporter() {
jest.isolateModules(() => {
GithubActionsReporter = require('../GithubActionsReporter').default;
});
}

beforeEach(() => {
process.stderr.write = result => results.push(result);
});

afterEach(() => {
results = [];
process.stderr.write = write;
});

const aggregatedResults = {
numFailedTestSuites: 1,
numFailedTests: 1,
numPassedTestSuites: 0,
numTotalTestSuites: 1,
numTotalTests: 1,
snapshot: {
added: 0,
didUpdate: false,
failure: false,
filesAdded: 0,
filesRemoved: 0,
filesRemovedList: [],
filesUnmatched: 0,
filesUpdated: 0,
matched: 0,
total: 0,
unchecked: 0,
uncheckedKeysByFile: [],
unmatched: 0,
updated: 0,
},
startTime: 0,
success: false,
testResults: [
{
numFailingTests: 1,
numPassingTests: 0,
numPendingTests: 0,
numTodoTests: 0,
openHandles: [],
perfStats: {
end: 1234,
runtime: 1234,
slow: false,
start: 0,
},
skipped: false,
snapshot: {
added: 0,
fileDeleted: false,
matched: 0,
unchecked: 0,
uncheckedKeys: [],
unmatched: 0,
updated: 0,
},
testFilePath: '/home/runner/work/jest/jest/some.test.js',
testResults: [
{
ancestorTitles: [Array],
duration: 7,
failureDetails: [Array],
failureMessages: [
`
Error: \u001b[2mexpect(\u001b[22m\u001b[31mreceived\u001b[39m\u001b[2m).\u001b[22mtoBe\u001b[2m(\u001b[22m\u001b[32mexpected\u001b[39m\u001b[2m) // Object.is equality\u001b[22m\n
\n
Expected: \u001b[32m\"b\"\u001b[39m\n
Received: \u001b[31m\"a\"\u001b[39m\n
at Object.<anonymous> (/home/runner/work/jest/jest/some.test.js:4:17)\n
at Object.asyncJestTest (/home/runner/work/jest/jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:106:37)\n
at /home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:45:12\n
at new Promise (<anonymous>)\n
at mapper (/home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:28:19)\n
at /home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:75:41\n
at processTicksAndRejections (internal/process/task_queues.js:93:5)
`,
],
fullName: 'asserts that a === b',
location: null,
numPassingAsserts: 0,
status: 'failed',
title: 'asserts that a === b',
},
],
},
],
};

test("reporter returns empty string if GITHUB_ACTIONS isn't set", () => {
requireReporter();
const testReporter = new GithubActionsReporter(globalConfig);
testReporter.onRunComplete(new Set(), aggregatedResults);
expect(results.join('').replace(/\\/g, '/')).toMatchSnapshot();
ockham marked this conversation as resolved.
Show resolved Hide resolved
});

test('reporter extracts the correct filename, line, and column', () => {
jest.doMock('ci-info', () => ({GITHUB_ACTIONS: true}));

requireReporter();
const testReporter = new GithubActionsReporter(globalConfig);
testReporter.onRunComplete(new Set(), aggregatedResults);
expect(results.join('').replace(/\\/g, '/')).toMatchSnapshot();
});
@@ -0,0 +1,8 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`reporter extracts the correct filename, line, and column 1`] = `
"::error file=/home/runner/work/jest/jest/some.test.js,line=4,col=17::%0A Error: <dim>expect(</><red>received</><dim>).</>toBe<dim>(</><green>expected</><dim>) // Object.is equality</>%0A%0A %0A%0A Expected: <green>"b"</>%0A%0A Received: <red>"a"</>%0A%0A at Object.<anonymous> (/home/runner/work/jest/jest/some.test.js:4:17)%0A%0A at Object.asyncJestTest (/home/runner/work/jest/jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:106:37)%0A%0A at /home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:45:12%0A%0A at new Promise (<anonymous>)%0A%0A at mapper (/home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:28:19)%0A%0A at /home/runner/work/jest/jest/node_modules/jest-jasmine2/build/queueRunner.js:75:41%0A%0A at processTicksAndRejections (internal/process/task_queues.js:93:5)%0A
ockham marked this conversation as resolved.
Show resolved Hide resolved
"
`;

exports[`reporter returns empty string if GITHUB_ACTIONS isn't set 1`] = `""`;
1 change: 1 addition & 0 deletions packages/jest-reporters/src/index.ts
Expand Up @@ -26,6 +26,7 @@ export {default as DefaultReporter} from './DefaultReporter';
export {default as NotifyReporter} from './NotifyReporter';
export {default as SummaryReporter} from './SummaryReporter';
export {default as VerboseReporter} from './VerboseReporter';
export {default as GithubActionsReporter} from './GithubActionsReporter';
ockham marked this conversation as resolved.
Show resolved Hide resolved
export type {
Context,
Reporter,
Expand Down
1 change: 1 addition & 0 deletions yarn.lock
Expand Up @@ -2743,6 +2743,7 @@ __metadata:
"@types/node": "*"
"@types/node-notifier": ^8.0.0
chalk: ^4.0.0
ci-info: ^3.2.0
collect-v8-coverage: ^1.0.0
exit: ^0.1.2
glob: ^7.1.2
Expand Down