Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meta: add flaky test issue template #33500

Closed
wants to merge 1 commit into from

Conversation

AshCripps
Copy link
Member

Add a flaky test issue template to ensure that enough information is provided for investigation

I think this will be useful especially for newer people interested in helping out the build WG as it requires no access to report flaky tests, and is often an approach we ask people to take to help familarise themselves with our CI/test suite.

Checklist

@nodejs-github-bot nodejs-github-bot added the meta Issues and PRs related to the general management of the project. label May 21, 2020
@bcoe
Copy link
Contributor

bcoe commented May 21, 2020

@AshCripps this is topical. I'm working with a group of interns this summer on a project related to accurately and automatically detecting flaky tests in a codebase (CC: @joyeecheung).

Would love to see if we could use Node.js as a place to test the methods we're coming up with.

I think developing a template for this is a great idea.

Copy link
Contributor

@sam-github sam-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggested text tweeks. I would find this template useful to bring some more structure to the flaky test reports.

.github/ISSUE_TEMPLATE/4-report-a-flaky-test.md Outdated Show resolved Hide resolved
.github/ISSUE_TEMPLATE/4-report-a-flaky-test.md Outdated Show resolved Hide resolved
@AshCripps
Copy link
Member Author

@bcoe @sam-github Ive addressed your comments, I've tried the details block, it didn't look too great locally but need to see how it looks on GitHub.

@sam-github
Copy link
Contributor

@bcoe WDYT? I'd be tempted to leave it out, the tap output is usually not that long.

Add a flaky test issue template to ensure that enough information is provided for investigation
@AshCripps
Copy link
Member Author

I updated this to remove the details block as it didn't look right IMO and have instead put a REPLACE ME inside a code block so the output will be formatted properly

@BridgeAR BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 26, 2020
sam-github pushed a commit that referenced this pull request May 28, 2020
Add a flaky test issue template to ensure that enough information is
provided for investigation

PR-URL: #33500
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
@sam-github
Copy link
Contributor

Landed in fb33b67

@sam-github sam-github closed this May 28, 2020
codebytere pushed a commit that referenced this pull request Jun 18, 2020
Add a flaky test issue template to ensure that enough information is
provided for investigation

PR-URL: #33500
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
@AshCripps AshCripps deleted the add-flaky-test-template branch June 25, 2020 14:01
@codebytere codebytere mentioned this pull request Jun 28, 2020
codebytere pushed a commit that referenced this pull request Jun 30, 2020
Add a flaky test issue template to ensure that enough information is
provided for investigation

PR-URL: #33500
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
codebytere pushed a commit that referenced this pull request Jul 8, 2020
Add a flaky test issue template to ensure that enough information is
provided for investigation

PR-URL: #33500
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
@codebytere codebytere mentioned this pull request Jul 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. meta Issues and PRs related to the general management of the project.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants