Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: parse data from output workflow #5226

Merged
merged 7 commits into from May 9, 2024

Conversation

thisislawatts
Copy link
Member

@thisislawatts thisislawatts commented May 7, 2024

Pull Request Submission

Please check the boxes once done.

The pull request must:

  • Reviewer Documentation
    • follow CONTRIBUTING rules
    • be accompanied by a detailed description of the changes
    • contain a risk assessment of the change (Low | Medium | High) with regards to breaking existing functionality. A change e.g. of an underlying language plugin can completely break the functionality for that language, but appearing as only a version change in the dependencies.
    • highlight breaking API if applicable
    • contain a link to the automatic tests that cover the updated functionality.
    • contain testing instructions in case that the reviewer wants to manual verify as well, to add to the manual testing done by the author.
    • link to the link to the PR for the User-facing documentation
  • User facing Documentation
    • update any relevant documentation in gitbook by submitting a gitbook PR, and including the PR link here
    • ensure that the message of the final single commit is descriptive and prefixed with either feat: or fix: , others might be used in rare occasions as well, if there is no need to document the changes in the release notes. The changes or fixes should be described in detail in the commit message for the changelog & release notes.
  • Testing
    • Changes, removals and additions to functionality must be covered by acceptance / integration tests or smoke tests - either already existing ones, or new ones, created by the author of the PR.

Pull Request Review

All pull requests must undergo a thorough review process before being merged.
The review process of the code PR should include code review, testing, and any necessary feedback or revisions.
Pull request reviews of functionality developed in other teams only review the given documentation and test reports.

Manual testing will not be performed by the reviewing team, and is the responsibility of the author of the PR.

For Node projects: It’s important to make sure changes in package.json are also affecting package-lock.json correctly.

If a dependency is not necessary, don’t add it.

When adding a new package as a dependency, make sure that the change is absolutely necessary. We would like to refrain from adding new dependencies when possible.
Documentation PRs in gitbook are reviewed by Snyk's content team. They will also advise on the best phrasing and structuring if needed.

Pull Request Approval

Once a pull request has been reviewed and all necessary revisions have been made, it is approved for merging into
the main codebase. The merging of the code PR is performed by the code owners, the merging of the documentation PR
by our content writers.

What does this PR do?

Adds support --severity-threshold when determining the exit code. This means that you can use snyk code test --severity-threshold=high with the go based workflow and get the success exit code if only low severity issues are found.

How should this be manually tested?

Run the freshly built binary against a project that only contains low|medium severity issues, in this case I am running against https://github.com/snyk-fixtures/shallow-goof where .gitignore has been removed after installing so the scanner runs over node_modules.

~/snyk/cli/binary-releases/snyk-macos-arm64 code test ~/snyk/shallow-goof --org=<org-uuid-with-feature-flag> --severity-threshold=medium
echo $? // Should be 0

What are the relevant tickets?

CLI-304

@thisislawatts thisislawatts changed the title Feat/parse data from output workflow feat: parse data from output workflow May 7, 2024
@PeterSchafer PeterSchafer force-pushed the feat/parse-data-from-output-workflow branch from 7975a7c to 7aa49cd Compare May 8, 2024 11:44
@thisislawatts thisislawatts force-pushed the feat/parse-data-from-output-workflow branch from 73bad24 to 9a66f03 Compare May 8, 2024 19:04
@thisislawatts thisislawatts marked this pull request as ready for review May 8, 2024 19:08
@thisislawatts thisislawatts requested a review from a team as a code owner May 8, 2024 19:08
Copy link
Contributor

@cmars cmars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question below but nothing blocking necessarily.

func Test_runWorkflowAndProcessData(t *testing.T) {
defer cleanup()
globalConfiguration = configuration.New()
globalConfiguration.Set(configuration.DEBUG, true)
globalEngine = workflow.NewWorkFlowEngine(globalConfiguration)

testCmnd := "subcmd1"
addEmptyWorkflows(t, globalEngine, []string{"output"})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Should we keep this test as it was and add a new one specifically about the test summary result?

Or is it that we've changed the engine in this GAF update such that we must have a test summary result here and this is the new baseline?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, as a result of this change, we are now returning Test Summary objects if there are passed through to the output workflow. This represents a new base line behaviour.


_, err := globalEngine.Register(workflowId1, workflowConfig, outputFn)
if err != nil {
t.Fatal(err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

observation: assert.NoError(t, err) would be a nicer way to fail the test but that does go against the prevailing grain here so maybe that's a follow-up tidy-up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot, raised #5230 to address this refactor

@thisislawatts thisislawatts merged commit 36e2460 into main May 9, 2024
16 checks passed
@thisislawatts thisislawatts deleted the feat/parse-data-from-output-workflow branch May 9, 2024 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants