Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix default values for fail-on-severity #451

Merged
merged 10 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
Expand Up @@ -128,11 +128,12 @@ Start by specifying that you will be using an external configuration file:
config-file: './.github/dependency-review-config.yml'
```

And then create the file in the path you just specified:
And then create the file in the path you just specified. Please note
that the **option names in external files use underscores instead of dashes**:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I assume there's some known friction involved 👍 but I wonder could we relax the option name parsing to accept both formats? Feels like this will be tough for end users to remember

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, it's on the TODO. This PR only fixes the bug with defaults value, but going forward I think supporting both options by replacing - with _ when found is the way.


```yaml
fail-on-severity: 'critical'
allow-licenses:
fail_on_severity: 'critical'
allow_licenses:
- 'GPL-3.0'
- 'BSD-3-Clause'
- 'MIT'
Expand Down
94 changes: 1 addition & 93 deletions __tests__/config.test.ts
Expand Up @@ -2,35 +2,7 @@ import {expect, test, beforeEach} from '@jest/globals'
import {readConfig} from '../src/config'
import {getRefs} from '../src/git-refs'
import * as Utils from '../src/utils'

// GitHub Action inputs come in the form of environment variables
// with an INPUT prefix (e.g. INPUT_FAIL-ON-SEVERITY)
function setInput(input: string, value: string): void {
process.env[`INPUT_${input.toUpperCase()}`] = value
}

// We want a clean ENV before each test. We use `delete`
// since we want `undefined` values and not empty strings.
function clearInputs(): void {
const allowedOptions = [
'FAIL-ON-SEVERITY',
'FAIL-ON-SCOPES',
'ALLOW-LICENSES',
'DENY-LICENSES',
'ALLOW-GHSAS',
'LICENSE-CHECK',
'VULNERABILITY-CHECK',
'CONFIG-FILE',
'BASE-REF',
'HEAD-REF',
'COMMENT-SUMMARY-IN-PR'
]

// eslint-disable-next-line github/array-foreach
allowedOptions.forEach(option => {
delete process.env[`INPUT_${option.toUpperCase()}`]
})
}
import {setInput, clearInputs} from './test-helpers'

beforeAll(() => {
jest.spyOn(Utils, 'isSPDXValid').mockReturnValue(true)
Expand Down Expand Up @@ -105,60 +77,6 @@ test('it raises an error when no refs are provided and the event is not a pull r
).toThrow()
})

test('it reads an external config file', async () => {
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml')

const config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')
expect(config.allow_licenses).toEqual(['BSD', 'GPL 2'])
})

test('raises an error when the config file was not found', async () => {
setInput('config-file', 'fixtures/i-dont-exist')
await expect(readConfig()).rejects.toThrow(/Unable to fetch/)
})

test('it parses options from both sources', async () => {
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml')

let config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')

setInput('base-ref', 'a-custom-base-ref')
config = await readConfig()
expect(config.base_ref).toEqual('a-custom-base-ref')
})

test('in case of conflicts, the inline config is the source of truth', async () => {
setInput('fail-on-severity', 'low')
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml') // this will set fail-on-severity to 'critical'

const config = await readConfig()
expect(config.fail_on_severity).toEqual('low')
})

test('it uses the default values when loading external files', async () => {
setInput('config-file', './__tests__/fixtures/no-licenses-config.yml')
let config = await readConfig()
expect(config.allow_licenses).toEqual(undefined)
expect(config.deny_licenses).toEqual(undefined)

setInput('config-file', './__tests__/fixtures/license-config-sample.yml')
config = await readConfig()
expect(config.fail_on_severity).toEqual('low')
})

test('it accepts an external configuration filename', async () => {
setInput('config-file', './__tests__/fixtures/no-licenses-config.yml')
const config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')
})

test('it raises an error when given an unknown severity in an external config file', async () => {
setInput('config-file', './__tests__/fixtures/invalid-severity-config.yml')
await expect(readConfig()).rejects.toThrow()
})

test('it defaults to runtime scope', async () => {
const config = await readConfig()
expect(config.fail_on_scopes).toEqual(['runtime'])
Expand Down Expand Up @@ -234,16 +152,6 @@ test('it is not possible to disable both checks', async () => {
)
})

test('it supports comma-separated lists', async () => {
setInput(
'config-file',
'./__tests__/fixtures/inline-license-config-sample.yml'
)
const config = await readConfig()

expect(config.allow_licenses).toEqual(['MIT', 'GPL-2.0-only'])
})

describe('licenses that are not valid SPDX licenses', () => {
beforeAll(() => {
jest.spyOn(Utils, 'isSPDXValid').mockReturnValue(false)
Expand Down
111 changes: 111 additions & 0 deletions __tests__/external-config.test.ts
@@ -0,0 +1,111 @@
import {expect, test, beforeEach} from '@jest/globals'
import {readConfig} from '../src/config'
import * as Utils from '../src/utils'
import {setInput, clearInputs} from './test-helpers'

const externalConfig = `fail_on_severity: 'high'
allow_licenses: ['GPL-2.0-only']
`
const mockOctokit = {
rest: {
repos: {
getContent: jest.fn().mockReturnValue({data: externalConfig})
}
}
}

jest.mock('octokit', () => {
return {
// eslint-disable-next-line @typescript-eslint/no-extraneous-class
Octokit: class {
constructor() {
return mockOctokit
}
}
}
})

beforeAll(() => {
jest.spyOn(Utils, 'isSPDXValid').mockReturnValue(true)
})

beforeEach(() => {
clearInputs()
})

test('it reads an external config file', async () => {
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml')

const config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')
expect(config.allow_licenses).toEqual(['BSD', 'GPL 2'])
})

test('raises an error when the config file was not found', async () => {
setInput('config-file', 'fixtures/i-dont-exist')
await expect(readConfig()).rejects.toThrow(/Unable to fetch/)
})

test('it parses options from both sources', async () => {
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml')

let config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')

setInput('base-ref', 'a-custom-base-ref')
config = await readConfig()
expect(config.base_ref).toEqual('a-custom-base-ref')
})

test('in case of conflicts, the inline config is the source of truth', async () => {
setInput('fail-on-severity', 'low')
setInput('config-file', './__tests__/fixtures/config-allow-sample.yml') // this will set fail-on-severity to 'critical'

const config = await readConfig()
expect(config.fail_on_severity).toEqual('low')
})

test('it uses the default values when loading external files', async () => {
setInput('config-file', './__tests__/fixtures/no-licenses-config.yml')
let config = await readConfig()
expect(config.allow_licenses).toEqual(undefined)
expect(config.deny_licenses).toEqual(undefined)

setInput('config-file', './__tests__/fixtures/license-config-sample.yml')
config = await readConfig()
expect(config.fail_on_severity).toEqual('low')
})

test('it accepts an external configuration filename', async () => {
setInput('config-file', './__tests__/fixtures/no-licenses-config.yml')
const config = await readConfig()
expect(config.fail_on_severity).toEqual('critical')
})

test('it raises an error when given an unknown severity in an external config file', async () => {
setInput('config-file', './__tests__/fixtures/invalid-severity-config.yml')
await expect(readConfig()).rejects.toThrow()
})

test('it supports comma-separated lists', async () => {
setInput(
'config-file',
'./__tests__/fixtures/inline-license-config-sample.yml'
)
const config = await readConfig()

expect(config.allow_licenses).toEqual(['MIT', 'GPL-2.0-only'])
})

test('it reads a config file hosted in another repo', async () => {
setInput(
'config-file',
'future-funk/anyone-cualkiera/external-config.yml@main'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I wonder if we should put this file into a static repo under the actions org or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

)
setInput('external-repo-token', 'gh_viptoken')

const config = await readConfig()

expect(config.fail_on_severity).toEqual('high')
expect(config.allow_licenses).toEqual(['GPL-2.0-only'])
})
2 changes: 1 addition & 1 deletion __tests__/fixtures/inline-license-config-sample.yml
@@ -1 +1 @@
allow-licenses: MIT, GPL-2.0-only
allow-licenses: "MIT, GPL-2.0-only"
4 changes: 2 additions & 2 deletions __tests__/fixtures/invalid-severity-config.yml
@@ -1,3 +1,3 @@
fail-on-severity: 'so many zombies'
deny-licenses:
fail_on_severity: 'so many zombies'
deny_licenses:
- MIT
28 changes: 28 additions & 0 deletions __tests__/test-helpers.ts
@@ -0,0 +1,28 @@
// GitHub Action inputs come in the form of environment variables
// with an INPUT prefix (e.g. INPUT_FAIL-ON-SEVERITY)
export function setInput(input: string, value: string): void {
process.env[`INPUT_${input.toUpperCase()}`] = value
}

// We want a clean ENV before each test. We use `delete`
// since we want `undefined` values and not empty strings.
export function clearInputs(): void {
const allowedOptions = [
'FAIL-ON-SEVERITY',
'FAIL-ON-SCOPES',
'ALLOW-LICENSES',
'DENY-LICENSES',
'ALLOW-GHSAS',
'LICENSE-CHECK',
'VULNERABILITY-CHECK',
'CONFIG-FILE',
'BASE-REF',
'HEAD-REF',
'COMMENT-SUMMARY-IN-PR'
]

// eslint-disable-next-line github/array-foreach
allowedOptions.forEach(option => {
delete process.env[`INPUT_${option.toUpperCase()}`]
})
}
4 changes: 2 additions & 2 deletions action.yml
@@ -1,3 +1,5 @@
# Avoid using default values for options here since they will
# end up overriding external configurations.
name: 'Dependency Review'
description: 'Prevent the introduction of dependencies with known vulnerabilities'
author: 'GitHub'
Expand All @@ -9,11 +11,9 @@ inputs:
fail-on-severity:
description: Don't block PRs below this severity. Possible values are `low`, `moderate`, `high`, `critical`.
required: false
default: 'low'
fail-on-scopes:
description: Dependency scopes to block PRs on. Comma-separated list. Possible values are 'unknown', 'runtime', and 'development' (e.g. "runtime, development")
required: false
default: 'runtime'
base-ref:
description: The base git ref to be used for this check. Has a default value when the workflow event is `pull_request` or `pull_request_target`. Must be provided otherwise.
required: false
Expand Down