Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: github/codeql-action
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.1.25
Choose a base ref
...
head repository: github/codeql-action
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.1.26
Choose a head ref

Commits on Sep 21, 2022

  1. Update changelog and version after v2.1.25

    github-actions[bot] committed Sep 21, 2022
    Copy the full SHA
    4706007 View commit details
  2. Update checked-in dependencies

    github-actions[bot] committed Sep 21, 2022
    Copy the full SHA
    6188cbd View commit details
  3. Merge pull request #1263 from github/mergeback/v2.1.25-to-main-86f3159a

    Mergeback v2.1.25 refs/heads/releases/v2 into main
    henrymercer authored Sep 21, 2022
    Copy the full SHA
    a643eb3 View commit details

Commits on Sep 23, 2022

  1. Copy the full SHA
    b98b2de View commit details
  2. Merge pull request #1265 from github/edoardo/minimum-cache-size

    TRAP Caching: Skip uploading of small caches
    edoardopirovano authored Sep 23, 2022
    Copy the full SHA
    4c8f137 View commit details

Commits on Sep 26, 2022

  1. Fix broken unit test

    edoardopirovano committed Sep 26, 2022
    Copy the full SHA
    b96c754 View commit details
  2. Merge pull request #1266 from github/edoardo/fix-unit-test

    Fix broken unit test
    edoardopirovano authored Sep 26, 2022
    Copy the full SHA
    314ede6 View commit details

Commits on Sep 27, 2022

  1. Add the CliConfigFileEnabled feature flag

    Also, wire it up to the `useCodeScanningConfigInCli` function.
    aeisenberg committed Sep 27, 2022
    Copy the full SHA
    e37b0d6 View commit details
  2. Update src/util.test.ts

    aeisenberg committed Sep 27, 2022
    Copy the full SHA
    cab46c5 View commit details
  3. Update unset environment variables PR check (#1269)

    * Only test Java for CLI v2.5+
    
    * Improve bash code style
    
    * Set Actions error messages
    
    Co-authored-by: Andrew Eisenberg <aeisenberg@github.com>
    angelapwen and aeisenberg authored Sep 27, 2022
    Copy the full SHA
    1f0700d View commit details

Commits on Sep 28, 2022

  1. Copy the full SHA
    cc4ee05 View commit details
  2. Correct program name

    jsoref committed Sep 28, 2022
    Copy the full SHA
    a4e4529 View commit details
  3. Merge pull request #1268 from jsoref/rev-parse

    Correct program name
    edoardopirovano authored Sep 28, 2022
    Copy the full SHA
    28c63d1 View commit details
  4. Merge pull request #1272 from github/update-supported-enterprise-serv…

    …er-versions
    
    Update supported GitHub Enterprise Server versions.
    henrymercer authored Sep 28, 2022
    Copy the full SHA
    39064e0 View commit details
  5. Copy the full SHA
    a711c76 View commit details
  6. Merge pull request #1267 from github/cklin/codeql-cli-2.11.0

    Update default CodeQL version to 2.11.0
    cklin authored Sep 28, 2022
    Copy the full SHA
    2466f0c View commit details
  7. Copy the full SHA
    1ec8ea9 View commit details
  8. Merge pull request #1270 from github/aeisenberg/cli-config-feature-flag

    Add the `CliConfigFileEnabled` feature flag
    aeisenberg authored Sep 28, 2022
    Copy the full SHA
    74740ee View commit details
  9. Copy the full SHA
    3400e51 View commit details
  10. Update the checks script

    Avoid requiring test-setup-python-scripts checks.
    aeisenberg committed Sep 28, 2022
    Copy the full SHA
    fb0f747 View commit details

Commits on Sep 29, 2022

  1. Merge pull request #1274 from github/aeisenberg/update-checks-script

    Update the checks script
    aeisenberg authored Sep 29, 2022
    Copy the full SHA
    9705474 View commit details
  2. Update changelog for v2.1.26

    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    c60b854 View commit details
  3. Merge pull request #1276 from github/update-v2.1.26-97054749

    Merge main into releases/v2
    cklin authored Sep 29, 2022
    Copy the full SHA
    e0e5ded View commit details
  4. Revert "Update version and changelog for v1.1.25"

    This reverts commit e4a1505.
    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    c9559f9 View commit details
  5. Revert "Update checked-in dependencies"

    This reverts commit b27be7b.
    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    af3f782 View commit details
  6. Merge remote-tracking branch 'origin/releases/v2' into update-v1.1.26…

    …-e0e5ded3
    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    d6b82d8 View commit details
  7. Update version and changelog for v1.1.26

    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    7e79eb6 View commit details
  8. Update checked-in dependencies

    github-actions[bot] committed Sep 29, 2022
    Copy the full SHA
    6548772 View commit details
  9. Merge pull request #1278 from github/update-v1.1.26-e0e5ded3

    Merge releases/v2 into releases/v1
    cklin authored Sep 29, 2022
    Copy the full SHA
    64b7bbc View commit details
Showing with 528 additions and 232 deletions.
  1. +1 −0 .github/workflows/__go-custom-queries.yml
  2. +0 −97 .github/workflows/__unset-environment.yml
  3. +1 −1 .github/workflows/script/update-required-checks.sh
  4. +95 −0 .github/workflows/unset-environment-new-cli.yml
  5. +89 −0 .github/workflows/unset-environment-old-cli.yml
  6. +4 −0 CHANGELOG.md
  7. +1 −1 lib/actions-util.js
  8. +1 −0 lib/analyze-action-env.test.js
  9. +1 −1 lib/analyze-action-env.test.js.map
  10. +1 −0 lib/analyze-action-input.test.js
  11. +1 −1 lib/analyze-action-input.test.js.map
  12. +1 −1 lib/analyze-action.js
  13. +1 −1 lib/analyze-action.js.map
  14. +2 −2 lib/analyze.js
  15. +1 −1 lib/analyze.js.map
  16. +3 −2 lib/analyze.test.js
  17. +1 −1 lib/analyze.test.js.map
  18. +1 −1 lib/api-compatibility.json
  19. +3 −3 lib/codeql.js
  20. +1 −1 lib/codeql.js.map
  21. +1 −1 lib/config-utils.js
  22. +1 −1 lib/config-utils.js.map
  23. +1 −1 lib/defaults.json
  24. +1 −0 lib/feature-flags.js
  25. +1 −1 lib/feature-flags.js.map
  26. +1 −1 lib/runner.js
  27. +1 −1 lib/runner.js.map
  28. +16 −17 lib/trap-caching.js
  29. +1 −1 lib/trap-caching.js.map
  30. +1 −0 lib/trap-caching.test.js
  31. +1 −1 lib/trap-caching.test.js.map
  32. +40 −5 lib/util.js
  33. +1 −1 lib/util.js.map
  34. +35 −0 lib/util.test.js
  35. +1 −1 lib/util.test.js.map
  36. +1 −1 node_modules/.package-lock.json
  37. +2 −2 package-lock.json
  38. +1 −1 package.json
  39. +2 −0 pr-checks/checks/go-custom-queries.yml
  40. +0 −49 pr-checks/checks/unset-environment.yml
  41. +1 −1 src/actions-util.ts
  42. +1 −0 src/analyze-action-env.test.ts
  43. +1 −0 src/analyze-action-input.test.ts
  44. +3 −1 src/analyze-action.ts
  45. +5 −2 src/analyze.test.ts
  46. +3 −2 src/analyze.ts
  47. +1 −1 src/api-compatibility.json
  48. +8 −3 src/codeql.ts
  49. +1 −1 src/config-utils.ts
  50. +1 −1 src/defaults.json
  51. +1 −0 src/feature-flags.ts
  52. +2 −1 src/runner.ts
  53. +1 −0 src/trap-caching.test.ts
  54. +24 −14 src/trap-caching.ts
  55. +109 −0 src/util.test.ts
  56. +48 −6 src/util.ts
1 change: 1 addition & 0 deletions .github/workflows/__go-custom-queries.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

97 changes: 0 additions & 97 deletions .github/workflows/__unset-environment.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/script/update-required-checks.sh
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@ fi
echo "Getting checks for $GITHUB_SHA"

# Ignore any checks with "https://", CodeQL, LGTM, and Update checks.
CHECKS="$(gh api repos/github/codeql-action/commits/"${GITHUB_SHA}"/check-runs --paginate | jq --slurp --compact-output --raw-output '[.[].check_runs | .[].name | select(contains("https://") or . == "CodeQL" or . == "LGTM.com" or contains("Update") or contains("update") | not)] | unique | sort')"
CHECKS="$(gh api repos/github/codeql-action/commits/"${GITHUB_SHA}"/check-runs --paginate | jq --slurp --compact-output --raw-output '[.[].check_runs | .[].name | select(contains("https://") or . == "CodeQL" or . == "LGTM.com" or contains("Update") or contains("update") or contains("test-setup-python-scripts") | not)] | unique | sort')"

echo "$CHECKS" | jq

95 changes: 95 additions & 0 deletions .github/workflows/unset-environment-new-cli.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# See `unset-environment-old-cli.yml` for reasoning behind the separate tests.
name: PR Check - Test unsetting environment variables for CLI version >= 2.5.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v1
- releases/v2
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
workflow_dispatch: {}
jobs:
unset-environment:
strategy:
matrix:
include:
- os: ubuntu-latest
version: stable-20210809
- os: ubuntu-latest
version: cached
- os: ubuntu-latest
version: latest
- os: ubuntu-latest
version: nightly-latest
name: Test unsetting environment variables
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@v3
- name: Prepare test
id: prepare-test
uses: ./.github/prepare-test
with:
version: ${{ matrix.version }}
- uses: ./../action/init
with:
db-location: ${{ runner.temp }}/customDbLocation
tools: ${{ steps.prepare-test.outputs.tools-url }}
env:
TEST_MODE: true
- name: Build code
shell: bash
run: env -i PATH="$PATH" HOME="$HOME" ./build.sh
- uses: ./../action/analyze
id: analysis
env:
TEST_MODE: true
- shell: bash
run: |
CPP_DB="${{ fromJson(steps.analysis.outputs.db-locations).cpp }}"
if [[ ! -d "$CPP_DB" ]] || [[ ! "$CPP_DB" == "${RUNNER_TEMP}/customDbLocation/cpp" ]]; then
echo "::error::Did not create a database for CPP, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/cpp' but actual was '${CPP_DB}'"
exit 1
fi
CSHARP_DB="${{ fromJson(steps.analysis.outputs.db-locations).csharp }}"
if [[ ! -d "$CSHARP_DB" ]] || [[ ! "$CSHARP_DB" == "${RUNNER_TEMP}/customDbLocation/csharp" ]]; then
echo "::error::Did not create a database for C Sharp, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/csharp' but actual was '${CSHARP_DB}'"
exit 1
fi
GO_DB="${{ fromJson(steps.analysis.outputs.db-locations).go }}"
if [[ ! -d "$GO_DB" ]] || [[ ! "$GO_DB" == "${RUNNER_TEMP}/customDbLocation/go" ]]; then
echo "::error::Did not create a database for Go, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/go' but actual was '${GO_DB}'"
exit 1
fi
JAVA_DB="${{ fromJson(steps.analysis.outputs.db-locations).java }}"
if [[ ! -d "$JAVA_DB" ]] || [[ ! "$JAVA_DB" == "${RUNNER_TEMP}/customDbLocation/java" ]]; then
echo "::error::Did not create a database for Java, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/java' but actual was '${JAVA_DB}'"
exit 1
fi
JAVASCRIPT_DB="${{ fromJson(steps.analysis.outputs.db-locations).javascript }}"
if [[ ! -d "$JAVASCRIPT_DB" ]] || [[ ! "$JAVASCRIPT_DB" == "${RUNNER_TEMP}/customDbLocation/javascript" ]]; then
echo "::error::Did not create a database for Javascript, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/javascript' but actual was '${JAVASCRIPT_DB}'"
exit 1
fi
PYTHON_DB="${{ fromJson(steps.analysis.outputs.db-locations).python }}"
if [[ ! -d "$PYTHON_DB" ]] || [[ ! "$PYTHON_DB" == "${RUNNER_TEMP}/customDbLocation/python" ]]; then
echo "::error::Did not create a database for Python, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/python' but actual was '${PYTHON_DB}'"
exit 1
fi
env:
INTERNAL_CODEQL_ACTION_DEBUG_LOC: true
89 changes: 89 additions & 0 deletions .github/workflows/unset-environment-old-cli.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# There was a bug, fixed in CLI v2.5.1, that didn't propagate environment
# variables that the Java tracer needed. Here we test all languages
# except Java for these CLI versions. In `unset-environment-new-cli.yml`
# we test all languages for recent CLI versions.
name: PR Check - Test unsetting environment variables for CLI version < 2.5.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GO111MODULE: auto
on:
push:
branches:
- main
- releases/v1
- releases/v2
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
workflow_dispatch: {}
jobs:
unset-environment:
strategy:
matrix:
include:
- os: ubuntu-latest
version: stable-20210308
- os: ubuntu-latest
version: stable-20210319
name: Test unsetting environment variables
timeout-minutes: 45
runs-on: ${{ matrix.os }}
steps:
- name: Check out repository
uses: actions/checkout@v3
- name: Prepare test
id: prepare-test
uses: ./.github/prepare-test
with:
version: ${{ matrix.version }}
- uses: ./../action/init
with:
languages: csharp,cpp,go,javascript,python
db-location: ${{ runner.temp }}/customDbLocation
tools: ${{ steps.prepare-test.outputs.tools-url }}
env:
TEST_MODE: true
- name: Build code
shell: bash
run: env -i PATH="$PATH" HOME="$HOME" ./build.sh
- uses: ./../action/analyze
id: analysis
env:
TEST_MODE: true
- shell: bash
run: |
CPP_DB="${{ fromJson(steps.analysis.outputs.db-locations).cpp }}"
if [[ ! -d "$CPP_DB" ]] || [[ ! "$CPP_DB" == "${RUNNER_TEMP}/customDbLocation/cpp" ]]; then
echo "::error::Did not create a database for CPP, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/cpp' but actual was '${CPP_DB}'"
exit 1
fi
CSHARP_DB="${{ fromJson(steps.analysis.outputs.db-locations).csharp }}"
if [[ ! -d "$CSHARP_DB" ]] || [[ ! "$CSHARP_DB" == "${RUNNER_TEMP}/customDbLocation/csharp" ]]; then
echo "::error::Did not create a database for C Sharp, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/csharp' but actual was '${CSHARP_DB}'"
exit 1
fi
GO_DB="${{ fromJson(steps.analysis.outputs.db-locations).go }}"
if [[ ! -d "$GO_DB" ]] || [[ ! "$GO_DB" == "${RUNNER_TEMP}/customDbLocation/go" ]]; then
echo "::error::Did not create a database for Go, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/go' but actual was '${GO_DB}'"
exit 1
fi
JAVASCRIPT_DB="${{ fromJson(steps.analysis.outputs.db-locations).javascript }}"
if [[ ! -d "$JAVASCRIPT_DB" ]] || [[ ! "$JAVASCRIPT_DB" == "${RUNNER_TEMP}/customDbLocation/javascript" ]]; then
echo "::error::Did not create a database for Javascript, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/javascript' but actual was '${JAVASCRIPT_DB}'"
exit 1
fi
PYTHON_DB="${{ fromJson(steps.analysis.outputs.db-locations).python }}"
if [[ ! -d "$PYTHON_DB" ]] || [[ ! "$PYTHON_DB" == "${RUNNER_TEMP}/customDbLocation/python" ]]; then
echo "::error::Did not create a database for Python, or created it in the wrong location." \
"Expected location was '${RUNNER_TEMP}/customDbLocation/python' but actual was '${PYTHON_DB}'"
exit 1
fi
env:
INTERNAL_CODEQL_ACTION_DEBUG_LOC: true
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# CodeQL Action Changelog

## 1.1.26 - 29 Sep 2022

- Update default CodeQL bundle version to 2.11.0. [#1267](https://github.com/github/codeql-action/pull/1267)

## 1.1.25 - 21 Sep 2022

- We will soon be rolling out a feature of the CodeQL Action that stores some information used to make future runs faster in the GitHub Actions cache. Initially, this will only be enabled on JavaScript repositories, but we plan to add more languages to this soon. The new feature can be disabled by passing the `trap-caching: false` option to your workflow's `init` step, for example if you are already using the GitHub Actions cache for a different purpose and are near the storage limit for it.
2 changes: 1 addition & 1 deletion lib/actions-util.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions lib/analyze-action-env.test.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/analyze-action-env.test.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions lib/analyze-action-input.test.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/analyze-action-input.test.js.map
2 changes: 1 addition & 1 deletion lib/analyze-action.js
2 changes: 1 addition & 1 deletion lib/analyze-action.js.map
4 changes: 2 additions & 2 deletions lib/analyze.js
2 changes: 1 addition & 1 deletion lib/analyze.js.map

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions lib/analyze.test.js
2 changes: 1 addition & 1 deletion lib/analyze.test.js.map

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/api-compatibility.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{ "maximumVersion": "3.7", "minimumVersion": "3.2" }
{ "maximumVersion": "3.7", "minimumVersion": "3.3" }
6 changes: 3 additions & 3 deletions lib/codeql.js
2 changes: 1 addition & 1 deletion lib/codeql.js.map

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/config-utils.js
2 changes: 1 addition & 1 deletion lib/config-utils.js.map

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/defaults.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"bundleVersion": "codeql-bundle-20220908"
"bundleVersion": "codeql-bundle-20220923"
}
1 change: 1 addition & 0 deletions lib/feature-flags.js
2 changes: 1 addition & 1 deletion lib/feature-flags.js.map
2 changes: 1 addition & 1 deletion lib/runner.js
2 changes: 1 addition & 1 deletion lib/runner.js.map

Large diffs are not rendered by default.

33 changes: 16 additions & 17 deletions lib/trap-caching.js
2 changes: 1 addition & 1 deletion lib/trap-caching.js.map
1 change: 1 addition & 0 deletions lib/trap-caching.test.js
2 changes: 1 addition & 1 deletion lib/trap-caching.test.js.map
45 changes: 40 additions & 5 deletions lib/util.js
2 changes: 1 addition & 1 deletion lib/util.js.map

Large diffs are not rendered by default.

35 changes: 35 additions & 0 deletions lib/util.test.js
2 changes: 1 addition & 1 deletion lib/util.test.js.map

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion node_modules/.package-lock.json
4 changes: 2 additions & 2 deletions package-lock.json
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "1.1.25",
"version": "1.1.26",
"private": true,
"description": "CodeQL action",
"scripts": {
2 changes: 2 additions & 0 deletions pr-checks/checks/go-custom-queries.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
name: "Go: Custom queries"
description: "Checks that Go works in conjunction with a config file specifying custom queries"
env:
DOTNET_GENERATE_ASPNET_CERTIFICATE: "false"
steps:
- uses: actions/setup-go@v3
with:
49 changes: 0 additions & 49 deletions pr-checks/checks/unset-environment.yml

This file was deleted.

2 changes: 1 addition & 1 deletion src/actions-util.ts
Original file line number Diff line number Diff line change
@@ -545,7 +545,7 @@ export async function getRef(): Promise<string> {
// in actions/checkout@v1 this may not be true as it checks out the repository
// using GITHUB_REF. There is a subtle race condition where
// git rev-parse GITHUB_REF != GITHUB_SHA, so we must check
// git git-parse GITHUB_REF == git rev-parse HEAD instead.
// git rev-parse GITHUB_REF == git rev-parse HEAD instead.
const hasChangedRef =
sha !== head &&
(await getCommitOid(
1 change: 1 addition & 0 deletions src/analyze-action-env.test.ts
Original file line number Diff line number Diff line change
@@ -36,6 +36,7 @@ test("analyze action with RAM & threads from environment variables", async (t) =
gitHubVersion,
languages: [],
packs: [],
trapCaches: {},
} as unknown as configUtils.Config);
const requiredInputStub = sinon.stub(actionsUtil, "getRequiredInput");
requiredInputStub.withArgs("token").returns("fake-token");
1 change: 1 addition & 0 deletions src/analyze-action-input.test.ts
Original file line number Diff line number Diff line change
@@ -36,6 +36,7 @@ test("analyze action with RAM & threads from action inputs", async (t) => {
gitHubVersion,
languages: [],
packs: [],
trapCaches: {},
} as unknown as configUtils.Config);
const requiredInputStub = sinon.stub(actionsUtil, "getRequiredInput");
requiredInputStub.withArgs("token").returns("fake-token");
4 changes: 3 additions & 1 deletion src/analyze-action.ts
Original file line number Diff line number Diff line change
@@ -245,6 +245,7 @@ async function run() {
logger,
featureFlags
);

if (actionsUtil.getRequiredInput("skip-queries") !== "true") {
runStats = await runQueries(
outputDir,
@@ -253,7 +254,8 @@ async function run() {
threads,
actionsUtil.getOptionalInput("category"),
config,
logger
logger,
featureFlags
);
}

7 changes: 5 additions & 2 deletions src/analyze.test.ts
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@ import {
import { setCodeQL } from "./codeql";
import { Config } from "./config-utils";
import * as count from "./count-loc";
import { createFeatureFlags } from "./feature-flags";
import { Language } from "./languages";
import { getRunnerLogger } from "./logging";
import { setupTests, setupActionsVars } from "./testing-utils";
@@ -138,7 +139,8 @@ test("status report fields and search path setting", async (t) => {
threadsFlag,
undefined,
config,
getRunnerLogger(true)
getRunnerLogger(true),
createFeatureFlags([])
);
const hasPacks = language in packs;
const statusReportKeys = Object.keys(builtinStatusReport).sort();
@@ -187,7 +189,8 @@ test("status report fields and search path setting", async (t) => {
threadsFlag,
undefined,
config,
getRunnerLogger(true)
getRunnerLogger(true),
createFeatureFlags([])
);
t.deepEqual(Object.keys(customStatusReport).length, 2);
t.true(
5 changes: 3 additions & 2 deletions src/analyze.ts
Original file line number Diff line number Diff line change
@@ -213,7 +213,8 @@ export async function runQueries(
threadsFlag: string,
automationDetailsId: string | undefined,
config: configUtils.Config,
logger: Logger
logger: Logger,
featureFlags: FeatureFlags
): Promise<QueriesStatusReport> {
const statusReport: QueriesStatusReport = {};

@@ -256,7 +257,7 @@ export async function runQueries(
}

try {
if (await util.useCodeScanningConfigInCli(codeql)) {
if (await util.useCodeScanningConfigInCli(codeql, featureFlags)) {
// If we are using the codescanning config in the CLI,
// much of the work needed to generate the query suites
// is done in the CLI. We just need to make a single
2 changes: 1 addition & 1 deletion src/api-compatibility.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"maximumVersion": "3.7", "minimumVersion": "3.2"}
{"maximumVersion": "3.7", "minimumVersion": "3.3"}
11 changes: 8 additions & 3 deletions src/codeql.ts
Original file line number Diff line number Diff line change
@@ -819,7 +819,11 @@ async function getCodeQLForCmd(
}
}

const configLocation = await generateCodescanningConfig(codeql, config);
const configLocation = await generateCodescanningConfig(
codeql,
config,
featureFlags
);
if (configLocation) {
extraArgs.push(`--codescanning-config=${configLocation}`);
}
@@ -1269,9 +1273,10 @@ async function runTool(cmd: string, args: string[] = []) {
*/
async function generateCodescanningConfig(
codeql: CodeQL,
config: Config
config: Config,
featureFlags: FeatureFlags
): Promise<string | undefined> {
if (!(await util.useCodeScanningConfigInCli(codeql))) {
if (!(await util.useCodeScanningConfigInCli(codeql, featureFlags))) {
return;
}
const configLocation = path.resolve(config.tempDir, "user-config.yaml");
2 changes: 1 addition & 1 deletion src/config-utils.ts
Original file line number Diff line number Diff line change
@@ -1704,7 +1704,7 @@ export async function initConfig(
// When using the codescanning config in the CLI, pack downloads
// happen in the CLI during the `database init` command, so no need
// to download them here.
if (!(await useCodeScanningConfigInCli(codeQL))) {
if (!(await useCodeScanningConfigInCli(codeQL, featureFlags))) {
const registries = parseRegistries(registriesInput);
await downloadPacks(
codeQL,
2 changes: 1 addition & 1 deletion src/defaults.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"bundleVersion": "codeql-bundle-20220908"
"bundleVersion": "codeql-bundle-20220923"
}
1 change: 1 addition & 0 deletions src/feature-flags.ts
Original file line number Diff line number Diff line change
@@ -12,6 +12,7 @@ export enum FeatureFlag {
MlPoweredQueriesEnabled = "ml_powered_queries_enabled",
TrapCachingEnabled = "trap_caching_enabled",
GolangExtractionReconciliationEnabled = "golang_extraction_reconciliation_enabled",
CliConfigFileEnabled = "cli_config_file_enabled",
}

/**
3 changes: 2 additions & 1 deletion src/runner.ts
Original file line number Diff line number Diff line change
@@ -517,7 +517,8 @@ program
threads,
cmd.category,
config,
logger
logger,
createFeatureFlags([])
);

if (!cmd.upload) {
1 change: 1 addition & 0 deletions src/trap-caching.test.ts
Original file line number Diff line number Diff line change
@@ -165,6 +165,7 @@ test("upload cache key contains right fields", async (t) => {
const loggedMessages = [];
const logger = getRecordingLogger(loggedMessages);
sinon.stub(actionsUtil, "isAnalyzingDefaultBranch").resolves(true);
sinon.stub(util, "tryGetFolderBytes").resolves(999_999_999);
const stubSave = sinon.stub(cache, "saveCache");
process.env.GITHUB_SHA = "somesha";
await uploadTrapCaches(stubCodeql, testConfigWithoutTmpDir, logger);
38 changes: 24 additions & 14 deletions src/trap-caching.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
import * as fs from "fs";
import * as path from "path";
import { promisify } from "util";

import * as cache from "@actions/cache";
import getFolderSize from "get-folder-size";

import * as actionsUtil from "./actions-util";
import { CodeQL, CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES } from "./codeql";
import { Config } from "./config-utils";
import { Language } from "./languages";
import { Logger } from "./logging";
import { codeQlVersionAbove } from "./util";
import { codeQlVersionAbove, tryGetFolderBytes } from "./util";

// This constant should be bumped if we make a breaking change
// to how the CodeQL Action stores or retrieves the TRAP cache,
@@ -22,6 +20,10 @@ const CACHE_VERSION = 1;
// This constant sets the size of each TRAP cache in megabytes.
const CACHE_SIZE_MB = 1024;

// This constant sets the minimum size in megabytes of a TRAP
// cache for us to consider it worth uploading.
const MINIMUM_CACHE_MB_TO_UPLOAD = 10;

export async function getTrapCachingExtractorConfigArgs(
config: Config
): Promise<string[]> {
@@ -138,6 +140,19 @@ export async function uploadTrapCaches(
for (const language of config.languages) {
const cacheDir = config.trapCaches[language];
if (cacheDir === undefined) continue;
const trapFolderSize = await tryGetFolderBytes(cacheDir, logger);
if (trapFolderSize === undefined) {
logger.info(
`Skipping upload of TRAP cache for ${language} as we couldn't determine its size`
);
continue;
}
if (trapFolderSize < MINIMUM_CACHE_MB_TO_UPLOAD * 1_048_576) {
logger.info(
`Skipping upload of TRAP cache for ${language} as it is too small`
);
continue;
}
const key = await cacheKey(
codeql,
language,
@@ -201,17 +216,12 @@ export async function getTotalCacheSize(
trapCaches: Partial<Record<Language, string>>,
logger: Logger
): Promise<number> {
try {
const sizes = await Promise.all(
Object.values(trapCaches).map(async (cacheDir) => {
return promisify<string, number>(getFolderSize)(cacheDir);
})
);
return sizes.reduce((a, b) => a + b, 0);
} catch (e) {
logger.warning(`Encountered an error while getting TRAP cache size: ${e}`);
return 0;
}
const sizes = await Promise.all(
Object.values(trapCaches).map((cacheDir) =>
tryGetFolderBytes(cacheDir, logger)
)
);
return sizes.map((a) => a || 0).reduce((a, b) => a + b, 0);
}

async function cacheKey(
109 changes: 109 additions & 0 deletions src/util.test.ts
Original file line number Diff line number Diff line change
@@ -9,7 +9,9 @@ import test, { ExecutionContext } from "ava";
import * as sinon from "sinon";

import * as api from "./api-client";
import { CodeQL } from "./codeql";
import { Config } from "./config-utils";
import { createFeatureFlags, FeatureFlag } from "./feature-flags";
import { getRunnerLogger, Logger } from "./logging";
import { setupTests } from "./testing-utils";
import * as util from "./util";
@@ -492,3 +494,110 @@ test("listFolder", async (t) => {
]);
});
});

test("useCodeScanningConfigInCli with no env var", async (t) => {
t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
))
);

// Yay! It works!
t.assert(
await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
)
);
});

for (const val of ["TRUE", "true", "True"]) {
test(`useCodeScanningConfigInCli with env var ${val}`, async (t) => {
process.env[util.EnvVar.CODEQL_PASS_CONFIG_TO_CLI] = val;
t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
))
);

// Yay! It works!
t.assert(
await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
)
);

t.assert(
await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([])
)
);
});
}

for (const val of ["FALSE", "false", "False"]) {
test(`useCodeScanningConfigInCli with env var ${val}`, async (t) => {
// Never turned on when env var is false
process.env[util.EnvVar.CODEQL_PASS_CONFIG_TO_CLI] = val;
t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.0"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([FeatureFlag.CliConfigFileEnabled])
))
);

t.assert(
!(await util.useCodeScanningConfigInCli(
mockVersion("2.10.1"),
createFeatureFlags([])
))
);
});
}

function mockVersion(version) {
return {
async getVersion() {
return version;
},
} as CodeQL;
}
54 changes: 48 additions & 6 deletions src/util.ts
Original file line number Diff line number Diff line change
@@ -2,9 +2,11 @@ import * as fs from "fs";
import * as os from "os";
import * as path from "path";
import { Readable } from "stream";
import { promisify } from "util";

import * as core from "@actions/core";
import del from "del";
import getFolderSize from "get-folder-size";
import * as semver from "semver";

import * as api from "./api-client";
@@ -487,7 +489,7 @@ export enum Mode {
* CLI. These environment variables are relevant for both the runner
* and the action.
*/
enum EnvVar {
export enum EnvVar {
/**
* The mode of the codeql-action, either 'actions' or 'runner'.
*/
@@ -591,6 +593,10 @@ export function getRequiredEnvParam(paramName: string): string {
return value;
}

function getOptionalEnvParam(paramName: string): string {
return process.env[paramName] || "";
}

export class HTTPError extends Error {
public status: number;

@@ -787,12 +793,28 @@ export function isInTestMode(): boolean {
* that gets passed to the CLI.
*/
export async function useCodeScanningConfigInCli(
codeql: CodeQL
codeql: CodeQL,
featureFlags: FeatureFlags
): Promise<boolean> {
return (
process.env[EnvVar.CODEQL_PASS_CONFIG_TO_CLI] === "true" &&
(await codeQlVersionAbove(codeql, CODEQL_VERSION_CONFIG_FILES))
);
const envVarIsEnabled = getOptionalEnvParam(EnvVar.CODEQL_PASS_CONFIG_TO_CLI);

// If the user has explicitly turned off the feature, then don't use it.
if (envVarIsEnabled.toLocaleLowerCase() === "false") {
return false;
}

// If the user has explicitly turned on the feature, then use it.
// Or if the feature flag is enabled, then use it.
const isEnabled =
envVarIsEnabled.toLocaleLowerCase() === "true" ||
(await featureFlags.getValue(FeatureFlag.CliConfigFileEnabled));

if (!isEnabled) {
return false;
}

// If the CLI version is too old, then don't use it.
return await codeQlVersionAbove(codeql, CODEQL_VERSION_CONFIG_FILES);
}

/*
@@ -836,3 +858,23 @@ export async function isGoExtractionReconciliationEnabled(
))
);
}

/**
* Get the size a folder in bytes. This will log any filesystem errors
* as a warning and then return undefined.
*
* @param cacheDir A directory to get the size of.
* @param logger A logger to log any errors to.
* @returns The size in bytes of the folder, or undefined if errors occurred.
*/
export async function tryGetFolderBytes(
cacheDir: string,
logger: Logger
): Promise<number | undefined> {
try {
return await promisify<string, number>(getFolderSize)(cacheDir);
} catch (e) {
logger.warning(`Encountered an error while getting size of folder: ${e}`);
return undefined;
}
}