Skip to content

Latest commit

 

History

History
419 lines (237 loc) · 19.1 KB

CI-CD_DOCUMENTATION.md

File metadata and controls

419 lines (237 loc) · 19.1 KB
Raw

Notepads CI/CD documentation

  • after merging the PR, the first run of the "Notepads CI/CD Pipeline" workflow will not complete successfully, because it requires specific setup explained in this documentation. The two other workflows "CodeQL Analysis" and "Build", should complete successfully.

1. Set up SonarCloud

SonarCloud is a cloud-based code quality and security service

Create SonarCloud project

  • Go to https://sonarcloud.io/

  • Click the "Log in" button and create a new account or connect with GitHub account (recommended)

  • At the top right corner click the "+" sign

  • From the dropdown select "Create new Organization"

  • Click the "Choose an organization on GitHub" button

  • Select an account for the organization setup

  • On Repository Access select "Only select repositories" and select the project and click the "Save" button

  • On the "Create organization page" don't change the Key and click "Continue"

  • Select the Free plan then click the "Create Organization" button to finalize the creation of the Organization

Configure SonarCloud project

  • At the top right corner click the "+" sign and select "Analyze new project"

  • Select the project and click the "Set Up" button in the box on the right

  • Under "Choose your analysis method" click "With GitHub Actions" and keep the following page open

  • Create a new PAT with repo_deployment and read:packages permissions and copy the value of the generated token

  • In the project's GitHub repository, go to the Settings tab -> Secrets

  • Click on New Repository secret and create a new secret with the name SONAR_GITHUB_TOKEN and the token you just copied as the value

  • Create another secret with the two values from the SonarCloud page you kept open, which you can close after completing this step

SonarCloud_1

Set Quality Gate

  • After the "Notepads CI/CD Pipeline" workflow has executed successfully, go to https://sonarcloud.io/projects and click on the project

  • In the alert bar above the results, click the "Set new code definition" button and select "Previous version" (notice the "New Code definition has been updated" alert at the top)

  • The Quality Gate will become active as soon as the next SonarCloud scan completes successfully


2. Run workflow manually

Once you've set up all the steps above correctly, you should be able to successfully complete a manual execution of the "Notepads CI/CD Pipeline" workflow.

  1. Go to the project's GitHub repository and click on the Actions tab

  2. From the "Workflows" list on the left, click on "Notepads CI/CD Pipeline"

  3. On the right, next to the "This workflow has a workflow_dispatch event trigger" label, click on the "Run workflow" dropdown, make sure the default branch is selected (if not manually changed, should be main or master) in the "Use workflow from" dropdown and click the "Run workflow" button

  4. You can optionally fill the argument textbox with "release" to trigger GitHub Release and Store Upload

Actions_workflow_dispatch

  1. Once the workflow run has completed successfully, move on to the next step of the documentation

NOTE: screenshots are only exemplary


3. Set up Dependabot

Dependabot is a GitHub native security tool that goes through the dependencies in the project and creates alerts, and PRs with updates when a new and/or non-vulnerable version is found.

  • for PRs with version updates, this pipeline comes pre-configured for all current dependency sources in the project, so at "Insights" tab -> "Dependency graph" -> "Dependabot", you should be able to see all tracked sources of dependencies, when they have been checked last and view a full log of the last check

Dependabot_tab

Dependabot_log_page

Set up security alerts and updates

- GitHub, through Dependabot, also natively offers a security check for vulnerable dependencies

  1. Go to "Settings" tab of the repo

  2. Go to "Security & analysis" section

  3. Click "Enable" for both "Dependabot alerts" and "Dependabot security updates"

  • By enabling "Dependabot alerts", you would be notified for any vulnerable dependencies in the project. At "Security" tab -> "Dependabot alerts", you can manage all alerts. By clicking on an alert, you would be able to see a detailed explanation of the vulnerability and a viable solution.

Dependabot_alerts_page

Dependabot_alert_page

  • By enabling "Dependabot security updates", you authorize Dependabot to create PRs specifically for security updates

Dependabot_PRs

Set up Dependency graph

- The "Dependency graph" option should be enabled by default for all public repos, but in case it isn't:

  1. Go to "Settings" tab of the repo

  2. Go to "Security&Analysis" section

  3. Click "Enable" for the "Dependency graph" option

  • this option enables the "Insights" tab -> "Dependency graph" section -> "Dependencies" tab, in which all the dependencies for the project are listed, under the different manifests they are included in

Dependabot_dependency_graph

NOTE: screenshots are only exemplary


4. CodeQL

CodeQL is GitHub's own industry-leading semantic code analysis engine. CodeQL requires no setup, because it comes fully pre-configured by us.

To activate it and see its results, only a push commit or a merge of a PR to the default branch of the repository, is required.

We've also configured CodeQL to run on schedule, so every day at 8:00AM UTC, it automatically scans the code.

  • you can see the results here at Security tab -> Code scanning alerts -> CodeQL:

CodeQL_results

  • on the page of each result, you can see an explanation of what the problem is and also one or more solutions:

CodeQL_alert_page

Code scanning alerts bulk dismissal tool

- currently, GitHub allows for only 25 code scanning alerts to be dismissed at a time. Sometimes, you might have hundreds you would like to dismiss, so you will have to click many times and wait for a long time to dismiss them. Via the "csa-bulk-dismissal.yml", you can automatically dismiss unnecessary alerts or manually do that with one click.

NOTE: This tool executes automatically when Notepads CI/CD Pipeline action completes.

1. Setup

  1. In the repository, go to the Settings tab -> Secrets

CSA_secrets

  1. Add the following secrets with the name and the corresponding value, by at the upper right of the section, clicking on the New repository secret button:

CSA_new_secret

CSA_secret_add

2. Execution

  1. This tool is automatically triggered when Notepads CI/CD Pipeline task completes, if you want to manually execute this follow next steps

  2. In your repo, click on the Actions tab and on the left, in the Workflows list, click on the "Code scanning alerts bulk dismissal"

CSA_execute_1

  1. On the right, click on the "Run workflow" dropdown. Under "Use workflow from" choose your default branch (usually main/master), in the Type of filter to use field type "path"/"desc" depending upon whether dismiss alerts based on predefined paths or description respectively (default is "path"), in the Reason for dismissal type "fp"/"wf"/"ut" for "false positive"/"won't fix"/"used in tests" respectively (default is "wf") and click on the Run workflow button

CSA_execute_2

NOTE: if any unsupported values are entered default values will be used

  1. If everything was set up currently in the "Setup" phase, the "Code scanning alerts bulk dismissal" workflow is going to be executed successfully, which after some time, would result in all previously open code scanning alerts, with a certain description be dismissed

CSA_execute_3

CSA_execute_4

CSA_execute_5

NOTE: "closed" refers to "dismissed" alerts

3. Customization

The "setup" job in the pipeline, allows for more precise filtering of alerts to bulk dismiss. It uses the filter type to choose (filter based on path or description) from the alert to determine if it has to be dismissed or not. We've added the following paths and alert descriptions by default:

Paths:
  • "*/obj/*" (if path contains obj folder at any position)
Descriptions:
  • "Calls to unmanaged code"
  • "Unmanaged code"
To add more paths, follow these steps:

  1. In your source code, open ".github/workflows/csa-bulk-dismissal.yml"

  2. From line 50 to 56, notice "$MATRIX = **". This is the powershell hashtable of filters that the CSABD (Code scanning alerts bulk dismissal) tool uses to filter through the alerts:

CSA_custom_3

  1. To add more paths under include element use comma separation and followed from next line add @{ filter = "New path" }. Replace "New path" with the path (with or without wild cards) you want:

CSA_custom_4

To add more descriptions, follow these steps:

  1. In your source code, open ".github/workflows/csa-bulk-dismissal.yml"

  2. From line 58 to 67, notice "$MATRIX = **". This is the powershell hashtable of filters that the CSABD (Code scanning alerts bulk dismissal) tool uses to filter through the alerts:

CSA_custom_1

  1. To add more descriptions under include element use comma separation and followed from next line add @{ filter = "New description" }. Replace "New description" with the description you want:

CSA_custom_2

To change default filter type and dismissal reason, follow these steps:

  1. In your source code, open ".github/workflows/csa-bulk-dismissal.yml"

  2. To change default filter type change $FILTER_TYPE variable in line 31 to something else (default is "path", supported are: "desc" and "path"):

CSA_custom_5

  1. To change dismissal reason change $REASON variable in line 45 to something else (default is "won't fix", supported are: "false positive", "won't fix" and "used in tests"):

CSA_custom_6

NOTE: changing default filter type and dismissal reason won't change dafault value typed when manually executing tool, change values in line 13 and 17 respectively to reflect the change

CSA_custom_7


5. Automated GitHub release

When triggered bumps up the GitHub tag in the repo and executes the CD job and produces release with changelogs

Note: not every commit to your master branch are included in changelog

Setup

Add the following secrets by going to the repo Settings tab -> Secrets:

  1. PACKAGE_CERTIFICATE_BASE64
  • used to dynamically create the PFX file required for the signing of the msixbundle
  • use the following PowerShell code locally to turn your PFX file into Base64:
# read from PFX as binary
$PFX_FILE = [IO.File]::ReadAllBytes('absolute_path_to_PFX')
# convert to Base64 and write in txt
[System.Convert]::ToBase64String($PFX_FILE) | Out-File 'absolute_path\cert.txt'
  • copy the contents of the cert.txt and paste as the value of the secret
  1. PACKAGE_CERTIFICATE_PWD
  • used in the build of the project to authenticate the PFX
  • copy and paste the password of your PFX as the value of this secret

NOTE:

  • none of those values are visible in the logs of the pipeline, nor are available to anyone outside of the original repository e.g. forks, anonymous clones etc.
  • the dynamically created PFX file lives only for the duration of the pipeline execution

Execution

Once you've set up all the steps for manual execution of the "Notepads CI/CD Pipeline" workflow correctly, you should be able to successfully trigger release with the same workflow.

  1. Go to the project's GitHub repository and click on the Actions tab

  2. From the "Workflows" list on the left, click on "Notepads CI/CD Pipeline"

  3. On the right, next to the "This workflow has a workflow_dispatch event trigger" label, click on the "Run workflow" dropdown, make sure the default branch is selected (if not manually changed, should be main or master) in the "Use workflow from" dropdown, type "release" in the argument textbox (By default "test" is typed) and click the "Run workflow" button

Actions_workflow_dispatch

  1. The workflow will produce release assets and calculate version, generate changelogs from valid commits since previous tag.

NOTE: screenshots are only exemplary


- follow these instructions for any commit (push or PR merge) to your master branch, you would like to see in changelog and count towards version change.

You would need one of three keywords at the start of your commit title. Each of the three keywords corresponds to a number in your release version i.e. v1.2.3. The release versioning uses the "Conventional Commits" specification:

  • "fix: ..." - this keyword corresponds to the last number v1.2.3, also known as PATCH;
  • "feat: ..." - this keyword corresponds to the middle number v1.2.3, also known as MINOR;
  • "perf: ..." - this keyword corresponds to the first number v1.2.3, also known as MAJOR. In addition, to trigger a MAJOR release, you would need to write "BREAKING CHANGE: ..." in the description of the commit, with an empty line above it to indicate it is in the portion of the description;

Note: when making a MAJOR release by committing through a terminal, use the multiple line syntax to add the commit title on one line and then adding an empty line, and then adding the "BREAKING CHANGE: " label

Examples

Example(fix/PATCH):
git commit -a -m "fix: this is a PATCH release triggering commit"
git push origin master

On triggering Release:
Result: v1.2.3 -> v1.2.4


Example(feat/MINOR):
git commit -a -m "feat: this is a MINOR release triggering commit"
git push origin master

On triggering Release:
Result: v1.2.3 -> v1.3.0


Example(perf/MAJOR):
git commit -a -m "perf: this is a MAJOR release triggering commit `
>>
>> BREAKING CHANGE: this is the breaking change"
git push origin master

On triggering Release:
Result: v1.2.3 -> v2.0.0

Note: in the MAJOR release example, the PowerShell multiline syntax ` (backtick) is used. After writing a backtick, a press of the Enter key should open a new line.


6. Setup automated publishing to the Windows Store

- for the automation to work, at least one submission needs to be already created manually

  1. AZURE_AD_TENANT_ID and AZURE_AD_APPLICATION_CLIENT_ID

    • copy and paste the values shown in the screenshot below to the appropriate secret:

Publish_to_store_1

Note: screenshot is taken from portal.azure.com -> Azure AD -> App registrations -> app-name page

  1. AZURE_AD_APPLICATION_SECRET

    • copy and paste the value you get on the page following from Account settings -> User management -> Azure AD applications -> click on the added application:

Publish_to_store_2

  1. STORE_APP_ID

    • copy and paste the highlighted code as the value of this secret:

Publish_to_store_3

  • If everything was setup correctly, on your next push commit to the master branch with a new Identity.Version in the Package.appxmanifest, a new submission in the Microsoft Partner Center with the new *.msixupload package should appear and be automatically submitted if all verifications pass

7. How to create a PAT

  • In a new tab open GitHub, at the top right corner, click on your profile picture and click on Settings from the dropdown.

    CSA_new_pat_1

  • Go to Developer Settings -> Personal access tokens.

    CSA_new_pat_2

    CSA_new_pat_3

  • Click the Generate new token button and enter password if prompted.

    CSA_new_pat_4

  • Name the token, from the permissions list choose the ones needed and at the bottom click on the Generate token button.

    CSA_new_pat_5

  • Copy the token value and paste it wherever its needed

    CSA_new_pat_6

NOTE: once you close or refresh the page, you won't be able to copy the value of the PAT again!

Built with ❤ by Pipeline Foundation