Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix code coverage #8663

Closed
wants to merge 11 commits into from
2 changes: 1 addition & 1 deletion build/ci/templates/steps/generate_upload_coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ steps:
summaryFileLocation: "$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml"
reportDirectory: "$(System.DefaultWorkingDirectory)/coverage"

- bash: cat ./coverage/lcov.info | ./node_modules/.bin/codecov --pipe
- bash: cat ./coverage/cobertura-coverage.xml | ./node_modules/.bin/codecov --pipe
displayName: 'Upload coverage to codecov'
continueOnError: true
condition: contains(variables['TestsToRun'], 'testUnitTests')
Expand Down
30 changes: 23 additions & 7 deletions build/ci/templates/steps/merge_upload_coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,32 @@ steps:
workingDirectory: $(Build.SourcesDirectory)
compile: 'false'

- task: DownloadBuildArtifacts@0
# - task: DownloadBuildArtifacts@0
# inputs:
# buildType: "current"
# allowPartiallySucceededBuilds: true
# downloadType: "Specific"
# itemPattern: "**/.nyc_output/**"
# downloadPath: "$(Build.SourcesDirectory)"
# displayName: "Restore Coverage Info"
# condition: always()

# Download previously generated coverage artifacts
- task: DownloadPipelineArtifact@2
inputs:
buildType: "current"
allowPartiallySucceededBuilds: true
downloadType: "Specific"
itemPattern: "**/.nyc_output/**"
downloadPath: "$(Build.SourcesDirectory)"
displayName: "Restore Coverage Info"
patterns: '**/.nyc_output/**'
displayName: "Download .nyc_output coverage artifacts"
condition: always()

- bash: |
pwd
cd $(Pipeline.Workspace)
pwd
ls -dlU .*/ */
ls
displayName: "See the content of Pipeline.Workspace dir"
condition: always()

# Now that we have downloaded artifacts from `coverage-output-`, copy them
# into the root directory (they'll go under `.nyc_output/...`)
# These are the coverage output files that can be merged and then we can generate a report from them.
Expand Down
2 changes: 1 addition & 1 deletion build/ci/templates/test_phases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -481,5 +481,5 @@ steps:
- task: PublishBuildArtifacts@1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: $(Agent.JobName)
artifactName: 'coverage-output-$(Agent.Os)'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is based on the assumption that we'll only have coverage reports from 3 jobs (one for each OS).
In the past we had coverage reports from others tests as well, may have changed since, please double check.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what we have at the moment, I took this from the PublishBuildArtifacts@1 step in build/ci/templates/test_phases.yml

condition: always()
121 changes: 60 additions & 61 deletions build/ci/vscode-python-pr-validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ variables:
- template: templates/globals.yml

stages:
- stage: Build
jobs:
- template: templates/jobs/build_compile.yml
# - stage: Build
# jobs:
# - template: templates/jobs/build_compile.yml

- stage: Tests
dependsOn: []
Expand All @@ -44,86 +44,85 @@ stages:
VMImageName: 'vs2017-win2016'
TestsToRun: 'testUnitTests, pythonUnitTests, pythonInternalTools'
NeedsPythonTestReqs: true
'Win-Py2.7 Unit':
PythonVersion: '2.7'
VMImageName: 'vs2017-win2016'
TestsToRun: 'pythonUnitTests'
NeedsPythonTestReqs: true
# 'Win-Py2.7 Unit':
# PythonVersion: '2.7'
# VMImageName: 'vs2017-win2016'
# TestsToRun: 'pythonUnitTests'
# NeedsPythonTestReqs: true
'Mac-Py3.7 Unit':
PythonVersion: '3.7'
VMImageName: 'macos-10.13'
TestsToRun: 'testUnitTests, pythonUnitTests, pythonInternalTools'
NeedsPythonTestReqs: true
'Mac-Py3.7 Single Workspace':
PythonVersion: '3.7'
VMImageName: 'macos-10.13'
TestsToRun: 'testSingleWorkspace'
NeedsPythonTestReqs: true
'Mac-Py2.7 Unit+Single':
PythonVersion: '2.7'
VMImageName: 'macos-10.13'
TestsToRun: 'pythonUnitTests, testSingleWorkspace'
NeedsPythonTestReqs: true
# 'Mac-Py3.7 Single Workspace':
# PythonVersion: '3.7'
# VMImageName: 'macos-10.13'
# TestsToRun: 'testSingleWorkspace'
# NeedsPythonTestReqs: true
# 'Mac-Py2.7 Unit+Single':
# PythonVersion: '2.7'
# VMImageName: 'macos-10.13'
# TestsToRun: 'pythonUnitTests, testSingleWorkspace'
# NeedsPythonTestReqs: true
'Linux-Py3.7 Unit':
PythonVersion: '3.7'
VMImageName: 'ubuntu-16.04'
TestsToRun: 'testUnitTests, pythonUnitTests, pythonInternalTools'
NeedsPythonTestReqs: true
'Linux-Py3.7 Single Workspace':
PythonVersion: '3.7'
VMImageName: 'ubuntu-16.04'
TestsToRun: 'testSingleWorkspace'
NeedsPythonTestReqs: true
'Linux-Py2.7 Unit+Single':
PythonVersion: '2.7'
VMImageName: 'ubuntu-16.04'
TestsToRun: 'pythonUnitTests, testSingleWorkspace'
NeedsPythonTestReqs: true
'Linux-Py3.7 Functional':
PythonVersion: '3.7'
VMImageName: 'ubuntu-16.04'
TestsToRun: 'testfunctional'
NeedsPythonTestReqs: true
NeedsPythonFunctionalReqs: true
# 'Linux-Py3.7 Single Workspace':
# PythonVersion: '3.7'
# VMImageName: 'ubuntu-16.04'
# TestsToRun: 'testSingleWorkspace'
# NeedsPythonTestReqs: true
# 'Linux-Py2.7 Unit+Single':
# PythonVersion: '2.7'
# VMImageName: 'ubuntu-16.04'
# TestsToRun: 'pythonUnitTests, testSingleWorkspace'
# NeedsPythonTestReqs: true
# 'Linux-Py3.7 Functional':
# PythonVersion: '3.7'
# VMImageName: 'ubuntu-16.04'
# TestsToRun: 'testfunctional'
# NeedsPythonTestReqs: true
# NeedsPythonFunctionalReqs: true

pool:
vmImage: $(VMImageName)

steps:
- template: templates/test_phases.yml

- stage: Smoke
dependsOn: []
jobs:
- job: 'Smoke'
dependsOn: []
strategy:
matrix:
'Mac-Py3.7':
PythonVersion: '3.7'
VMImageName: 'macos-10.13'
TestsToRun: 'testSmoke'
NeedsPythonTestReqs: true
'Linux-Py3.7':
PythonVersion: '3.7'
VMImageName: 'ubuntu-16.04'
TestsToRun: 'testSmoke'
NeedsPythonTestReqs: true
'Win-Py3.7':
PythonVersion: '3.7'
VMImageName: 'vs2017-win2016'
TestsToRun: 'testSmoke'
NeedsPythonTestReqs: true
# - stage: Smoke
# dependsOn: []
# jobs:
# - job: 'Smoke'
# dependsOn: []
# strategy:
# matrix:
# 'Mac-Py3.7':
# PythonVersion: '3.7'
# VMImageName: 'macos-10.13'
# TestsToRun: 'testSmoke'
# NeedsPythonTestReqs: true
# 'Linux-Py3.7':
# PythonVersion: '3.7'
# VMImageName: 'ubuntu-16.04'
# TestsToRun: 'testSmoke'
# NeedsPythonTestReqs: true
# 'Win-Py3.7':
# PythonVersion: '3.7'
# VMImageName: 'vs2017-win2016'
# TestsToRun: 'testSmoke'
# NeedsPythonTestReqs: true

pool:
vmImage: $(VMImageName)
# pool:
# vmImage: $(VMImageName)

steps:
- template: templates/test_phases.yml
# steps:
# - template: templates/test_phases.yml

- stage: Reports
dependsOn:
- Smoke
- Tests
condition: always()
jobs:
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2678,7 +2678,7 @@
"test:unittests:cover": "nyc --silent --no-clean --nycrc-path build/.nycrc mocha --opts ./build/.mocha.unittests.ts.opts",
"test:functional": "mocha --require source-map-support/register --opts ./build/.mocha.functional.opts",
"test:functional:cover": "npm run test:functional",
"test:cover:report": "nyc --nycrc-path build/.nycrc report --reporter=lcov --reporter=text --reporter=html --reporter=text-summary --reporter=cobertura",
"test:cover:report": "nyc --nycrc-path build/.nycrc report --reporter=text --reporter=html --reporter=text-summary --reporter=cobertura",
"testDebugger": "node ./out/test/testBootstrap.js ./out/test/debuggerTest.js",
"testSingleWorkspace": "node ./out/test/testBootstrap.js ./out/test/standardTest.js",
"testMultiWorkspace": "node ./out/test/testBootstrap.js ./out/test/multiRootTest.js",
Expand Down