Generate SBOM during build/release process #3805
Conversation
|
I assume you are going to add some GitHub workflow for this? |
Right, added it. I tested it, here is the run: Here is the resulting SBOM file: |
Sure, I first tried Syft but I had problems with it detecting |
| "https://api.github.com/repos/open-telemetry/" | ||
| "opentelemetry-python/dependency-graph/sbom" |
There was a problem hiding this comment.
This API seems pretty barebones, I'm guessing this just generates an SBOM against the main branch and maybe cached at some random commit?
What if you are making a patch release, would the dep graph be correct?
If it is looking at main branch, I think our release process would show the .dev0 dependencies instead of actual released versions
There was a problem hiding this comment.
This API seems pretty barebones, I'm guessing this just generates an SBOM against the main branch and maybe cached at some random commit?
No, apparently this functionality creates an SBOM file using the latest commit in main. There is a way to generate this file using the web interface (https://github.com/open-telemetry/opentelemetry-python/three dots button in the upper right corner/Insights/Dependency graph/Export SBOM). This creates a file that includes the latest commit from main in its name. I assume the same file gets generated when the REST API is being used.
There was a problem hiding this comment.
What if you are making a patch release, would the dep graph be correct? If it is looking at main branch, I think our release process would show the .dev0 dependencies instead of actual released versions
The reason why the SBOM file includes released versions is because we have requirements.txt-like files that include released versions. For example, I made these changes, and pushed them to main:
diff --git a/docs/examples/metrics/reader/requirements.txt b/docs/examples/metrics/reader/requirements.txt
index 2ccffaf3..23ff5bf6 100644
--- a/docs/examples/metrics/reader/requirements.txt
+++ b/docs/examples/metrics/reader/requirements.txt
@@ -1,5 +1,5 @@
Deprecated==1.2.13
-opentelemetry-api==1.15.0
+opentelemetry-api==1.20.0
opentelemetry-sdk==1.15.0
opentelemetry-semantic-conventions==0.36b0
typing_extensions==4.3.0
diff --git a/docs/examples/metrics/views/requirements.txt b/docs/examples/metrics/views/requirements.txt
index be612711..d9590b3f 100644
--- a/docs/examples/metrics/views/requirements.txt
+++ b/docs/examples/metrics/views/requirements.txt
@@ -1,5 +1,5 @@
Deprecated==1.2.13
-opentelemetry-api==1.12.0
+opentelemetry-api==1.20.0
opentelemetry-sdk==1.12.0
opentelemetry-semantic-conventions==0.33b0
typing_extensions==4.3.0Those changes produced the following changes in the SBOM file:
5c5
< "created": "2024-04-03T19:17:07Z",
---
> "created": "2024-04-03T17:15:55Z",
16c16
< "documentNamespace": "https://github.com/SecuringCarter/opentelemetry-python/dependency_graph/sbom-7c05aa1b812baa89",
---
> "documentNamespace": "https://github.com/SecuringCarter/opentelemetry-python/dependency_graph/sbom-75a76526c2dd97b4",
814a815,830
> "SPDXID": "SPDXRef-pip-opentelemetry-api-1.15.0",
> "name": "pip:opentelemetry-api",
> "versionInfo": "1.15.0",
> "downloadLocation": "NOASSERTION",
> "filesAnalyzed": false,
> "licenseConcluded": "Apache-2.0",
> "supplier": "NOASSERTION",
> "externalRefs": [
> {
> "referenceCategory": "PACKAGE-MANAGER",
> "referenceLocator": "pkg:pypi/opentelemetry-api@1.15.0",
> "referenceType": "purl"
> }
> ]
> },
> {
878a895,910
> "SPDXID": "SPDXRef-pip-opentelemetry-api-1.12.0",
> "name": "pip:opentelemetry-api",
> "versionInfo": "1.12.0",
> "downloadLocation": "NOASSERTION",
> "filesAnalyzed": false,
> "licenseConcluded": "Apache-2.0",
> "supplier": "NOASSERTION",
> "externalRefs": [
> {
> "referenceCategory": "PACKAGE-MANAGER",
> "referenceLocator": "pkg:pypi/opentelemetry-api@1.12.0",
> "referenceType": "purl"
> }
> ]
> },
> {
2317a2350,2354
> "relatedSpdxElement": "SPDXRef-pip-opentelemetry-api-1.15.0"
> },
> {
> "relationshipType": "DEPENDS_ON",
> "spdxElementId": "SPDXRef-com.github.SecuringCarter-opentelemetry-python",
2333a2371,2375
> },
> {
> "relationshipType": "DEPENDS_ON",
> "spdxElementId": "SPDXRef-com.github.SecuringCarter-opentelemetry-python",
> "relatedSpdxElement": "SPDXRef-pip-opentelemetry-api-1.12.0"So, I now think that we probably should do something about these requirements-txt files that do not actually represent project dependencies.
There was a problem hiding this comment.
After many more changes I managed to reduce the amount of packages in the SBOM file with an empty value of versionInfo to only one:
{
"SPDXID": "SPDXRef-com.github.SecuringCarter-opentelemetry-python",
"name": "com.github.SecuringCarter/opentelemetry-python",
"versionInfo": "",
"downloadLocation": "git+https://github.com/SecuringCarter/opentelemetry-python",
"licenseDeclared": "Apache-2.0",
"filesAnalyzed": false,
"supplier": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:github/SecuringCarter/opentelemetry-python"
}
]
}
I think the mechanism to generate an SBOM file that this PR introduces is valid. We probably need to change our repo in order to produce a "better" or more useful SBOM file.
Can we merge this and work on making the SBOM file better in subsequent PRs?
|
|
||
| from requests import get | ||
|
|
||
| with open("opentelemetry-python.spdx.json", "w") as sbom_file: |
There was a problem hiding this comment.
I took a look at the actual API response
- It says
Exact versions could not be resolved for some packages. For more information: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-the-dependency-graph#dependencies-included.
- The relationships in the file look like
My reading of this is "the opetnelemetry-python repo depends on X" rather than linking individual artifacts (e.g. opentelemetry-api) to their dependencies. I'm not sure how helpful that is for consumers of the SBOM.
{ "relationshipType": "DEPENDS_ON", "spdxElementId": "SPDXRef-com.github.open-telemetry-opentelemetry-python", "relatedSpdxElement": "SPDXRef-pip-asgiref-3.7.2" }
There was a problem hiding this comment.
I took a look at the actual API response
- It says
Exact versions could not be resolved for some packages. For more information: docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-the-dependency-graph#dependencies-included.
Right, I looked into the SBOM file, found several packages with an empty value for versionInfo, here they are:
{
"SPDXID": "SPDXRef-com.github.open-telemetry-opentelemetry-python",
"name": "com.github.open-telemetry/opentelemetry-python",
"versionInfo": "",
"downloadLocation": "git+https://github.com/open-telemetry/opentelemetry-python",
"licenseDeclared": "Apache-2.0",
"filesAnalyzed": false,
"supplier": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": "pkg:github/open-telemetry/opentelemetry-python"
}
]
}
{
"SPDXID": "SPDXRef-pip-opencensus",
"name": "pip:opencensus",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opencensus-ext-flask",
"name": "pip:opencensus-ext-flask",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-api",
"name": "pip:opentelemetry-api",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-exporter-jaeger",
"name": "pip:opentelemetry-exporter-jaeger",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-instrumentation-sqlite3",
"name": "pip:opentelemetry-instrumentation-sqlite3",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-opencensus-shim",
"name": "pip:opentelemetry-opencensus-shim",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-sdk",
"name": "pip:opentelemetry-sdk",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-opentelemetry-opentracing-shim",
"name": "pip:opentelemetry-opentracing-shim",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-redis",
"name": "pip:redis",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-redis-opentracing",
"name": "pip:redis-opentracing",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-googleapis-common-protos",
"name": "pip:googleapis-common-protos",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-requests",
"name": "pip:requests",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
{
"SPDXID": "SPDXRef-pip-asgiref",
"name": "pip:asgiref",
"versionInfo": "",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"supplier": "NOASSERTION"
}
Fixes #3540