SBOM is different on package.json with same command on different systems

[Serraniel]

Describe the bug

Hello,

we ran into an issue, where the output of the generated SBOM is different if executed in the Jenkins environment compared to the results if I execute the command locally.
We use the following command to generate SBOM
npx --yes -d @cyclonedx/cyclonedx-npm@^1 --output-file "..\..\..\bom\components\portal_frontend.json" --omit dev

The major difference is that in the Jenkins environment an additional hashes property is generated for each entry. That itself isn´t an issue for us at all in general but there is one exception:
We also have included the fontawesome-pro package, which doesn´t come from npmjs but the FA npm registry. In this case the hashes property is not generated but the PURL is extended.
If I execute the mentioned command locally the tool generates
"purl": "pkg:npm/%40fortawesome/fontawesome-pro@6.2.1?download_url=https://npm.fontawesome.com/@fortawesome/fontawesome-pro/-/6.2.1/fontawesome-pro-6.2.1.tgz"

In the Jenkins environment it instead generates
"purl": "pkg:npm/%40fortawesome/fontawesome-pro@6.2.1?checksum=sha-512:74793b8a209fe4c0a6a14be6ad8cdf37f23781e6e9829035a2a94e7df80e4e19ec680478929690e58313764746f7d39fd619cdd01a24e91f87136f335dbdd23a&download_url=https://npm.fontawesome.com/@fortawesome/fontawesome-pro/-/6.2.1/fontawesome-pro-6.2.1.tgz"

This leads to the issue, that our Dependency Track rejects the SBOM file cause that specific purl entry exceeds the max length

I don´t know if that limitation comes from the specification or is an internal limitation of Dependency Track.
I was able to use the short purl options as a first workaround, however my main questions are:
Why is the output different from when I execute the command locally and is there a reason, that for the external registry, the hash is written into the PURL which isn´t the case for packages coming from npmjs?

Expected behavior

Identical output SBOM in local and jenkins environment

Screenshots or output-paste

If applicable, add screenshots or past the output to help explain your problem.

Environment

Local

• @cyclonedx/cyclonedx-npm version: ^1
• NPM version: 8.19.2
• Node version: 18.12.1
• OS: Windows 10

Jenkins

• @cyclonedx/cyclonedx-npm version: ^1
• NPM version: 8.8.0
• Node version: 18.1.0
• OS: Windows

Additional context

Some screenshot if I diff the local (left) and jenkins (right) output:

[jkowalleck]

regarding the result differences:

I understand that the result differs, if you run it in different systems, which each differ in installed npm packages/versions.

You expect, that the "same command" results in the same output.
Did you check that the version of @cyclonedx/cyclonedx-npm and its dependencies are the same on all systems?
I doubt that they are all the same, because you described a behavior that was added as a feature in one version, but did not exist in an older one.

---

regarding that problem parsing the PURL in DependencyTrack(DT): it is a known issue of DT which allows PURLS of a certain length at max. They know it is an issue and might have fixed it in the past. And in addition, it appeared to be a problem with some OSS indexes, which allowed PURLs with a certain max length in their APIs. All that together caused unexpected issues across multiple foreign systems, and it might be fixed in the future.

See also: #225

Please upgrade your DependencyTrack.
If you still have the issue, upgrade to @cyclonedx/cyclonedx-npm@^1.1 and utilize the --short-PURLs that was introduced via v1.1.0.

[jkowalleck]

@Serraniel if the above answered your question, then please mark it as "answer" on GitHub. If it did not work, let me known, so we can find a solution.

[Serraniel]

Sorry for my late reply I just saw your comment after the I got the mails about the latest discussion. Latest DTrack version is still from 3 months ago, which we are running, but the short purls option has fixed the issue for us, indeed.

[igord]

Different npm versions can produce different results. Maybe it will be good to have option to pass path to npm binary or have the latest npm as dependency and use it for consistency, or both?

[jkowalleck]

Different npm versions can produce different results.

True. @cyclonedx/cyclonedx-npm collects evidences as produced by npm.
if you run a non-equal versions of npm on a non-equal systems in a non-equal environments, then you might end up with a non-equal sets of evidence, and therefore the SBOM might be different.

There are already mechanisms built in to streamline results a bit, but only to a degree where no evidence is altered or ignored.
You might not have noticed them or "used it wrong", so I need to ask:
how do you call @cyclonedx/cyclonedx-npm in the first place? like npx ...? like npm exec ...? do you have it installed globally? do you ave it as a npm-script for npm run ...?

[jkowalleck]

[...] have the latest npm as dependency and use it for consistency

Not an option. It is to use the npm that is used on the host system. Exactly the one npm that was used to manage the project that is to be analyzed.
Using a different version of npm might cause wrong results or modify the project while it is analyzed, because that is how npm works.

[jkowalleck]

reply here: #502 (comment)

[igord]

Thanks for your quick response!

how do you call @cyclonedx/cyclonedx-npm in the first place?

I am not using it yet, just evaluating. What is the preferred way to call it?

Exactly the one npm that was used to manage the project that is to be analyzed.

Here is how I would like to use it:
Application is being developed by multiple teams. Prior to release, SBOM is being generated by this library, as part of automated release workflow, like Github Actions.
How can we guarantee that npm version used by automated release is the same one used by development teams?

[jkowalleck]

What is the preferred way to call it?

This depends on your setup. Therefore, as the Readme states, there are multiple supported ways.
Ask your developers or principal engineer.

How can we guarantee that npm version used by automated release is the same one used by development teams?

This is not required at all!
why would you care for an SBOM of a development environment? half of the things you have there are not part of your software, and the setup is twisted to the personal needs of your developer. This does not reflect a final product or a productive runtime environment.
And if you'd care for an dev-SBOM, then it would not matter if it was the same as on production – because the purpose of this very discussion is "differences/equality of SBOM results" -- not general SBOM education. ;-)

[igord]

why would you care for an SBOM of a development environment?

Because endusers will get the sourcecode so you want to make sure that the dependencies they get are the same as dependencies in SBOM. By using engines field in package.json and package-lock.json

[jkowalleck]

why would you care for an SBOM of a development environment?

Because endusers will get the sourcecode so you want to make sure that the dependencies they get are the same as dependencies in SBOM. By using engines field in package.json and package-lock.json

You you were talking about development environments at first, but are changing the subject to end users? Well, I assume dev environments are out of scope, just in the matter of @cyclonedx/cyclonedx-npm.

If you ship the lock file, then npn takes care of setups and equality. You should be fine.
@cyclonedx/cyclonedx-npm will not artificially alter any facts, but it collects and reports. A lock-file is a state of a dependency tree and an idea of assembly. @cyclonedx/cyclonedx-npm utilizes npm for evidence collection – so if anything is "different", then you need to blame the setup.
Just try it and see the qualitative differences (if any) yourself.

Because endusers will get the sourcecode so you want to make sure that the dependencies they get are the same as dependencies in SBOM.

Then you might need to take care of it. ;-) A lockfile reflects a "planned-SBOM", not an actual setup-based SBOM.
I will not give free node workshops here, but you might want to look up what bundledDependencies are.

[igord]

You you were talking about development environments at first, but are changing the subject to end users?

From my point of view it is still the same subject as endusers are developers...

If you ship the lock file, then npn takes care of setups and equality. You should be fine.

If peerDependencies are used (which is quite common) version of npm that is used can make a difference. For example, up to v7 of npm, install (or ls) will produce warnings if there is conflict in peerDependencies but it will finish, but from v7 it will fail.
So version of npm does matter, thus my suggestion for a feature.
To make it worst, npm announced more breaking changes soon, see https://docs.npmjs.com/cli/v9/commands/npm-ls#note-design-changes-pending

Then you might need to take care of it

Would you accept PR with feature to optionally use specific npm binary?

I will not give free node workshops here

There ain't no such thing as a free lunch : )

you might want to look up what bundledDependencies are

They are useful in some cases but this is not one of them.

[jkowalleck]

Would you accept PR with feature to optionally use specific npm binary?

Definitely.

PS: result -> #647