Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: generate SBOM from "global" scope #471

Open
schlenk opened this issue Jan 31, 2023 · 8 comments · May be fixed by #503
Open

feat: generate SBOM from "global" scope #471

schlenk opened this issue Jan 31, 2023 · 8 comments · May be fixed by #503
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@schlenk
Copy link

schlenk commented Jan 31, 2023

Is your feature request related to a problem? Please describe.

I try to package a nodejs + NPM installation that ships a few global tools like yarn/dart-saas as part of a larger software installer.

So basically i do the following (with some in-house BSD/Mac ports style system):

  1. Unzip a nodejs distro, e.g. node-v18.12.1-win-x64.zip and rename the folder to 'img'

  2. Provide some .tar.gz of sass & yarn, e.g. sass-1.57.0.tar.gz and yarn-v1.22.10.tar.gz in the folder 'base'

  3. Run a global npm install for the packages, resolving dependencies from the npm registry

     img\npm install --cache base/ --global saas@1.57.0
     img\npm install --cache base/ --global yarn@1.22.10
    
  4. Try to get an SBOM for the installed global packages, either as a SBOM with multiple application components included or one SBOM per application.

Describe the solution you'd like

I would like to have an option to use this package to get SBOMs for the globally installed packages in my node_modules folder.

Describe alternatives you've considered

I used the cyclonedx-bom package before, just pointing it at the created img/node_modules to collect the actually installed global tools with dependencies. That worked mostly fine and created a useable SBOM.

With this package this does not work, as the code complains about a missing package-lock.json when i run it for the individual apps and has no option at all to just consume the img/node_modules folder.

C:\code\repro\img>.\cyclonedx-npm --output-file bom.json node_modules\yarn\package.json
DEBUG | options: {"ignoreNpmErrors":false,"packageLockOnly":false,"omit":[],"flattenComponents":false,"shortPURLs":false,"specVersion":"1.4","outputFormat":"JSON","outputFile":"bom.json","mcType":"application"}
DEBUG | packageFile: C:\code\repro\img\node_modules\yarn\package.json
INFO  | projectDir: C:\code\repro\img\node_modules\yarn
LOG   | No evidence: no package lock file nor npm shrinkwrap file
LOG   | No evidence: no node_modules dir
INFO  | ? Did you forget to run `npm install` on your project accordingly ?

Error: missing evidence

There seems to be no way to tell npm install --global to create any form of lock file. Or i could not find it.

@schlenk schlenk added the enhancement New feature or request label Jan 31, 2023
@schlenk
Copy link
Author

schlenk commented Feb 1, 2023

Ok, i got it to work (somewhat) by simply creating a mostly empty package.json in the toplevel directory. The output isn't perfect but with some post processing it should be useable.

C:\code\repro\img>cat package.json
{}

C:\code\repro\img>.\cyclonedx-npm --output-file bom.json .\package.json
DEBUG | options: {"ignoreNpmErrors":false,"packageLockOnly":false,"omit":[],"flattenComponents":false,"shortPURLs":false,"specVersion":"1.4","outputFormat":"JSON","outputFile":"bom.json","mcType":"application"}
DEBUG | packageFile: C:\code\repro\img\package.json
INFO  | projectDir: C:\code\repro\img
DEBUG | detected a node_modules dir
DEBUG | makeNpmRunner caused execSync "npm"
INFO  | detect NPM version ...
DEBUG | detected NPM version [8,19,2]
INFO  | gather dependency tree ...
DEBUG | npm-ls: run npm with ["ls","--json","--long","--all"] in "C:\\code\\repro\\img"
INFO  | build BOM ...
LOG   | writing BOM to bom.json

@jkowalleck
Copy link
Member

jkowalleck commented Feb 13, 2023

This tool does not actually read the package.json or lock-file itself, nor does it search node_modules folders.
It utilized npm to gather all information. see https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/docs/how.md

Therefore, I sense a feature request to building an SBOM on the globally installed npm packages.

@schlenk , could you do me a favor and tell:
if you run the following on your image, does it contain everything you expected?

npm  ls  --global

@schlenk
Copy link
Author

schlenk commented Feb 13, 2023

Yes, the npm ls lists at least lists the toplevel packages, adding appropriate --depth gets the dependencies as well.

img\npm ls --global

+-- @cyclonedx/cyclonedx-npm@1.7.2
+-- corepack@0.14.2
+-- npm@8.19.2
+-- sass@1.57.0
`-- yarn@1.22.10

@jkowalleck
Copy link
Member

re: #471 (comment)

if it lists the top levels, we are good to continue, thanks.
I will push you a preview version of this very tool, soon, which will be able to create an SBOM of your global env.

@jkowalleck jkowalleck self-assigned this Feb 13, 2023
@jkowalleck jkowalleck linked a pull request Feb 13, 2023 that will close this issue
7 tasks
@jkowalleck jkowalleck changed the title Documenting installed global npm packages? feat: generate SBOM from "global" scope Feb 14, 2023
@jkowalleck
Copy link
Member

got feedback to a POC/preview implementation: #503 (comment)

@jkowalleck Thanks.

I gave it a try and it basically works with the --global switch.

There are two small feature requests still lurking in there, to make it more useful.

  1. Optionally exclude the cyclonedx-npm global install from the SBOM, as thats basically only installed to create the SBOM and mentioned in the tools metadata anyway, but not really of interest for the SBOM. (or if you want to turn it around, allow a cyclonedx-npm --global to specify the npm installation to actually use, inspecting the global SBOM of a different NPM install instead of the one it is installed into.)

  2. It would be nice to specify toplevel component name/version/purl, as the component that ends up in the metadata is a bit lacking, and always need post processing. This is usually provided by the package.json file, so it is missing for the global case e.g.

    "component": {
      "type": "application",
      "name": "img",
      "bom-ref": "-/img@-",
      "purl": "pkg:npm/img",
      "properties": [
        {
          "name": "cdx:npm:package:path",
          "value": ""
        }
      ]
    }

I do have a question, @schlenk

[...] inspecting the global SBOM of a different NPM install instead of the one it is installed into.

What is the story behind this?
After the expectations were made clear, we might look for a technical solution.

@jkowalleck jkowalleck removed their assignment Feb 14, 2023
@schlenk
Copy link
Author

schlenk commented Feb 14, 2023

@jkowalleck: The story about the external SBOM of a global install is actually fairly easy, even if unusual.

I build some kind of SDK for developing specific enterprisey web applications based on React. So i want to get an SBOM for the components in that SDK, which is a wide variety of stuff, from a customized Apache httpd server to complete NodeJS install and some globally installed tools for that installation, merged into a big SBOM for all the parts.

The NodeJS part ships with a few global installs, but cyclonedx-npm is not one of the tools that get bundled.
So if I install cyclonedx-npm into the SDK image to compute the SBOM, i basically pollute my setup. The tool that packages the parts of the SDK would filter out the cyclonedx-npm files/directories for bundling, but does not manipulate the SBOM.

So i have three options to choose from:

  • Include cyclonedx-npm in the global installs of the SDK and get approval for that
  • Post process the SBOM in the packaging tool, so it removes the components it does not bundle
  • Have a way to inspect some global installation without polluting it

The third option is not much different to the usual case of using cyclonedx-npm on a package. If you use a global install of cyclonedx-npm the package does not get polluted with a new cyclonedx-npm dependency. So it basically asks: I want to treat a node_modules directory of a npm global install like i would treat it for package with a package.json that listed all the globally installed tools as dependencies.

@jkowalleck
Copy link
Member

No need to install cyclonedx-npm globally, or in the env you are analyzing.
Was this unclear from the documentation and help page?

You could install it in any node-env (like global encapsulated npx). You could install it in a temporary location, then run it, and remove it afterwards.
example bash script

TD=$(mktemp -d)
npm --prefix "$TD" install --no-save @cyclonedx/cyclonedx-npm@^1.7
npm --prefix "$TD" exec cyclonedx-npm -- <options> <path-to-project>
rm -rf "$TD"

@schlenk
Copy link
Author

schlenk commented Feb 15, 2023

Ok, thank you. Yes, i did not understand it from the documentation. But in hindsight, its documented there.

So that part is unnecessary, but the --global is still useful.

@jkowalleck jkowalleck added the help wanted Extra attention is needed label Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants