[YARN] use only top-level .lock file to avoid inflating BOM with uninstalled package versions

[thediveo]

When I run cyclonedx-bom on a yarn-managed project with a yarn.lock file, the BOM contains lots of package versions that aren't installed, as yarn resolves into a single version. Based on a seemingly similar issue that turned up in the context of the syft BOM scanner, anchore/syft#431, this might be due to not ignoring nested .lock files.

[jkowalleck]

THanks for the report, @thediveo.
What do you mean by "he BOM contains lots of package versions that aren't installed"?
Could you prepare an example setup as a gthub-repo that we can use to reproduce the issue?

[jkowalleck]

this issue might be related to read-installed.
so #215 is blocking it.

[jkowalleck]

this issue is related to #86

[thediveo]

Please use @thediveo/lxkns as an example. Change into web/lxkns and run yarn install there, this will resolve and install the npm packages. Now, please run cyclonedx-bom and then check the resulting bom: it will not only list the resolved, final npm package versions, but also older and unselected versions of the same packages. This causes lots of false positives on CVE scanning, but also in other BOM use case scenarios.

[jkowalleck]

Thanks @thediveo

could you share what you see VS what you expect? Which package did you find in the bom, but expect it not to be in there?
what does your SBOM file look like (upload result it here or in a pastebin)?
what does yarn list print (upload output it here or in a pastebin) ?

[jkowalleck]

YARN support was added via #282

this should close the issue, then.

[thediveo]

will give it a try, thank you very much!

[thediveo]

hm, this still looks very very strange. In the above mentioned open source project I run yarn install inside web/lxkns. Then I created a bom cyclonedx-node -o bom.json using the latest version installed from npm, it reports 3.7.0. Then I grepped the BOM for some arbitrary package using grep bom-ref /tmp/bom.json | sort | grep yargs-parser:

"bom-ref": "pkg:npm/%40types/yargs-parser@21.0.0",                                                                "bom-ref": "pkg:npm/yargs-parser@13.1.2",
"bom-ref": "pkg:npm/yargs-parser@18.1.3",

eh? But let's ask yarn: yarn why yargs-parser ... which gives

=> Found "yargs-parser@20.2.9"

Hmm, what am I doing wrong here or assuming that isn't correct? This BOM doesn't reflect what yarn tells me.

[jkowalleck]

reproduced the report

---

ran

yarn install 
yarn install v1.22.18
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > @babel/plugin-proposal-private-property-in-object@7.16.7" has unmet peer dependency "@babel/core@^7.0.0-0".
warning "@babel/plugin-proposal-private-property-in-object > @babel/helper-create-class-features-plugin@7.17.6" has unmet peer dependency "@babel/core@^7.0.0".
warning "@babel/plugin-proposal-private-property-in-object > @babel/plugin-syntax-private-property-in-object@7.14.5" has unmet peer dependency "@babel/core@^7.0.0-0".
warning "@emotion/react > @emotion/babel-plugin@11.7.2" has unmet peer dependency "@babel/core@^7.0.0".
warning "@emotion/react > @emotion/babel-plugin > @babel/plugin-syntax-jsx@7.16.7" has unmet peer dependency "@babel/core@^7.0.0-0".
warning " > @testing-library/user-event@13.5.0" has unmet peer dependency "@testing-library/dom@>=7.21.4".
warning " > babel-loader@8.1.0" has unmet peer dependency "@babel/core@^7.0.0".
warning " > babel-loader@8.1.0" has unmet peer dependency "webpack@>=2".
warning " > material-ui-confirm@2.1.4" has unmet peer dependency "@material-ui/core@>= 3.0.0".
warning " > url-loader@4.1.1" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning " > ts-loader@8.3.0" has unmet peer dependency "webpack@*".
warning " > @cypress/webpack-dev-server@1.8.2" has unmet peer dependency "html-webpack-plugin@>=4".
warning " > @cypress/webpack-dev-server@1.8.2" has unmet peer dependency "webpack@>=4".
warning " > @cypress/webpack-dev-server@1.8.2" has unmet peer dependency "webpack-dev-server@>=3.0.0".
warning "react-styleguidist > clean-webpack-plugin@3.0.0" has unmet peer dependency "webpack@*".
warning "react-styleguidist > copy-webpack-plugin@6.4.1" has unmet peer dependency "webpack@^4.37.0 || ^5.0.0".
warning "react-styleguidist > mini-html-webpack-plugin@3.1.3" has unmet peer dependency "webpack@>=4".
warning "react-styleguidist > react-simple-code-editor@0.10.0" has incorrect peer dependency "react@^16.0.0".
warning "react-styleguidist > react-simple-code-editor@0.10.0" has incorrect peer dependency "react-dom@^16.0.0".
warning "react-styleguidist > webpack-dev-server@3.11.3" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "react-styleguidist > react-dev-utils > fork-ts-checker-webpack-plugin@6.5.0" has unmet peer dependency "webpack@>= 4".
[4/4] Building fresh packages...
Done in 98.56s.

then cyclonedx-node -o bom.json - resulted in (had to rename the file for uploading here)
bom.json.txt
finding multiple yargs-parser in the resulting BOM file, ... and multiples in the lock file as well.

i am finding one in node_modules/ folder - version 20.2.9

searching for the yargs-parser ...

yarn why yargs-parser
yarn why v1.22.18
[1/4] Why do we have the module "yargs-parser"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "yargs-parser@20.2.9"
info Has been hoisted to "yargs-parser"
info Reasons this module exists
   - Hoisted from "react-styleguidist#react-docgen#estree-to-babel#c8#yargs-parser"
   - Hoisted from "react-styleguidist#react-docgen#estree-to-babel#c8#yargs#yargs-parser"
info Disk size without dependencies: "156KB"
info Disk size with unique dependencies: "156KB"
info Disk size with transitive dependencies: "156KB"
info Number of shared dependencies: 0
=> Found "yargs#yargs-parser@13.1.2"
info This module exists because "yargs" depends on it.
info Disk size without dependencies: "76KB"
info Disk size with unique dependencies: "112KB"
info Disk size with transitive dependencies: "112KB"
info Number of shared dependencies: 2
=> Found "jest-cli#yargs-parser@18.1.3"
info Reasons this module exists
   - "react-scripts#jest#jest-cli#yargs" depends on it
   - Hoisted from "react-scripts#jest#jest-cli#yargs#yargs-parser"
info Disk size without dependencies: "92KB"
info Disk size with unique dependencies: "128KB"
info Disk size with transitive dependencies: "128KB"
info Number of shared dependencies: 2
=> Found "jest-runtime#yargs-parser@18.1.3"
info Reasons this module exists
   - "react-scripts#jest-circus#jest-runtime#yargs" depends on it
   - Hoisted from "react-scripts#jest-circus#jest-runtime#yargs#yargs-parser"
info Disk size without dependencies: "92KB"
info Disk size with unique dependencies: "128KB"
info Disk size with transitive dependencies: "128KB"
info Number of shared dependencies: 2
Done in 2.97s.

[jkowalleck]

looking at lock files is still dump.
need to have the actual env as evidence. so need to check the data in the node_modules dir of the project and globally.

could be fixed via #215

[thediveo]

there goes my simple belief in lock files 😆 ... many thanks for locking, erm, looking into this convoluted topic further!

[jkowalleck]

❗ the yarn support got a dedicated project in the making: https://github.com/CycloneDX/cyclonedx-node-yarn