FEAT: Option to add license text to SBOM result

[rkg-mm]

Is your feature request related to a problem? Please describe.

For legal documentation, we need the original text of the licenses of components.

Describe the solution you'd like

An option to enable integration of the license-text in the BOM file, like the old @cyclonedx/bom package had, would be great to have again here.

---

read https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence

Acceptance criteria

• the feature to add license texts should be enabled by a CLI switch called --gather-license-evidence (name to be discussed)
• the feature is disabled per default
• only if the feature is enabled:
  • for all components, meta-components, root-components and nested components:
    regardless of SPDX license ID, SPDX license expression or named license, the deteced license texts should be added, each as an evidence
    Examples:

    {
      //...
      "evidence": { 
        "licenses": [
          {"name":"file: LICENSE", "text": {
            "contentType": "text/plain",
            "encoding": "base64",
            // base64 of content of file `LICENSE`
            "content": "bG9yZW0gaXBzdW0="
          }}
          {"name":"file: NOTICE", "text": {
            "contentType": "text/plain",
            "encoding": "base64",
            // base46 of content of file `NOTICE`
            "content": "bG9yZW0gaXBzdW0="
          }}
        ]
      },
      // ...
    }

  • if a license text is detected with the package, it would be added to Component's @.evicence.licenses
    • @.name would be 'License of : '
    • @.text would hold the test
      • the content type is to be derived from file extension
      • the content SHOULD be base64 encoded
  • license files patterns are:
    • LICEN[CS]E*
    • NOTICE* -- addendum for Apache-2.0 and others
  • if no license text is shipped with a package, no license test is added as a evidence.
    Nope, no license template is derived from package's declared SPDX license id.
    Reason: license templates (like BSD clause 3) are designed to be modified (unlike others, like Apache2, which is not a template but a complete text)

[jkowalleck]

this very feature request is similar to CycloneDX/cyclonedx-node-module#343 (comment)

[jkowalleck]

previous license text gathering implementation: https://github.com/CycloneDX/cyclonedx-node-module/blob/3.x/model/LicenseChoice.js#L72

[jkowalleck]

reminder: npm6 had the readme as part of the result of npm ls
this was dropped on npm7 ...

unfortunately there is no such feature for the license. it needs to be gathered manually.

[jkowalleck]

feel free to open pull-requests to bring this feature to the tool.

[jkowalleck]

this very feature request is similar to CycloneDX/cyclonedx-javascript-library#404

[jkowalleck]

I am investigating which library could be used for license text extraction.
Do not want to implement nor maintain a solution, if there is one already.

🔍
result of my research:

• we could utilize license-scanner after Programmable interface - machine-readable output license-scanner#9 was done.

PS: if no lib nor scanner can be found, lets simply pull the license texts from files that seam appropriate.

[jkowalleck]

Similar to CycloneDX/cyclonedx-webpack-plugin#676

[jkowalleck]

@stevespringett mentioned:

Keep in mind that compositions are the only way of saying that something is complete. Our build implementations do not produce compositions nor should they. Its really up to the end user org to attest if the BOM is complete or not and add the corresponding composition if it is.
Also note, that licenses go in two places. 1) the declared license for the component goes into component/licenses and all the evidence of copyrights and licenses for the entirety of the component goes into component/evidence

read: https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence

[jkowalleck]

after #726 is done,
the underlying data types allow the use of any stringable, so Buffers will be possible to use.

PS: or helper classes that return the proper base64-sting when toString() is called on them.

[ghost]

I would love to help implement the feature, but I am not sure where this code would reside. For SPDX identifiers, we could parse the package.json of the dependencies or rather execute npm for this, right? For specific license text we could use the old logic to identify custom text, if the SPDX identifier is not present.

Could you give me some hints on where to start and what you expect as a maintainer?

[hacst]

Is the spdx id based license template text really the expected thing to add as license text in the sbom? The actual project license contains things like copyright holders, date ranges etc that will not be part of the license template but are quite critical. E.g. one use case that comes to mind is reproducing all OSS licenses of dependencies that require you to do so for the user.

[jkowalleck]

re #256 (comment)

I would love to help implement the feature, but I am not sure where this code would reside.
when the code exists, we will find a place. it might start here: src/licenseText.ts

For SPDX identifiers, we could parse the package.json of the dependencies or rather execute npm for this, right?

SPDX licenses ID are already detected. a subroutine called by BomBuilder.makeComponent() handles it properly. see the internals of

cyclonedx-node-npm/src/builders.ts

Line 414 in 42208c6

const component = this.componentBuilder.makeComponent(_dataC, type)

here https://github.com/CycloneDX/cyclonedx-javascript-library/blob/main/src/builders/fromNodePackageJson.node.ts

For specific license text we could use the old logic to identify custom text, if the SPDX identifier is not present.

yes, basically we could use it to detect all kinds of license files and addendum and notice and others.

when it comes to the question "where do i put the detected license texts in the BOM", please read #256 (comment)
License texts and such are evidence and should be placed in the respective section of the BOM.

I updated the acceptance criteria on the issue, so it is much clearer what to expect.
Did this help?

---

implementation-wise an attachment can be anything that has a toString() method that returns base64. see https://github.com/CycloneDX/cyclonedx-javascript-library/blob/main/src/models/attachment.ts
So instead of pulling all the files' contents from disc and putting them into the RAM at the same time, it would be great to use a wrapper class, that stores the file name, and when it comes to calling the toString() method of an instance, it returns the base64 of the file content.
Let me know if you have trouble creating such a Class.

[jkowalleck]

re #256 (comment)

Is the spdx id based license template text really the expected thing to add as license text in the sbom?

nope. goal is to collect evidence, not the template texts. see https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence
The original request for this issue came from a law-perspective.
I know layers love texts, but they are not stupid, they can gather the template texts themselves.

I updated the acceptance criteria on the issue, so it is much clearer what to expect.
Did this help?

[jkowalleck]

A similar feature was added to the webpack plugin
see CycloneDX/cyclonedx-webpack-plugin#1309
see CycloneDX/cyclonedx-webpack-plugin#1312

[jkowalleck]

Since this feature is nothing new, there is no need to reinvent the wheel. It is expected to simply copy existing art.
Please be aware of existing art:

• Implementation for webpack:
  • ([BUG] not detecting license files LICENSE.BSD cyclonedx-webpack-plugin#1337 via fix: Properly detect license evidences like LICEN[CS]E.{Apache,BSD,GPL,MIT} cyclonedx-webpack-plugin#1339)
  • (also find *.license files as evidence cyclonedx-webpack-plugin#1321 via feat: license evidence *.licen[cs]e cyclonedx-webpack-plugin#1322)
  • (FEAT: Option to add license text to BOM output cyclonedx-webpack-plugin#676 via feat: collect evidences for license cyclonedx-webpack-plugin#1309, fix: pollish evidence gathering cyclonedx-webpack-plugin#1312)
• Implementation for yarn:
  • Extraction of license text from files. cyclonedx-node-yarn#193

just in case: Please keep the scope of your PR, don't mix unrelated changes into a feature-PR