Add digest property to parent and nested java package metadata

[spiffcs]

Summary

Adding digests to the discovered java packages will give downstream SBOM consumers more options for querying artifact registries regarding specific details surrounding SBOM content.

A followup PR in grype will be made after this to allow it to optionally consume this information for matching against an upstream Maven repository for better vulnerability match fidelity

Addtions

• Add digest property to JavaMetadata
• Populate external formats (cyclonedx, cyclonedx-json, spdx, spdx-json, syft-json) with new digest values

Todo

• Update CyconeDX and SPDX output with new digest properties and filesAnalyzed switch
• Refactor and export digest cataloger functionality to be shared across code

Signed-off-by: Christopher Phillips christopher.phillips@anchore.com

[github-actions]

Benchmark Test Results

Benchmark results from the latest changes vs base branch

name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.04ms ± 4%    1.45ms ± 3%  +39.84%  (p=0.008 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            2.59ms ± 1%    3.78ms ± 0%  +45.96%  (p=0.016 n=5+4)
ImagePackageCatalogers/php-composer-installed-cataloger-2     877µs ±10%    1189µs ± 3%  +35.53%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         624µs ± 4%     793µs ± 2%  +27.06%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     758µs ± 2%     940µs ± 2%  +23.93%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      693µs ± 5%     832µs ± 3%  +20.05%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      12.3ms ± 4%    17.3ms ± 2%  +40.38%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.16ms ± 4%    1.45ms ± 3%  +25.00%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          2.02µs ± 1%    2.59µs ± 3%  +28.44%  (p=0.008 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               185kB ± 0%     184kB ± 0%   -0.21%  (p=0.016 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             896kB ± 0%     894kB ± 0%   -0.19%  (p=0.008 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     196kB ± 0%     196kB ± 0%     ~     (p=0.690 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         140kB ± 0%     140kB ± 0%     ~     (p=0.690 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     175kB ± 0%     175kB ± 0%     ~     (p=0.548 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      163kB ± 0%     163kB ± 0%     ~     (p=0.841 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.19MB ± 0%    3.29MB ± 0%   +3.32%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.24MB ± 0%    1.24MB ± 0%     ~     (p=0.310 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            672B ± 0%      672B ± 0%     ~     (all equal)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/ruby-gemspec-cataloger-2               3.66k ± 0%     3.66k ± 0%     ~     (all equal)
ImagePackageCatalogers/python-package-cataloger-2             14.8k ± 0%     14.8k ± 0%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     4.94k ± 0%     4.94k ± 0%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         2.72k ± 0%     2.72k ± 0%     ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     3.93k ± 0%     3.93k ± 0%     ~     (all equal)
ImagePackageCatalogers/rpmdb-cataloger-2                      4.01k ± 0%     4.01k ± 0%     ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       52.2k ± 0%     52.2k ± 0%   +0.10%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      4.81k ± 0%     4.81k ± 0%     ~     (p=0.992 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            15.0 ± 0%      15.0 ± 0%     ~     (all equal)

[kzantow]

Generally, this looks okay, but I'm a little concerned about the java/archive_parser_test.go that these hashes are different locally vs CI, what is the reason for this?

[spiffcs]

Thanks for the feedback everyone! I'll take another stab and incorporate your comments. I committed some of the suggested changes and need to start looking at what's failing.

[spiffcs]

@wagoodman and @kzantow this has now been updated to address the last round of comments

[kzantow]

I had another pass at this, I think I identified one nil pointer panic, where there's for _, hash := range *ref.Hashes {, I'm pretty sure ref could be nil if not found.

One other thing is the CycloneDX decoder is handling the hashes, it looks like but there isn't anything for SPDX decoding. You're probably not seeing any failing tests because they haven't been added to the encode-decode-encode cycle test. They should be but I suspect that would involve some work outside the scope of this PR. So know we either need a follow-on task to do this (which I would be okay with, personally, to add the spdx formats to encode-decode-encode, which would involve adding the decoding support for the digests) or add decoding support here.

[wagoodman]

nit: could unexport this

[wagoodman]

nice work 🙌

[spiffcs]

@kzantow @wagoodman I added decoding for spdx22 support can you take a look