Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for npm lockfile version 3 #1206

Merged
merged 1 commit into from
Nov 18, 2022

Conversation

robcresswell
Copy link
Contributor

This PR adds support for npm lockfile version 3, which drops the "dependencies" key and uses "packages" instead. I've refactored the lockfile parser to make the distinction between the versions explicit rather than the implicit behaviour before. It might be worth splitting into separate files at some point, but the logic is so minimal that I haven't done it.

Some open questions;

  • Does the code look vaguely correct? I don't know Go well at all
  • I can't find good documentation around the presence of the "license" key under the "packages" entries. It seems to be present in the v2 fixture, but I couldn't recreate that locally.
  • Are there other places that I need to add / update tests?

Fixes #1203

@robcresswell robcresswell self-assigned this Sep 15, 2022
@github-actions
Copy link

github-actions bot commented Sep 15, 2022

Benchmark Test Results

Benchmark results from the latest changes vs base branch
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/alpmdb-cataloger-2                    11.5ms ± 1%    11.4ms ± 9%    ~     (p=0.190 n=4+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.32ms ± 1%    1.33ms ± 7%    ~     (p=0.690 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.37ms ± 1%    3.29ms ± 4%    ~     (p=0.151 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.09ms ± 1%    1.07ms ± 1%  -2.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         753µs ± 0%     739µs ± 2%  -1.82%  (p=0.008 n=5+5)
ImagePackageCatalogers/node-binary-cataloger-2               6.78µs ± 1%    6.77µs ± 1%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     867µs ± 1%     832µs ± 3%  -4.00%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                    1.28ms ± 1%    1.24ms ± 1%  -2.65%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      14.4ms ± 1%    14.1ms ± 2%  -1.81%  (p=0.016 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.30ms ± 1%    1.23ms ± 0%  -5.17%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          6.79µs ± 1%    6.90µs ± 3%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/dotnet-deps-cataloger-2               1.37ms ± 1%    1.41ms ± 3%  +2.79%  (p=0.032 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    719µs ± 0%     743µs ± 1%  +3.30%  (p=0.008 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                      4.44ms ± 0%    4.61ms ± 2%  +3.79%  (p=0.008 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/alpmdb-cataloger-2                    5.26MB ± 0%    5.26MB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               205kB ± 0%     205kB ± 0%    ~     (p=0.095 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             961kB ± 0%     961kB ± 0%    ~     (p=0.730 n=4+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     217kB ± 0%     217kB ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         159kB ± 0%     159kB ± 0%    ~     (p=0.310 n=5+5)
ImagePackageCatalogers/node-binary-cataloger-2               1.12kB ± 0%    1.12kB ± 0%    ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     199kB ± 0%     199kB ± 0%    ~     (p=0.548 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                     303kB ± 0%     303kB ± 0%    ~     (p=0.690 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.49MB ± 0%    3.49MB ± 0%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.26MB ± 0%    1.26MB ± 0%    ~     (p=0.151 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          1.12kB ± 0%    1.12kB ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                374kB ± 0%     375kB ± 0%  +0.12%  (p=0.008 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    139kB ± 0%     138kB ± 0%    ~     (p=0.056 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                       722kB ± 0%     722kB ± 0%  +0.02%  (p=0.008 n=5+5)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/alpmdb-cataloger-2                     85.7k ± 0%     85.7k ± 0%    ~     (p=0.159 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               4.24k ± 0%     4.24k ± 0%    ~     (all equal)
ImagePackageCatalogers/python-package-cataloger-2             16.5k ± 0%     16.5k ± 0%    ~     (p=0.444 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     5.50k ± 0%     5.50k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         3.33k ± 0%     3.33k ± 0%    ~     (all equal)
ImagePackageCatalogers/node-binary-cataloger-2                 38.0 ± 0%      38.0 ± 0%    ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     4.46k ± 0%     4.46k ± 0%    ~     (all equal)
ImagePackageCatalogers/rpm-db-cataloger-2                     8.11k ± 0%     8.11k ± 0%    ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       57.5k ± 0%     57.5k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      5.45k ± 0%     5.45k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            38.0 ± 0%      38.0 ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                7.12k ± 0%     7.12k ± 0%    ~     (all equal)
ImagePackageCatalogers/portage-cataloger-2                    3.58k ± 0%     3.58k ± 0%    ~     (all equal)
ImagePackageCatalogers/sbom-cataloger-2                       24.4k ± 0%     24.4k ± 0%    ~     (all equal)

@spiffcs
Copy link
Contributor

spiffcs commented Oct 19, 2022

Just to get this on the main thread this PR is BLOCKED: npm/cli#5532

@spiffcs spiffcs added the blocked Progress is being stopped by something label Oct 19, 2022
@robcresswell robcresswell marked this pull request as ready for review November 18, 2022 15:43
This PR adds support for npm lockfile version 3, which drops the
"dependencies" key and uses "packages" instead. I've refactored the
lockfile parser to make the distinction between the versions explicit
rather than the implicit behaviour before. It _might_ be worth splitting
into separate files at some point, but the logic is so minimal that I
haven't done it.

Fixes #1203
Signed-off-by: Rob Cresswell <robcresswell@users.noreply.github.com>
Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kzantow kzantow merged commit 9d8244b into main Nov 18, 2022
@kzantow kzantow deleted the feat/support-package-lock-v3 branch November 18, 2022 17:41
@Mikcl
Copy link
Contributor

Mikcl commented Nov 18, 2022

made some follow up comments
9d8244b

addressing in PR:
#1349

GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
This PR adds support for npm lockfile version 3, which drops the
"dependencies" key and uses "packages" instead. I've refactored the
lockfile parser to make the distinction between the versions explicit
rather than the implicit behaviour before. It _might_ be worth splitting
into separate files at some point, but the logic is so minimal that I
haven't done it.

Fixes anchore#1203
Signed-off-by: Rob Cresswell <robcresswell@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Progress is being stopped by something
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NPM package-lock.json version 3
5 participants