Upgraded dependencies with CVEs #2792

RaulGracia · 2021-09-14T10:24:19Z

Motivation

Minor upgrades of several dependencies to resolve publicly reported security vulnerabilities.

Changes

Minor upgrades of the following libraries:

bouncycastle: 1.69
commonsIO: 2.7
jetty: 9.4.43.v20210629
log4j: 1.2.27

Master Issue: #2791

Signed-off-by: Raúl <raul.gracia@emc.com>

eolivelli

LGTM

@lhotari PTAL as well

eolivelli · 2021-09-14T12:35:18Z

@RaulGracia LICENSE check failed PTAL

dev/check-binary-license dev/../bookkeeper-dist/server/target/bookkeeper-server-4.15.0-SNAPSHOT-bin.tar.gz
commons-io-commons-io-2.7.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-http-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-io-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-security-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-server-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-servlet-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-util-9.4.43.v20210629.jar unaccounted for in LICENSE
org.eclipse.jetty-jetty-util-ajax-9.4.43.v20210629.jar unaccounted for in LICENSE
org.slf4j-slf4j-api-1.7.32.jar unaccounted for in LICENSE
org.slf4j-slf4j-log4j12-1.7.32.jar unaccounted for in LICENSE
commons-io-commons-io-2.4.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-http-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-io-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-security-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar mentioned in LICENSE, but not bundled
org.slf4j-slf4j-api-1.7.25.jar mentioned in LICENSE, but not bundled
org.slf4j-slf4j-log4j12-1.7.25.jar mentioned in LICENSE, but not bundled
org.eclipse.jetty-jetty-http-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-io-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-security-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled
org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar mentioned in NOTICE, but not bundled

nicoloboschi

+1

Signed-off-by: Raúl <raul.gracia@emc.com>

RaulGracia · 2021-09-28T08:54:03Z

@eolivelli Can this one be merged?

eolivelli · 2021-09-28T20:36:09Z

@RaulGracia I am restarting the failed job.
I will merge as soon as CI passes

eolivelli

Lgtm

pkumar-singh · 2021-10-13T22:07:45Z

dependencies.gradle

    jmh: "1.19",
    jmock: "2.8.2",
    jna: "3.2.7",
    jsr305: "3.0.2",
    junit: "4.12",
    junitFoundation: "11.0.0",
    kerby: "1.1.1",
-    log4j: "1.2.17",
+    log4j: "1.2.27",


This version does not even exist.

@pkumar-singh seems that I made a mistake with this version number and the build compiled and passed all the checks, I'm very sorry for that. PR #2816 upgrades Bookkeeper to log4j2, which basically replaces this version and should fix this problem. @eolivelli before merging #2816 and fix the build, I will perform tomorrow (my morning) additional checks to validate that we are not going to find any problem when the PR is merged (looks like the PR build is not covering everything).

No worries. I will take care of this.

BP-44: USE metrics. A proposal for improving BookKeeper metrics so that operators can employ the USE method for diagnosing performance issues. Reviewers: Henry Saputra <hsaputra@apache.org>, Andrey Yegorov <None>, Enrico Olivelli <eolivelli@gmail.com> This closes #2835 from Vanlightly/BP-44-use-metrics and squashes the following commits: 8d9baab [Jack Vanlightly] Added link to USE method and listed each term of USE 5a0f67d [Jack Vanlightly] BP-44 USE metrics a9b576d [Yunze Xu] Release semaphore when addEntry accepts the same entries (#2832) 148bf22 [Yun Tang] Ensure to release cache during KeyValueStorageRocksDB#closec (#2821) 4dc4260 [gaozhangmin] Heap memory leak problem when ledger replication failed (#2794) a522fa3 [Raúl Gracia] Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571 (#2816) 0465052 [Nicolò Boschi] Upgrade httpclient from 4.5.5 to 4.5.13 (#2793) 594a056 [Raúl Gracia] Issue 2795: Bookkeeper upgrade using Bookie ID may fail due to cookie mismatch (#2796) 354cf37 [Raúl Gracia] Upgraded dependencies with CVEs (#2792) e413c70 [Raúl Gracia] Issue 2728: Entry Log GC may get blocked when using entryLogPerLedgerEnabled option (#2779) 883231e [pradeepbn] Building bookkeeper with gradle on java11

(cherry picked from commit 354cf37)

### Motivation In order to complete migration to Gradle we must build all the subprojects. ### Changes - Enabled `sh` integration tests with gradle, located in `tests/scripts/src/test/bash/gradle` - Added these modules to the build - `bookkeeper-http:servlet-http-server` - `metadata-drivers:etcd` - `tests:backward-compat:*` - `tests:shaded:*` - `stream:bk-grpc-name-resolver` - DL shading process is now performed (before it didn't build any jar) - Groovy tests (`tests:backward-compat:*`) now are triggered by the build/tests itself; with Maven, there is a "runner" project (`tests/integration-tests-base-groovy`); in Gradle is useless so it is skipped ### Test - Both `bin/bookkeper standalone` and `bin/bookkeper_gradle standalone` work locally - Tests are passing locally Master Issue: #2849 Reviewers: Henry Saputra <hsaputra@apache.org>, Prashant Kumar <None> This closes #2850 from nicoloboschi/fix/2849/gradle and squashes the following commits: 00b49f4 [Nicolò Boschi] Fix common_gradle.sh regex bd739fd [Nicolò Boschi] fix sh tests 43230ba [Nicolò Boschi] revert sh files. Avoid to modify maven files, create gradle versions to faciltate migration d1f95e4 [Nicolò Boschi] fix shaded deps bcab40d [Nicolò Boschi] fix build 5fd0341 [Nicolò Boschi] fix build 0082e0e [Nicolò Boschi] fix build 2c32ac1 [Nicolò Boschi] fixes 3bc0b26 [Nicolò Boschi] bookkeeper-server-shaded-tests ba89132 [Nicolò Boschi] shaded tests 6d39e33 [Nicolò Boschi] sh tests e0032bc [Nicolò Boschi] actually run arquillian groovy tests 08dcc39 [Nicolò Boschi] backwards 2361f79 [Nicolò Boschi] hierarchical-ledger-manager 8388e11 [Nicolò Boschi] current-server-old-clients 6a24344 [Nicolò Boschi] bc-non-fips 2faca01 [Nicolò Boschi] bk-grpc-name-resolver 991bc11 [Nicolò Boschi] servlet-http-server 675ef7b [Nicolò Boschi] etcd b1d5e14 [ZhangJian He] A empty implement in EtcdLedgerManagerFactory to let the project can compile (#2845) bd5c50b [shustsud] Add error handling to readLedgerMetadata in over-replicated ledger GC (#2844) 746f9f6 [Matteo Merli] Remove direct ZK access for Auditor (#2842) 4117200 [ZhangJian He] the compare should be >= instead of > (#2782) 14ef56f [Prashant Kumar] BookieId can not be cast to BookieSocketAddress (#2843) e10f3fe [ZhangJian He] Forget to close preAllocator log on shutdown (#2819) 53954ca [shustsud] Add ensemble check to over-replicated ledger GC (#2813) 919fdd3 [Prashant Kumar] Issue:2840 Create bookie shellscript for gradle (#2841) 031d168 [gaozhangmin] fix-npe-when-pulsar-ZkBookieRackAffinityMapping-getBookieAddressResolver (#2788) 3dd671c [Prashant Kumar] Migrate bookkeepr-server:test to gradle run unit tests excepts org.apache.bookkeeper.bookie. org.apache.bookkeeper.client org.apache.bookkeeper.replication org.apache.bookkeeper.tls. (#2812) f6903b8 [Jack Vanlightly] BP-44 USE metrics a4afaa4 [Matteo Merli] Eliminate direct ZK access in ScanAndCompareGarbageCollector (#2833) a9b576d [Yunze Xu] Release semaphore when addEntry accepts the same entries (#2832) 148bf22 [Yun Tang] Ensure to release cache during KeyValueStorageRocksDB#closec (#2821) 4dc4260 [gaozhangmin] Heap memory leak problem when ledger replication failed (#2794) a522fa3 [Raúl Gracia] Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571 (#2816) 0465052 [Nicolò Boschi] Upgrade httpclient from 4.5.5 to 4.5.13 (#2793) 594a056 [Raúl Gracia] Issue 2795: Bookkeeper upgrade using Bookie ID may fail due to cookie mismatch (#2796) 354cf37 [Raúl Gracia] Upgraded dependencies with CVEs (#2792) e413c70 [Raúl Gracia] Issue 2728: Entry Log GC may get blocked when using entryLogPerLedgerEnabled option (#2779) 883231e [pradeepbn] Building bookkeeper with gradle on java11

BP-44: USE metrics. A proposal for improving BookKeeper metrics so that operators can employ the USE method for diagnosing performance issues. Reviewers: Henry Saputra <hsaputra@apache.org>, Andrey Yegorov <None>, Enrico Olivelli <eolivelli@gmail.com> This closes apache#2835 from Vanlightly/BP-44-use-metrics and squashes the following commits: 8d9baab [Jack Vanlightly] Added link to USE method and listed each term of USE 5a0f67d [Jack Vanlightly] BP-44 USE metrics a9b576d [Yunze Xu] Release semaphore when addEntry accepts the same entries (apache#2832) 148bf22 [Yun Tang] Ensure to release cache during KeyValueStorageRocksDB#closec (apache#2821) 4dc4260 [gaozhangmin] Heap memory leak problem when ledger replication failed (apache#2794) a522fa3 [Raúl Gracia] Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571 (apache#2816) 0465052 [Nicolò Boschi] Upgrade httpclient from 4.5.5 to 4.5.13 (apache#2793) 594a056 [Raúl Gracia] Issue 2795: Bookkeeper upgrade using Bookie ID may fail due to cookie mismatch (apache#2796) 354cf37 [Raúl Gracia] Upgraded dependencies with CVEs (apache#2792) e413c70 [Raúl Gracia] Issue 2728: Entry Log GC may get blocked when using entryLogPerLedgerEnabled option (apache#2779) 883231e [pradeepbn] Building bookkeeper with gradle on java11

### Motivation In order to complete migration to Gradle we must build all the subprojects. ### Changes - Enabled `sh` integration tests with gradle, located in `tests/scripts/src/test/bash/gradle` - Added these modules to the build - `bookkeeper-http:servlet-http-server` - `metadata-drivers:etcd` - `tests:backward-compat:*` - `tests:shaded:*` - `stream:bk-grpc-name-resolver` - DL shading process is now performed (before it didn't build any jar) - Groovy tests (`tests:backward-compat:*`) now are triggered by the build/tests itself; with Maven, there is a "runner" project (`tests/integration-tests-base-groovy`); in Gradle is useless so it is skipped ### Test - Both `bin/bookkeper standalone` and `bin/bookkeper_gradle standalone` work locally - Tests are passing locally Master Issue: apache#2849 Reviewers: Henry Saputra <hsaputra@apache.org>, Prashant Kumar <None> This closes apache#2850 from nicoloboschi/fix/2849/gradle and squashes the following commits: 00b49f4 [Nicolò Boschi] Fix common_gradle.sh regex bd739fd [Nicolò Boschi] fix sh tests 43230ba [Nicolò Boschi] revert sh files. Avoid to modify maven files, create gradle versions to faciltate migration d1f95e4 [Nicolò Boschi] fix shaded deps bcab40d [Nicolò Boschi] fix build 5fd0341 [Nicolò Boschi] fix build 0082e0e [Nicolò Boschi] fix build 2c32ac1 [Nicolò Boschi] fixes 3bc0b26 [Nicolò Boschi] bookkeeper-server-shaded-tests ba89132 [Nicolò Boschi] shaded tests 6d39e33 [Nicolò Boschi] sh tests e0032bc [Nicolò Boschi] actually run arquillian groovy tests 08dcc39 [Nicolò Boschi] backwards 2361f79 [Nicolò Boschi] hierarchical-ledger-manager 8388e11 [Nicolò Boschi] current-server-old-clients 6a24344 [Nicolò Boschi] bc-non-fips 2faca01 [Nicolò Boschi] bk-grpc-name-resolver 991bc11 [Nicolò Boschi] servlet-http-server 675ef7b [Nicolò Boschi] etcd b1d5e14 [ZhangJian He] A empty implement in EtcdLedgerManagerFactory to let the project can compile (apache#2845) bd5c50b [shustsud] Add error handling to readLedgerMetadata in over-replicated ledger GC (apache#2844) 746f9f6 [Matteo Merli] Remove direct ZK access for Auditor (apache#2842) 4117200 [ZhangJian He] the compare should be >= instead of > (apache#2782) 14ef56f [Prashant Kumar] BookieId can not be cast to BookieSocketAddress (apache#2843) e10f3fe [ZhangJian He] Forget to close preAllocator log on shutdown (apache#2819) 53954ca [shustsud] Add ensemble check to over-replicated ledger GC (apache#2813) 919fdd3 [Prashant Kumar] Issue:2840 Create bookie shellscript for gradle (apache#2841) 031d168 [gaozhangmin] fix-npe-when-pulsar-ZkBookieRackAffinityMapping-getBookieAddressResolver (apache#2788) 3dd671c [Prashant Kumar] Migrate bookkeepr-server:test to gradle run unit tests excepts org.apache.bookkeeper.bookie. org.apache.bookkeeper.client org.apache.bookkeeper.replication org.apache.bookkeeper.tls. (apache#2812) f6903b8 [Jack Vanlightly] BP-44 USE metrics a4afaa4 [Matteo Merli] Eliminate direct ZK access in ScanAndCompareGarbageCollector (apache#2833) a9b576d [Yunze Xu] Release semaphore when addEntry accepts the same entries (apache#2832) 148bf22 [Yun Tang] Ensure to release cache during KeyValueStorageRocksDB#closec (apache#2821) 4dc4260 [gaozhangmin] Heap memory leak problem when ledger replication failed (apache#2794) a522fa3 [Raúl Gracia] Issue 2815: Upgrade to log4j2 to get rid of CVE-2019-17571 (apache#2816) 0465052 [Nicolò Boschi] Upgrade httpclient from 4.5.5 to 4.5.13 (apache#2793) 594a056 [Raúl Gracia] Issue 2795: Bookkeeper upgrade using Bookie ID may fail due to cookie mismatch (apache#2796) 354cf37 [Raúl Gracia] Upgraded dependencies with CVEs (apache#2792) e413c70 [Raúl Gracia] Issue 2728: Entry Log GC may get blocked when using entryLogPerLedgerEnabled option (apache#2779) 883231e [pradeepbn] Building bookkeeper with gradle on java11

RaulGracia added 2 commits September 14, 2021 11:50

Upgraded dependencies with CVEs

ddf6fb8

Signed-off-by: Raúl <raul.gracia@emc.com>

Added upgrades in licenses file.

9540ad4

Signed-off-by: Raúl <raul.gracia@emc.com>

eolivelli requested review from dlg99, jiazhai and merlimat September 14, 2021 12:34

eolivelli approved these changes Sep 14, 2021

View reviewed changes

nicoloboschi approved these changes Sep 14, 2021

View reviewed changes

RaulGracia added 3 commits September 14, 2021 16:21

Updating license files.

4c17764

Signed-off-by: Raúl <raul.gracia@emc.com>

Fixing license check.

da224ac

Signed-off-by: Raúl <raul.gracia@emc.com>

Renamed slf4j license dir.

6cbebcc

Signed-off-by: Raúl <raul.gracia@emc.com>

dlg99 approved these changes Sep 24, 2021

View reviewed changes

eolivelli approved these changes Sep 28, 2021

View reviewed changes

eolivelli merged commit 354cf37 into apache:master Oct 13, 2021

eolivelli assigned RaulGracia Oct 13, 2021

eolivelli added area/security dependencies Pull requests that update a dependency file labels Oct 13, 2021

eolivelli added this to the 4.15.0 milestone Oct 13, 2021

pkumar-singh reviewed Oct 13, 2021

View reviewed changes

This was referenced Oct 14, 2021

Fix gradle.dependencies related to PR 2792 #2825

Closed

Issue 2825: Fix gradle.dependencies related to PR 2792 #2826

Closed

sijie mentioned this pull request Oct 14, 2021

ISSUE-2825: Fix gradle.dependencies related to PR 2792 streamnative/bookkeeper-achieved#414

Closed

RaulGracia mentioned this pull request Oct 15, 2021

Fix multiple CVEs related to library versions #2791

Closed

zymap added the release/4.14.3 label Oct 25, 2021

zymap pushed a commit that referenced this pull request Oct 26, 2021

Upgraded dependencies with CVEs (#2792)

911c2e9

(cherry picked from commit 354cf37)

Ghatage pushed a commit to sijie/bookkeeper that referenced this pull request Jul 12, 2024

Upgraded dependencies with CVEs (apache#2792)

d674a61

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgraded dependencies with CVEs #2792

Upgraded dependencies with CVEs #2792

RaulGracia commented Sep 14, 2021

eolivelli left a comment

eolivelli commented Sep 14, 2021

nicoloboschi left a comment

RaulGracia commented Sep 28, 2021

eolivelli commented Sep 28, 2021

eolivelli left a comment

pkumar-singh Oct 13, 2021

RaulGracia Oct 13, 2021

pkumar-singh Oct 13, 2021

Upgraded dependencies with CVEs #2792

Upgraded dependencies with CVEs #2792

Conversation

RaulGracia commented Sep 14, 2021

Motivation

Changes

eolivelli left a comment

Choose a reason for hiding this comment

eolivelli commented Sep 14, 2021

nicoloboschi left a comment

Choose a reason for hiding this comment

RaulGracia commented Sep 28, 2021

eolivelli commented Sep 28, 2021

eolivelli left a comment

Choose a reason for hiding this comment

pkumar-singh Oct 13, 2021

Choose a reason for hiding this comment

RaulGracia Oct 13, 2021

Choose a reason for hiding this comment

pkumar-singh Oct 13, 2021

Choose a reason for hiding this comment