Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for FIPS compliance mode #14912

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Support for FIPS compliance mode #14912

wants to merge 11 commits into from
+2,481 −2,004

Conversation

beanuwave
Copy link

@beanuwave beanuwave commented Jul 23, 2024 •
edited
Loading

Description

  • FIPS gradle build script is removed in order to support single build solution.
  • All BC dependencies are replaces by BCFIPS.
  • Password matcher inside Identity-Shiro that replies on BC to check if hashed passwords matches with OpenBSDBCrypt, is replaced by password4j implementation.
  • Adds full support for BCFKS format (*.bcfips) for Key & Truststores, also making it default.
  • Makes best guess which store type is provided based on filename extension.
  • Store types are strict and only allow JKS, PKCS12, PKCS11 and BCFKS.
  • Refactor parsing private keys with formats EC, PKCS8, PKCS1, DSA, w/wo encryption, w/wo parameters.
  • FIPS mode can be configured with environment variable OPENSEARCH_CRYPTO_STANDARD=FIPS-140-3.
  • java security file is added to the build, to distinguish between FIPS and non FIPS environments.
  • java security file for FIPS is altered due to evolving security standards.
  • java policy file is altered to grant necessary security permissions.

This PR provides FIPS 140-3 support by replacing all BC dependencies with BCFIPS dependencies and making FIPS approved-only mode configurable at launch. Running application in approved-only mode restricts BCFIPS provoder to rely solely on FIPS certified cyphers. Due to replacement of BC libraries, BCrypt password matching and private-key loading from file were replaced by alternative implementations.

Reasons for refactoring PemUtils.java that is used by Reindex API, in case of migrating data from a remote cluster that is TLS protected:

  • PKCS#8 implementation was not supported by BCFIPS library.
  • java type security.
  • Password Based Key Derivation Functions such as PKCS#12 and OpenSSL are not supported in BCFIPS approved-only mode, because only PBKDF2 standard is approved for use in FIPS.
  • generally good idea to let ASN1 annotation parsing be done by external security libraries.

Related Issues

opensearch-project/security#3420

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 6016d5d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@beanuwave beanuwave changed the title Draft to allow run in FIPS compliace mode Draft to allow run in FIPS compliance mode Jul 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 8e8ed47: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 6016d5d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@beanuwave beanuwave marked this pull request as ready for review July 24, 2024 10:02
@dblock
Copy link
Member

dblock commented Jul 24, 2024

Could use some help maybe from @cwperks or @peternied reviewing this, please.

@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for 2aa0d1d: SUCCESS

terryquigleysas
terryquigleysas reviewed Oct 25, 2024
View reviewed changes
build.gradle Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 246aeed: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for 7e16ec6: SUCCESS

@opensearch-ci-bot opensearch-ci-bot mentioned this pull request Oct 25, 2024
Open
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 350a69e: SUCCESS

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@iigonin
Draft to allow run in FIPS compliace mode
9bfa3f0 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>

# Conflicts:
#	server/build.gradle
@iigonin
make tests run without BC (not BCFIPS) libraries.
3e7d2d1 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>

� Please enter the commit message for your changes. Lines starting
� with '�' will be ignored, and an empty message aborts the commit.
�
� interactive rebase in progress; onto 4b284c5
� Last commands done (2 commands done):
�    pick a47f4e6 Draft to allow run in FIPS compliace mode
�    pick 0bee0a8 make tests run without BC (not BCFIPS) libraries.
� Next commands to do (8 remaining commands):
�    pick 4fc6201 disable approved-only mode for launch configuration of testcluster
�    pick 321929f update all BC libraries to support JAVA 21
� You are currently rebasing branch 'fips_compliance2' on '4b284c54270'.
�
� Changes to be committed:
�	modified:   buildSrc/build.gradle
�	modified:   buildSrc/src/main/java/org/opensearch/gradle/OpenSearchTestBasePlugin.java
�	modified:   buildSrc/src/main/java/org/opensearch/gradle/info/BuildParams.java
�	modified:   client/rest/build.gradle
�	new file:   client/rest/licenses/bc-fips-1.0.2.4.jar.sha1
�	new file:   client/rest/licenses/bctls-fips-1.0.19.jar.sha1
�	new file:   client/rest/licenses/bouncycastle-LICENSE.txt
�	new file:   client/rest/licenses/bouncycastle-NOTICE.txt
�	modified:   client/rest/src/test/java/org/opensearch/client/RestClientBuilderIntegTests.java
�	modified:   distribution/src/config/fips_java.security
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/AddFileKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/AddStringKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/ChangeKeyStorePasswordCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/KeyStoreWrapperTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/ListKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/RemoveSettingKeyStoreCommandTests.java
�	modified:   distribution/tools/launchers/src/main/java/org/opensearch/tools/launchers/SystemJvmOptions.java
�	modified:   distribution/tools/plugin-cli/build.gradle
�	modified:   gradle/libs.versions.toml
�	modified:   libs/ssl-config/build.gradle
�	deleted:    libs/ssl-config/licenses/bc-fips-1.0.2.5.jar.sha1
�	new file:   libs/ssl-config/licenses/bouncycastle-LICENSE.txt
�	new file:   libs/ssl-config/licenses/bouncycastle-NOTICE.txt
�	modified:   libs/ssl-config/src/main/java/org/opensearch/common/ssl/DefaultJdkTrustConfig.java
�	modified:   libs/ssl-config/src/main/java/org/opensearch/common/ssl/PemUtils.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemKeyConfigTests.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemTrustConfigTests.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemUtilsTests.java
�	modified:   modules/reindex/src/test/java/org/opensearch/index/reindex/ReindexRestClientSslTests.java
�	modified:   modules/transport-netty4/build.gradle
�	modified:   modules/transport-netty4/src/test/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransportTests.java
�	modified:   modules/transport-netty4/src/test/java/org/opensearch/transport/netty4/ssl/SimpleSecureNetty4TransportTests.java
�	deleted:    modules/transport-netty4/src/test/resources/netty4-secure.jks
�	new file:   modules/transport-netty4/src/test/resources/netty4-secure.p12
�	modified:   plugins/discovery-azure-classic/src/internalClusterTest/java/org/opensearch/discovery/azure/classic/AzureDiscoveryClusterFormationTests.java
�	deleted:    plugins/identity-shiro/licenses/bcprov-jdk18on-1.78.jar.sha1
�	deleted:    plugins/identity-shiro/licenses/bcprov-jdk18on-LICENSE.txt
�	new file:   plugins/identity-shiro/licenses/password4j-1.8.2.jar.sha1
�	new file:   plugins/identity-shiro/licenses/password4j-LICENSE.txt
�	renamed:    plugins/identity-shiro/licenses/bcprov-jdk18on-NOTICE.txt -> plugins/identity-shiro/licenses/password4j-NOTICE.txt
�	modified:   plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/realm/BCryptPasswordMatcher.java
�	modified:   plugins/repository-azure/build.gradle
�	modified:   plugins/telemetry-otel/build.gradle
�	modified:   server/build.gradle
�	new file:   server/licenses/bc-fips-1.0.2.4.jar.sha1
�	new file:   server/licenses/bctls-fips-1.0.19.jar.sha1
�	new file:   server/licenses/bouncycastle-LICENSE.txt
�	new file:   server/licenses/bouncycastle-NOTICE.txt
�	modified:   server/src/main/java/org/opensearch/bootstrap/Bootstrap.java
�	modified:   server/src/main/java/org/opensearch/common/settings/FipsSettings.java
�	modified:   server/src/main/java/org/opensearch/common/settings/KeyStoreWrapper.java
�	modified:   server/src/main/resources/org/opensearch/bootstrap/security.policy
�	modified:   server/src/main/resources/org/opensearch/bootstrap/test-framework.policy
�
@iigonin
disable approved-only mode for launch configuration of testcluster
5e25781 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>

# Conflicts:
#	buildSrc/version.properties
@iigonin
update all BC libraries to support JAVA 21
94534e3 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
try to pass bwc staged build by adding expected version
e0b8bc6 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
ensure FIPS-mode can be enabled by fixing existent and creating addit…
3abd861 
…ional tests.

Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
fixing additional tests.
bcc9440 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
Configure FIPS mode through global ENV VAR instead of thread-based.
2863739 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
write some additional unit tests.
cf4c268 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@iigonin
test SystemJvmOptions.java & replace FIPS-140-2 by FIPS-140-3
26fa0b1 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for 26fa0b1: SUCCESS

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 8, 2024

❌ Gradle check result for 7a44f38: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@iigonin
categorize JKS keystore as untrusted
17f6d8e 
Signed-off-by: Iwan Igonin <iigonin@sternad.de>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 8, 2024

❌ Gradle check result for 17f6d8e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@terryquigleysas terryquigleysas terryquigleysas left review comments

@stephen-crawford stephen-crawford stephen-crawford left review comments

@cwperks cwperks cwperks left review comments

@dancristiancecoi dancristiancecoi dancristiancecoi left review comments

@peternied peternied peternied requested changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@ashking94 ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@jainankitk jainankitk Awaiting requested review from jainankitk jainankitk is a code owner

@linuxpi linuxpi Awaiting requested review from linuxpi linuxpi is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@beanuwave @dblock @cwperks @peternied @terryquigleysas @dancristiancecoi @stephen-crawford @iigonin