[TEST] Run pre 6.4 nodes in non-FIPS JVMs #32901

jkakavas · 2018-08-16T10:05:19Z

Elasticsearch versions earlier than 6.4.0 cannot properly run in a
FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for
nodes that we spin up in BWC tests even when we're testing FIPS.

Resolves #32737

It also reverts e497173 and e64bb48 and as such resolves #32868

Elasticsearch versions earlier than 6.4.0 cannot properly run in a FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for nodes that we spin up in BWC tests even when we're testing FIPS.

…JVM (elastic#32775)" This reverts commit e497173.

This reverts commit e64bb48.

rjernst · 2018-08-16T15:28:50Z

buildSrc/src/main/groovy/org/elasticsearch/gradle/test/NodeInfo.groovy

@@ -177,6 +177,9 @@ class NodeInfo {
            javaVersion = 8
        } else if (nodeVersion.onOrAfter("6.2.0") && nodeVersion.before("6.3.0")) {
            javaVersion = 9
+        } else if (project.inFipsJvm && nodeVersion.onOrAfter("6.3.0") && nodeVersion.before("6.4.0")) {


I'm sure I'm missing something, but wouldn't this only change the version that we try to start on, but not stop the node from attempting to start?

Correct. The intention is to not mute the tests but keep running them even when we're in a FIPS JVM in CI. The way to achieve this is to make sure that older ES version ( not supporting fips ) nodes start with a non fips java version

This is not at all clear from the code. Can you please add a comment explaining it? If I understand correctly, by adding this other elseif condition, non fips testing will fall through and continue using the RUNTIME_JAVA_HOME? But isn't that a fips jvm in this case?

This is not at all clear from the code. Can you please add a comment explaining it?

Sure thing. I tried to capture it in // Versions before 6.4.0 cannot be run in a FIPS 140 JVM but I agree that's not very clear.

If I understand correctly, by adding this other elseif condition, non fips testing will fall through and continue using the RUNTIME_JAVA_HOME? But isn't that a fips jvm in this case?

Not sure I follow your thought. When RUNTIME_JAVA_HOME is a fips JVM, project.inFipsJvm will also be true.
In summary:

When running in a non FIPS JVM

Nothing changes from the previous behavior

We run < 6.2.0 nodes with Java 8

We run > 6.2.0 and < 6.3.0 nodes with Java 9

We run > 6.3.0 nodes with RUNTIME_JAVA_HOME ( non FIPS )

When running in a FIPS JVM

project.inFipsJvm is true

We run < 6.2.0 nodes with Java 8 (non fips)

We run > 6.2.0 and < 6.3.0 nodes with Java 9 (non FIPS)

We run > 6.3.0 and < 6.4.0 nodes with Java 10 (non FIPS)

We run > 6.4.0 nodes with RUNTIME_JAVA_HOME ( which is FIPS but > 6.4 nodes can run fine in a FIPS JVM)

Does this make more sense ? Bear with me if I've missed your point entirely.

Thanks, that is more clear.

@rjernst I updated the comment, let me know if this is clear enough, thanks !

rjernst

LGTM

jkakavas · 2018-08-17T05:05:16Z

jenkins test this please

jkakavas · 2018-08-17T07:53:22Z

Jenkins test this please

jkakavas · 2018-08-17T14:01:33Z

I'd like to merge this to get the fips CI green before the weekend. It seems that packaging test runs are still not getting started but the changes in here don't have any impact on packaging. Thoughts @rjernst ?

rjernst · 2018-08-17T15:15:55Z

I think this is ok to merge.

Elasticsearch versions earlier than 6.4.0 cannot properly run in a FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for nodes that we spin up in BWC tests even when we're testing FIPS.

* master: NETWORKING: Make RemoteClusterConn. Lazy Resolve DNS (elastic#32764) [DOCS] Splits the users API documentation into multiple pages (elastic#32825) [DOCS] Splits the token APIs into separate pages (elastic#32865) [DOCS] Creates redirects for role management APIs page Bypassing failing test PainlessDomainSplitIT#testHRDSplit (elastic#32966) TEST: Mute testRetentionPolicyChangeDuringRecovery [DOCS] Fixes more broken links to role management APIs [Docs] Tweaks and fixes to rollup docs [DOCS] Fixes links to role management APIs [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature [DOCS] Splits the roles API documentation into multiple pages (elastic#32794) [TEST] Run pre 6.4 nodes in non-FIPS JVMs (elastic#32901) Make Geo Context Mapping Parsing More Strict (elastic#32821)

* elastic/master: (46 commits) NETWORKING: Make RemoteClusterConn. Lazy Resolve DNS (#32764) [DOCS] Splits the users API documentation into multiple pages (#32825) [DOCS] Splits the token APIs into separate pages (#32865) [DOCS] Creates redirects for role management APIs page Bypassing failing test PainlessDomainSplitIT#testHRDSplit (#32966) TEST: Mute testRetentionPolicyChangeDuringRecovery [DOCS] Fixes more broken links to role management APIs [Docs] Tweaks and fixes to rollup docs [DOCS] Fixes links to role management APIs [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature [DOCS] Splits the roles API documentation into multiple pages (#32794) [TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901) Make Geo Context Mapping Parsing More Strict (#32821) [ML] fix updating opened jobs scheduled events (#31651) (#32881) Scripted metric aggregations: add deprecation warning and system property to control legacy params (#31597) Tests: Fix timezone conversion in DateTimeUnitTests Enable FIPS140LicenseBootstrapCheck (#32903) Fix InternalAutoDateHistogram reproducible failure (#32723) Remove assertion in testDocStats on deletedDocs counter (#32914) HLRC: Move ML request converters into their own class (#32906) ...

* 6.x: (42 commits) [DOCS] Splits the users API documentation into multiple pages (#32825) [DOCS] Splits the token APIs into separate pages (#32865) [DOCS] Creates redirects for role management APIs page Bypassing failing test PainlessDomainSplitIT#testHRDSplit (#32966) TEST: Mute testRetentionPolicyChangeDuringRecovery [DOCS] Fixes more broken links to role management APIs [Docs] Tweaks and fixes to rollup docs [DOCS] Fixes links to role management APIs [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature [DOCS] Splits the roles API documentation into multiple pages (#32794) [TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901) Remove assertion in testDocStats on deletedDocs counter (#32914) [ML] fix updating opened jobs scheduled events (#31651) (#32881) Enable FIPS140LicenseBootstrapCheck (#32903) HLRC: Move ML request converters into their own class (#32906) [DOCS] Update getting-started.asciidoc (#29518) Fix allowed value for HighlighterBuilder encoder in javadocs (#32780) [DOCS] Add "remove a tag" script logic as an example (#32556) RFC: Test that example plugins build stand-alone (#32235) Security: remove put privilege API (#32879) ...

jkakavas added 3 commits August 16, 2018 12:55

Run pre 6.4 nodes in non-FIPS JVMs

bbfd15e

Elasticsearch versions earlier than 6.4.0 cannot properly run in a FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for nodes that we spin up in BWC tests even when we're testing FIPS.

Revert "Tests: Disable rolling upgrade tests with system key on fips …

89f73cc

…JVM (elastic#32775)" This reverts commit e497173.

Revert "mute test elastic#32737"

d5c67cc

This reverts commit e64bb48.

jkakavas added >test Issues or PRs that are addressing/adding tests :Delivery/Build Build or test infrastructure v7.0.0 v6.4.0 v6.5.0 labels Aug 16, 2018

jkakavas requested review from rjernst and alpar-t August 16, 2018 10:05

jkakavas mentioned this pull request Aug 16, 2018

Mute bwc tests for old versions in FIPS JVMs #32839

Closed

alpar-t approved these changes Aug 16, 2018

View reviewed changes

rjernst reviewed Aug 16, 2018

View reviewed changes

jkakavas added 2 commits August 16, 2018 20:17

Address feedback

8f69f49

Merge remote-tracking branch 'origin/master' into pre6.4-fips-bwc

2d4cfaa

rjernst approved these changes Aug 16, 2018

View reviewed changes

jkakavas merged commit e3aa68b into elastic:master Aug 17, 2018

jkakavas deleted the pre6.4-fips-bwc branch September 14, 2018 06:49

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEST] Run pre 6.4 nodes in non-FIPS JVMs #32901

[TEST] Run pre 6.4 nodes in non-FIPS JVMs #32901

jkakavas commented Aug 16, 2018

rjernst Aug 16, 2018

jkakavas Aug 16, 2018

rjernst Aug 16, 2018

jkakavas Aug 16, 2018

rjernst Aug 16, 2018

jkakavas Aug 16, 2018

rjernst left a comment

jkakavas commented Aug 17, 2018

jkakavas commented Aug 17, 2018

jkakavas commented Aug 17, 2018

rjernst commented Aug 17, 2018

[TEST] Run pre 6.4 nodes in non-FIPS JVMs #32901

[TEST] Run pre 6.4 nodes in non-FIPS JVMs #32901

Conversation

jkakavas commented Aug 16, 2018

rjernst Aug 16, 2018

Choose a reason for hiding this comment

jkakavas Aug 16, 2018

Choose a reason for hiding this comment

rjernst Aug 16, 2018

Choose a reason for hiding this comment

jkakavas Aug 16, 2018

Choose a reason for hiding this comment

rjernst Aug 16, 2018

Choose a reason for hiding this comment

jkakavas Aug 16, 2018

Choose a reason for hiding this comment

rjernst left a comment

Choose a reason for hiding this comment

jkakavas commented Aug 17, 2018

jkakavas commented Aug 17, 2018

jkakavas commented Aug 17, 2018

rjernst commented Aug 17, 2018