HLREST: Bundle the x-pack protocol project #31904

nik9000 · 2018-07-09T16:27:37Z

The :x-pack:protocol project is an implementation detail shared by the
xpack projects and the high level rest client and really doesn't deserve
its own maven coordinants and published javadoc. This change bundles
:x-pack:protocol into the high level rest client.

Relates to #29827

The `:x-pack:protocol` project is an implementation detail shared by the xpack projects and the high level rest client and really doesn't deserve its own maven coordinants and published javadoc. This change bundles `:x-pack:protocol` into the high level rest client. Relates to elastic#29827

elasticmachine · 2018-07-09T16:27:39Z

Pinging @elastic/es-core-infra

nik9000 · 2018-07-09T16:42:21Z

client/rest-high-level/build.gradle

+  configurations.bundled.dependencies.all { Dependency dep ->
+    Project p = dependencyToProject(dep)
+    if (p != null) {
+      evaluationDependsOn(p.path)


I couldn't find a way to make this sufficiently lazy to avoid forcing evaluation of the other project. I believe I could use afterProjectsEvaluated but this feels much more precise.

Shouldn't this be something in BuildPlugin? Seems like we would want this for any cross project dependencies?

We only want this if we want to combine that javadocs from a bunch of different projects into one place. It feels like something we might only want in the high level rest client. Maybe into the plugin API when we make a jar for it. If we make a jar for it.

So after thinking about this some more, maybe! I think we might want this if we detect that the shadow plugin is around.

nik9000 · 2018-07-09T16:42:36Z

client/rest-high-level/build.gradle

+  dependsOn jar
+}
+integTestRunner {
+    classpath -= compileJava.outputs.files


Whoops, indentation doesn't match. I'll fix.

nik9000 · 2018-07-09T16:42:57Z

client/rest-high-level/build.gradle

@@ -34,14 +34,32 @@ publishing {
    }
 }

+// We bundle the x-pack:protocol project into the jar.
+configurations {
+    bundled


Whoops, another spot where indentation doesn't match the rest of the project. I'll fix.

nik9000 · 2018-07-09T17:02:42Z

client/rest-high-level/build.gradle

@@ -27,11 +27,24 @@ group = 'org.elasticsearch.client'
 archivesBaseName = 'elasticsearch-rest-high-level-client'

 publishing {
-    publications {


This didn't line up with the rest of the project and was confusing me so I fixed it.

nik9000 · 2018-07-09T19:13:06Z

@rjernst, could you have a look at this one?

rjernst

I have a couple questions.

rjernst · 2018-07-09T19:40:30Z

client/rest-high-level/build.gradle

+  configurations.bundled.dependencies.all { Dependency dep ->
+    Project p = dependencyToProject(dep)
+    if (p != null) {
+      evaluationDependsOn(p.path)


Shouldn't this be something in BuildPlugin? Seems like we would want this for any cross project dependencies?

rjernst · 2018-07-09T19:44:12Z

client/rest-high-level/build.gradle

+}
+jar {
+  from({configurations.bundled.collect { it.isDirectory() ? it : zipTree(it) }}) {
+    exclude 'META-INF/*'


Won't this break any SPI used by the "bundled" jar?

It seems like this is trying to manually do the work of the shadow plugin?

Yes on both counts. I started with the shadow plugin but it fought me and it feels so "big". I'll see about using it though. It absolutely handles SPI properly and this doesn't.

nik9000 · 2018-07-10T14:58:09Z

I've switched to the shadow plugin and everything work except the pom. Working on that now.

nik9000 · 2018-07-10T14:58:32Z

Eclipse works too.

nik9000 · 2018-07-10T17:00:53Z

@rjernst - I've switch to the shadow plugin here. It isn't a perfect fit but it gets the job done and seems to work fine in Eclipse. Right now I'm only shading the new x-pack:protocol jar. We could shade others if we wanted but I wanted to keep the change smaller.

I think lots of the things in here could be moved to BuildPlugin in a project.plugins.withType(ShadowPlugin).whenPluginAdded { block. If we decide that we want to do it I'd like to save it for a followup because it changes all of the places we use the shadow plugin so will require more manual testing.

nik9000 · 2018-07-10T17:50:02Z

Jarhell failed. That is almost certainly my fault. I'll look in a moment.

Fixes jarhell!

nik9000 · 2018-07-10T18:11:29Z

Jarhell failed. That is almost certainly my fault. I'll look in a moment.

Fixed. I just had to follow directions from the shadow plugin's docs.

rjernst

I double checked this branch with IntelliJ and running the rest HLC tests in IntelliJ appear to continue to work. In the past when trying to shade jars, this is one place we had issues. LGTM.

nik9000 · 2018-07-10T21:58:36Z

Thanks for reviewing @rjernst! I'll merge now, backport to 6.x, and then look into generalizing the changes I made here into BuildPlugin.

The `:x-pack:protocol` project is an implementation detail shared by the xpack projects and the high level rest client and really doesn't deserve its own maven coordinants and published javadoc. This change bundles `:x-pack:protocol` into the high level rest client. Relates to #29827

nik9000 · 2018-07-11T13:09:51Z

Backported!

* master: [TEST] Mute SlackMessageTests.testTemplateRender Docs: Explain closing the high level client [ML] Re-enable memory limit integration tests (#31328) [test] disable packaging tests for suse boxes Add nio transport to security plugin (#31942) XContentTests : Insert random fields at random positions (#30867) Force execution of fetch tasks (#31974) Fix unreachable error condition in AmazonS3Fixture (#32005) Tests: Fix SearchFieldsIT.testDocValueFields (#31995) Add Expected Reciprocal Rank metric (#31891) [ML] Get ForecastRequestStats doc in RestoreModelSnapshotIT (#31973) SQL: Add support for single parameter text manipulating functions (#31874) [ML] Ensure immutability of MlMetadata (#31957) Tests: Mute SearchFieldsIT.testDocValueFields() muted tests due to #31940 Work around reported problem in eclipse (#31960) Move build integration tests out of :buildSrc project (#31961) Tests: Remove use of joda time in some tests (#31922) [Test] Reactive 3rd party tests on CI (#31919) SQL: Support for escape sequences (#31884) SQL: HAVING clause should accept only aggregates (#31872) Docs: fix typo in datehistogram (#31972) Switch url repository rest tests to new style requests (#31944) Switch reindex tests to new style requests (#31941) Docs: Added note about cloud service to installation and getting started [DOCS] Removes alternative docker pull example (#31934) Add Snapshots Status API to High Level Rest Client (#31515) ingest: date_index_name processor template resolution (#31841) Test: fix null failure in watcher test (#31968) Switch test framework to new style requests (#31939) Switch low level rest tests to new style Requests (#31938) Switch high level rest tests to new style requests (#31937) [ML] Mute test failing due to Java 11 date time format parsing bug (#31899) [TEST] Mute SlackMessageTests.testTemplateRender Fix assertIngestDocument wrongfully passing (#31913) Remove unused reference to filePermissionsCache (#31923) rolling upgrade should use a replica to prevent relocations while running a scroll HLREST: Bundle the x-pack protocol project (#31904) Increase logging level for testStressMaybeFlush Added lenient flag for synonym token filter (#31484) [X-Pack] Beats centralized management: security role + licensing (#30520) HLRest: Move xPackInfo() to xPack().info() (#31905) Docs: add security delete role to api call table (#31907) [test] port archive distribution packaging tests (#31314) Watcher: Slack message empty text (#31596) [ML] Mute failing DetectionRulesIT.testCondition() test Fix broken NaN check in MovingFunctions#stdDev() (#31888) Date: Add DateFormatters class that uses java.time (#31856) [ML] Switch native QA tests to a 3 node cluster (#31757) Change trappy float comparison (#31889) Fix building AD URL from domain name (#31849) Add opaque_id to audit logging (#31878) re-enable backcompat tests add support for is_write_index in put-alias body parsing (#31674) Improve release notes script (#31833) [DOCS] Fix broken link in painless example Handle missing values in painless (#30975) Remove the ability to index or query context suggestions without context (#31007) Ingest: Enable Templated Fieldnames in Rename (#31690) [Docs] Fix typo in the Rollup API Quick Reference (#31855) Ingest: Add ignore_missing option to RemoveProc (#31693) Add template config for Beat state to X-Pack Monitoring (#31809) Watcher: Add ssl.trust email account setting (#31684) Remove link to oss-MSI (#31844) Painless: Restructure Definition/Whitelist (#31879) HLREST: Add x-pack-info API (#31870)

* 6.x: Force execution of fetch tasks (#31974) [TEST] Mute SlackMessageTests.testTemplateRender Docs: Explain closing the high level client [test] disable java packaging tests for suse XContentTests : Insert random fields at random positions (#30867) Add Get Snapshots High Level REST API (#31980) Fix unreachable error condition in AmazonS3Fixture (#32005) [6.x][ML] Ensure immutability of MlMetadata (#31994) Add Expected Reciprocal Rank metric (#31891) SQL: Add support for single parameter text manipulating functions (#31874) muted tests due to #31940 Work around reported problem in eclipse (#31960) Move build integration tests out of :buildSrc project (#31961) [Test] Reactive 3rd party tests on CI (#31919) Fix assertIngestDocument wrongfully passing (#31913) (#31951) SQL: Support for escape sequences (#31884) SQL: HAVING clause should accept only aggregates (#31872) Docs: fix typo in datehistogram (#31972) Switch url repository rest tests to new style requests (#31944) Switch reindex tests to new style requests (#31941) Switch test framework to new style requests (#31939) Docs: Added note about cloud service to installation and getting started [DOCS] Removes alternative docker pull example (#31934) ingest: date_index_name processor template resolution (#31841) Test: fix null failure in watcher test (#31968) Watcher: Slack message empty text (#31596) Switch low level rest tests to new style Requests (#31938) Switch high level rest tests to new style requests (#31937) HLREST: Bundle the x-pack protocol project (#31904) [ML] Mute test failing due to Java 11 date time format parsing bug (#31899) Increase logging level for testStressMaybeFlush rolling upgrade should use a replica to prevent relocations while running a scroll [test] port archive distribution packaging tests (#31314) HLRest: Move xPackInfo() to xPack().info() (#31905) Increase logging level for :qa:rolling-upgrade Backport: Add template config for Beat state to X-Pack Monitoring (#31809) (#31893) Fix building AD URL from domain name (#31849) Fix broken NaN check in MovingFunctions#stdDev() (#31888) Change trappy float comparison (#31889) Add opaque_id to audit logging (#31878) add support for is_write_index in put-alias body parsing (#31674) Ingest: Enable Templated Fieldnames in Rename (#31690) (#31896) Ingest: Add ignore_missing option to RemoveProc (#31693) (#31892) [Docs] Fix typo in the Rollup API Quick Reference (#31855) Watcher: Add ssl.trust email account setting (#31684) [PkiRealm] Invalidate cache on role mappings change (#31510) [Security] Check auth scheme case insensitively (#31490) HLREST: Add x-pack-info API (#31870) Remove link to oss-MSI (#31844) Painless: Restructure Definition/Whitelist (#31879)

nik9000 added >enhancement :Delivery/Build Build or test infrastructure v7.0.0 v6.4.0 labels Jul 9, 2018

nik9000 commented Jul 9, 2018

View reviewed changes

Fix indentation

ed5f1ea

nik9000 commented Jul 9, 2018

View reviewed changes

rjernst reviewed Jul 9, 2018

View reviewed changes

Switch to shadow!

3260c3d

nik9000 added 2 commits July 10, 2018 12:54

Publishing

578bf12

no more println

4786607

nik9000 added the blocker label Jul 10, 2018

nik9000 added 2 commits July 10, 2018 13:51

Merge branch 'master' into bundle_x_pack_protocol

1af3dbb

Use the shadow configuration

33fef73

Fixes jarhell!

rjernst approved these changes Jul 10, 2018

View reviewed changes

nik9000 merged commit 4d83a0d into elastic:master Jul 10, 2018

jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HLREST: Bundle the x-pack protocol project #31904

HLREST: Bundle the x-pack protocol project #31904

nik9000 commented Jul 9, 2018

elasticmachine commented Jul 9, 2018

nik9000 Jul 9, 2018

rjernst Jul 9, 2018

nik9000 Jul 9, 2018

nik9000 Jul 10, 2018

nik9000 Jul 9, 2018

nik9000 Jul 9, 2018

nik9000 Jul 9, 2018

nik9000 commented Jul 9, 2018

rjernst left a comment

rjernst Jul 9, 2018

rjernst Jul 9, 2018

rjernst Jul 9, 2018

nik9000 Jul 9, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

rjernst left a comment

nik9000 commented Jul 10, 2018

nik9000 commented Jul 11, 2018

HLREST: Bundle the x-pack protocol project #31904

HLREST: Bundle the x-pack protocol project #31904

Conversation

nik9000 commented Jul 9, 2018

elasticmachine commented Jul 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nik9000 commented Jul 9, 2018

rjernst left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

nik9000 commented Jul 10, 2018

rjernst left a comment

Choose a reason for hiding this comment

nik9000 commented Jul 10, 2018

nik9000 commented Jul 11, 2018