Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch container should not shade org.apache.http.HttpHost #958

Closed
herblover opened this issue Nov 6, 2018 · 5 comments
Closed

Comments

@herblover
Copy link

herblover commented Nov 6, 2018

Hello.

I've tested elasticsearch container, but test code compile failed by shaded org.apache.http.HttpHost.

> Task :compileTestJava
/elasticsearch-poc/src/test/java/ElasticsearchTest.java:15: error: no suitable method found for builder(org.testcontainers.shaded.org.apache.http.HttpHost)
        RestClient client = RestClient.builder(container.getHost()).build();
                                      ^
    method RestClient.builder(Node...) is not applicable
      (varargs mismatch; org.testcontainers.shaded.org.apache.http.HttpHost cannot be converted to Node)
    method RestClient.builder(org.apache.http.HttpHost...) is not applicable
      (varargs mismatch; org.testcontainers.shaded.org.apache.http.HttpHost cannot be converted to org.apache.http.HttpHost)

just for proof of concepts, i created repository showing this issue.

https://github.com/herblover/testcontainer-elasticsearch-poc

you can reproduce uncommenting compile files('libs/elasticsearch-jar.jar') in build.gradle file.

in my test, testcontainers elasticsearch module should not shade org.apache.http.HttpHost related library. (in case of ./gradlew elasticsearch:jar everything works fine. and ./gradlew elasticsearch:shadow generated jar file fails.)

thank you.

@rnorth
Copy link
Member

rnorth commented Nov 6, 2018

Ouch, sorry, and thank you for reporting. We’re on it and will release a bugfix ASAP.
FYI @dadoonet

@dadoonet
Copy link
Contributor

dadoonet commented Nov 6, 2018

OMG! This means that we are missing real integration tests then...

@kiview
Copy link
Member

kiview commented Nov 6, 2018

OMG! This means that we are missing real integration tests then...

That's an interesting point, testing the post shaded jar. Actually the examples Maven project kind of does this, but it not complete and it's not part of the Gradle build.

@dadoonet
Copy link
Contributor

dadoonet commented Nov 6, 2018

Note that I was not able to reproduce because my own project depends on elasticsearch Java Client which brings httpcomponents deps...

@rnorth
Copy link
Member

rnorth commented Nov 6, 2018

I'm pushing a branch which should fix the issue; will create a PR, and will then use the jitpack build to verify.

Unfortunately we used to have some specific integration tests just to look out for shading issues, but these were quite complex and did not survive the transition to Gradle. Looks like we're going to have to reinstate these, though.

@rnorth rnorth mentioned this issue Nov 7, 2018
rnorth added a commit that referenced this issue Nov 7, 2018
FYI @dadoonet, relating to #958 

As httpcore is a transitive dependency that we don't want to conflict with user's libs, we unfortunately need to:

* roll back #959, where we disabled the shading of this module. As is, we're at risk of internal breakage when transitive deps clash.
* in turn, stop exposing `HttpHost` in our public API. We should take a general stance of keeping our public APIs free of transitive deps (especially shaded ones)

The API method in ElasticsearchContainer will thus change, from:
```java
public HttpHost getHost()
```
to:
```java
public String getHttpHostAddress()
```

Usage in turn changes from:
```java
RestClient client = RestClient.builder(container.getHost()).build();
```
to the slightly more verbose but equivalent:
```java
RestClient client = RestClient.builder(HttpHost.create(container.getHttpHostAddress())).build();
```

@bsideup, @kiview and I have discussed at length and concluded that this is the only approach that we're comfortable with - despite the fact that this raises an unfortunate breaking change in the API of the Elasticsearch module, which we reluctantly have to do.

We are sorry that we did not catch this particular issue at PR stage - we're going to upgrade our static analysis to guard against inadvertent leakage of shaded dependencies. In the longer term we also, obviously, wish to reduce the weight of our dependency chain, both unshaded and shaded, so that this does not happen again.

We intend for this change to go out as 1.10.1.

I hope this is OK with you @dadoonet - our apologies again for not catching this before release.
@rnorth rnorth added this to the 1.10.1 milestone Nov 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants