Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testcontainers based integration test #57

Merged
merged 15 commits into from
Apr 30, 2018

Conversation

felixbarny
Copy link
Member

@felixbarny felixbarny commented Apr 12, 2018

TODO: war creation and debugging

closes #20

@felixbarny
Copy link
Member Author

Hey @bsideup,

I have two things left I'm not quite sure how to solve, see the two TODOs in the PR.

The biggest problem is the lifecycle of the war file creation which is necessary for the test to run. Did you already do sth like this? Is this the wrong approach? Do you know tools, which can create a war at runtime?

Also, I didn't quite get debugging to work.

@bsideup
Copy link

bsideup commented Apr 12, 2018

Hi @felixbarny,

Let me take a look at TODOs and the PR in general 👍

.withDockerfileFromBuilder(builder -> builder
.from("tomcat:9")
.run("rm -rf /usr/local/tomcat/webapps/*")
.copy("ROOT.war", "/usr/local/tomcat/webapps/ROOT.war")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend to mount it as a file instead of baking into your image. Otherwise you will create a new image every test run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how would I do that?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check .withFileSystemBind()

.expose(8080, 8000)
.entryPoint("catalina.sh", "jpda", "run")
)
// TODO chicken egg problem here: tests require the war to be present, which is built via mvn package, but mvn package executes the tests
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we were doing the same thing in ZeroTurnaround, we ended up with Groovy-based setup to avoid building test apps at all.
i.e. https://github.com/bsideup/javaagent-boilerplate/tree/master/src/test + https://gist.github.com/bsideup/70091d2f762b8bc7d9255260c0b5ed15 to test Servet-based app.

You can also adjust Maven's lifecycle to execute war packaging before the test.
Also, AFAIK maven-failsafe-plugin will run after WAR packaging

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a groovy based setup sounds good. But I also want to be able to test applications deployed in a traditional way.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, then maven-failsafe-plugin is your friend. The only thing is that you loose IDE friendliness of your tests (you can't run them from your IDE)

Ofc there is another option to use Gradle, and use "Delegate build/test actions to Gradle" in your IDE. This way you can have your test app as a dependency and Gradle will automatically build it for you (if not built) before running the test, and you will be able to run the test from IDE as well, but I understand that changing the build system is not what people usually want to do :D Just FYI-ing you about the possibility and advantages

public static void beforeClass() {
mockServerContainer.getClient().when(request("/v1/transactions")).respond(HttpResponse.response().withStatusCode(200));
mockServerContainer.getClient().when(request("/v1/errors")).respond(HttpResponse.response().withStatusCode(200));
c.followOutput(new Slf4jLogConsumer(logger));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also use fluent .withLogConsumer()


private static final Logger logger = LoggerFactory.getLogger(ServletIntegrationTest.class);

@ClassRule
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JUnit's @ClassRule doesn't work with abstract classes, I would suggest removing the rules and adding:

static {
    Stream.of(c, mockServerContainer).parallel().forEach(GenericContainer::start);
}

This "pattern" is also called "Singleton container" :D

</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI there is also a BOM: org.testcontainers:testcontainers-bom:1.7.0 in case you plan using other containers like PostgreSQL, Selenium, etc

.from("tomcat:9")
.run("rm -rf /usr/local/tomcat/webapps/*")
.copy("ROOT.war", "/usr/local/tomcat/webapps/ROOT.war")
// TODO debugging does not work
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember fighting with JVM debugging inside Docker (not related to Testcontainers), but I don't remember the config anymore :D I just remember that JVM's debugging protocol is not NAT friendly

What I could recommend is to do docker run -p 18000:8000 -e JPDA_ADDRESS=... -e JPDA_TRANSPORT=... tomcat:9 catalina.sh jpda run and play with parameters until you can debug it on port 18000 (while it is 8000 inside the container)

You can also manually add a fixed port mapping when you debug (but I would suggest to protect it with env variable, system property or something else to avoid port conflicts on CI environments)

Third option would be to attempt to start a SocatContainer and gracefully handle the failure because of the port conflict, so that it will not fail on CI envs, I can provide you an example if you want :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An example of the Socat container would be great!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will provide one later today :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it is a problem with the tomcat:9 container. It works with tomcat:8 ¯_(ツ)_/¯

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoa :D

@bsideup
Copy link

bsideup commented Apr 12, 2018

@felixbarny and don't forget to turn on Docker on Travis ;)

sudo: true
services:
  - docker

@codecov-io
Copy link

codecov-io commented Apr 12, 2018

Codecov Report

Merging #57 into master will increase coverage by 0.23%.
The diff coverage is 61.9%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master      #57      +/-   ##
============================================
+ Coverage     75.77%   76.01%   +0.23%     
- Complexity      723      724       +1     
============================================
  Files            78       78              
  Lines          2621     2643      +22     
  Branches        233      240       +7     
============================================
+ Hits           1986     2009      +23     
- Misses          497      501       +4     
+ Partials        138      133       -5
Impacted Files Coverage Δ Complexity Δ
...a/co/elastic/apm/report/ReporterConfiguration.java 100% <100%> (ø) 8 <1> (+1) ⬆️
...a/co/elastic/apm/report/ReportingEventHandler.java 100% <100%> (+8.33%) 10 <7> (+2) ⬆️
.../java/co/elastic/apm/report/ApmServerReporter.java 61.33% <20%> (-7.9%) 11 <0> (ø)
...i/src/main/java/co/elastic/apm/api/ElasticApm.java 43.58% <0%> (-1.15%) 6% <0%> (ø)
...main/java/co/elastic/apm/opentracing/ApmScope.java 100% <0%> (ø) 6% <0%> (+1%) ⬆️
...ava/co/elastic/apm/opentracing/ApmSpanBuilder.java 79.36% <0%> (ø) 20% <0%> (ø) ⬇️
...va/co/elastic/apm/opentracing/ApmScopeManager.java 100% <0%> (ø) 8% <0%> (+1%) ⬆️
...a/co/elastic/apm/impl/transaction/Transaction.java 73.46% <0%> (+0.94%) 28% <0%> (+1%) ⬆️
...java/co/elastic/apm/jdbc/ApmJdbcEventListener.java 62.22% <0%> (+1.8%) 10% <0%> (-2%) ⬇️
...ain/java/co/elastic/apm/impl/ElasticApmTracer.java 81.42% <0%> (+1.85%) 40% <0%> (-1%) ⬇️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65f4313...6fc7d9f. Read the comment docs.

@felixbarny
Copy link
Member Author

I really don't get why tests are failing on Jenkins.

Instead of returning the Hello World jsp, Tomcat responds with a 404, even though the war seems to be deployed successfully.

11:17:00 [tomcat] 13-Apr-2018 09:17:00.322 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
11:17:00 [tomcat] 13-Apr-2018 09:17:00.340 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
11:17:00 [tomcat] 13-Apr-2018 09:17:00.350 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
11:17:00 [tomcat] 13-Apr-2018 09:17:00.352 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
11:17:00 [tomcat] 13-Apr-2018 09:17:00.354 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 717 ms
11:17:00 [tomcat] 13-Apr-2018 09:17:00.397 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
11:17:00 [tomcat] 13-Apr-2018 09:17:00.397 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.30
11:17:00 [tomcat] 13-Apr-2018 09:17:00.414 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/ROOT.war]
11:17:00 09:17:00.828 [ForkJoinPool.commonPool-worker-1] INFO  docker[testcontainers/p7ack8jmx4ukkkwq] - Container testcontainers/p7ack8jmx4ukkkwq started

The relative referencing of ../simple-webapp/taget/ROOT.war when binding the war into the container does seem to work, as </var/lib/jenkins/workspace/elastic+apm-agent-java+pull-request+multijob-test/src/github.com/elastic/apm-agent-java/apm-agent-plugins/apm-servlet-plugin-it/simple-webapp-integration-test/../simple-webapp/target/ROOT.war does exist and the content is correct. But the symptoms are the same as deploying an empty directory.

The connection to the tomcat container doesn't seem to be the problem, otherwise tomcat wouldn't be able to respond with a 404.

The on init log message APM test application {} started in ApmTestHelperServlet does not appear in the logs, so there has to be a problem with the deployment. Investigating further...

@bsideup
Copy link

bsideup commented Apr 13, 2018

@felixbarny or, actually, I see ✔ File should be mountable in the build's log, it means that Testcontainers checked the file mounting logic and it works, hence I recommend checking the test's code, could be unrelated to Testcontainers

@felixbarny
Copy link
Member Author

I suspect something goes wrong when mounting the file.

I even downloaded the war from the workspace and executed locally with success. And assert on test startup that the war file exists and that it has content: https://github.com/elastic/apm-agent-java/pull/57/files#diff-86c386663b0d434b566084161d7f2a22R94.

are you running slave as a container?

I'm actually not sure if the jenkins slaves are bare metal, VMs or dockerized. I'll ask.

@bsideup
Copy link

bsideup commented Apr 13, 2018

@felixbarny try adding a .waitingFor(Wait.forHttp("/someHTTP200endpointOfYourTestApp")) to your container, maybe just it takes longer to start an app on CI than locally?

@felixbarny
Copy link
Member Author

Makes sense! Tomcat listening on a socket does not mean that the application has already been deployed!

Will have to wait until there are free build slaves..

@felixbarny
Copy link
Member Author

Nope

2:25:07 10:25:07.063 [ForkJoinPool.commonPool-worker-1] ERROR docker[testcontainers/nu5blc23zgbzjttz] - Could not start container
12:25:07 org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:32771/apm/status should return HTTP 200)

The deployment of the war is also suspiciously fast:

12:24:08 [tomcat] 13-Apr-2018 10:24:08.520 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/ROOT.war] has finished in [478] ms

Locally, it takes around 1.5 seconds.

@felixbarny
Copy link
Member Author

Note that is also says

Deployment of web application directory [/usr/local/tomcat/webapps/ROOT.war] has finished in [478] ms

When executing locally, it says Deployment of web application archive. So there must be something wrong either with the path or with the mounting.

@felixbarny
Copy link
Member Author

Also copy-ing the file when building the container and withClasspathResourceMapping for a file which is checked into git does not work :(

The Jenkins slaves are either EC2 instances or bare metal hosts and are not dockerized.

@bsideup
Copy link

bsideup commented Apr 13, 2018

@felixbarny out of curiosity - could you please try to deploy your app under a context (eg "app.war")?

@felixbarny
Copy link
Member Author

felixbarny commented Apr 13, 2018

Still no luck. The /usr/local/tomcat/webapps/app.war directory inside the docker container appears to be an empty folder:

total 8.0K
drwxr-xr-x 2 root root 4.0K Apr 13 12:05 .
drwxr-x--- 8 root root 4.0K Apr 13 12:11 ..

@bsideup
Copy link

bsideup commented Apr 13, 2018

@felixbarny there is a..... hm... questionable behaviour of Docker, when you mount non-existing file it mounts it as an empty folder.

One clue - does your CI environment runs as Docker Swarm?

@felixbarny
Copy link
Member Author

Do you mean if the workers have docker swarm installed or are managed by docker swarm? As they are not docker containers I don't think they are managed by docker swarm.

@bsideup
Copy link

bsideup commented Apr 13, 2018

@felixbarny more like if Swarm is initialized on the machines with agents...

Building remotely on slave-09c45e463891e2e0d (virtual x86_64 swarm ubuntu linux ubuntu-16.04) in workspace 

This line (and swarm in it) attracted my attention. Swarm is a distributed scheduler of Docker containers and might not guarantee that the container will be scheduled on the same machine, this is why the mapping might fail.
Imaging running tests on machine A, but the container is started on machine B where /home/user/some/path/app.war does not exist

@felixbarny
Copy link
Member Author

Good point. But then it's strange that copy-ing the war into the image does not work (2063085).

@elasticdog could you comment on wether docker images executed in the CI environment may be scheduled to other machines via docker swarm? Is there anything special in our deployment which could cause mounting a file to fail?

@elasticdog
Copy link

@felixbarny @bsideup the swarm in your comment above is not a reference to Docker swarm, but to a label on Jenkins worker nodes that is automatically applied by the jenkins-swarm plugin, which we use to dynamically attach worker nodes to masters in our Jenkins clusters.

I'm not personally a Docker expert, but I don't know of anything off-hand that would be unique to our infrastructure that would cause this to fail. Can you point us to the docker command that you're running to mount the file so other members on the infra team can chime in?

@felixbarny
Copy link
Member Author

Seems like testcontainers does not create a Dockerfile but uses the docker socket api. This is what is sent to the docker.sock, inclunding the bind:

{
  "Env": [
    "ELASTIC_APM_SERVER_URL=http://apm-server:1080",
    "ELASTIC_APM_IGNORE_URLS=/apm/*",
    "ELASTIC_APM_SERVICE_NAME=servlet-test-app"
  ],
  "Cmd": [],
  "Image": "testcontainers/jlsaqt1bpwtwpp31",
  "Volumes": {},
  "ExposedPorts": {
    "8080/tcp": {},
    "8000/tcp": {}
  },
  "HostConfig": {
    "Binds": [
      "/Users/felixbarnsteiner/projects/github/elastic/apm-agent-java/apm-agent-plugins/apm-servlet-plugin-it/simple-webapp-integration-test/../simple-webapp/target/ROOT.war:/usr/local/tomcat/webapps/ROOT.war:rw"
    ],
    "ExtraHosts": [],
    "Links": [],
    "NetworkMode": "bdd79fbf5e4f91806e7c8eff8a19ac1396dba10a8e9e1c8b5e4d96a98f6b4b14",
    "PortBindings": {},
    "PublishAllPorts": true,
    "VolumesFrom": []
  },
  "Labels": {
    "org.testcontainers": "true",
    "org.testcontainers.sessionId": "36cb2fb8-69e8-4e2e-b578-dda7c237ed5b"
  },
  "NetworkingConfig": {
    "EndpointsConfig": {
      "bdd79fbf5e4f91806e7c8eff8a19ac1396dba10a8e9e1c8b5e4d96a98f6b4b14": {
        "IPAMConfig": null,
        "Links": null,
        "Aliases": [
          "tc-LcXmZm2Q"
        ],
        "NetworkID": null,
        "EndpointID": null,
        "Gateway": null,
        "IPAddress": null,
        "IPPrefixLen": null,
        "IPv6Gateway": null,
        "GlobalIPv6Address": null,
        "GlobalIPv6PrefixLen": null,
        "MacAddress": null
      }
    }
  }
}            

@bsideup
Copy link

bsideup commented Apr 16, 2018

@elasticdog @felixbarny one can easily verify the environment in isolation by running something like:

docker run \
    -p 8080:8080 \
    -it --rm \
    -v /var/lib/jenkins/workspace/elastic+apm-agent-java+pull-request+multijob-test/src/github.com/elastic/apm-agent-java/apm-agent-plugins/apm-servlet-plugin-it/simple-webapp-integration-test/../simple-webapp/target/ROOT.war:/usr/local/tomcat/webapps/ROOT.war \
    tomcat:8

@felixbarny felixbarny self-assigned this Apr 18, 2018
@felixbarny felixbarny added this to the Beta milestone Apr 18, 2018
@felixbarny
Copy link
Member Author

@elasticdog can you or someone from the infra team help to investigate on this?

Steps:

  • execute the build for this PR (which fails)
  • execute the docker run command in the comment above
  • try to access localhost:8080
  • expected: returns 200 and a Hello World! response
  • actual: when executing the build, a 404 is returned

Problem:
The ROOT.war is not properly mounted into the container. When examining contents in the container, the ROOT.war is not the expected file but an empty folder.


private static final Logger logger = LoggerFactory.getLogger(ServletIntegrationTest.class);

private static final String pathToWar = "../simple-webapp/target/ROOT.war";

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to give the full path to docker for mounting a directory.

Example:

❯ docker run --rm -v ../shared:/app -w /app -ti alpine
docker: Error response from daemon: create ../shared: "../shared" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming withFileSystemBind doesn't automatically expand the path for you.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But let me check if we normalize it or not (we should)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realised this works on travis ci, so obviously it must be somehow :)

@Crazybus
Copy link

Not sure if my comment about the full path or not is related. However the path in the comment above doesn't seem to exist on the worker node.

docker run
-p 8080:8080
-it --rm
-v /var/lib/jenkins/workspace/elastic+apm-agent-java+pull-request+multijob-test/src/github.com/elastic/apm-agent-java/apm-agent-plugins/apm-servlet-plugin-it/simple-webapp-integration-test/../simple-webapp/target/ROOT.war:/usr/local/tomcat/webapps/ROOT.war
tomcat:8

The closest thing I can find is an empty directory at /var/lib/jenkins/workspace/elastic+apm-agent-java+pull-request+multijob-test/src/github.com/elastic/apm-agent-java/apm-agent-plugins/apm-servlet-plugin-it/simple-webapp/target/ROOT.war/

Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
to see if the application has started properly

Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
@felixbarny
Copy link
Member Author

One of those problems which resolve themselves over time... But that's actually very good news, even if I have to wait for a new docker-java release.

Funny thing is that the new docker-java version does not log the docker.sock requests anymore, not even on TRACE.

@bsideup
Copy link

bsideup commented Apr 25, 2018

Even with the fact that the problem is in docker-java and not in Testcontainers I still feel sorry about having you to debug the problem and me almost unable to help 😅

Actually, in Testcontainers 1.7.1 we replaced docker-java's NettyCmdFactory with our own which uses daemon threads instead (long awaited feature request from Spring guys), and removed the packages logger. While it might be helpful in some rare cases like yours, a lot of users (very unfortunately) doesn't know how to configure the logging and were complaining about HEX dumps in their tests, this is why we decided to drop the logger.
But now I'm thinking about adding a flag to turn it on (in Testcontainers' properties) for advanced users who want to debug the packets...

@bsideup
Copy link

bsideup commented Apr 25, 2018

@felixbarny FYI it seems that you can easily workaround the docker-java's bug by doing something like the following:

File tempWarFile = new File(System.getProperty("user.home"), UUID.randomUUID() + ".war");
tempWarFile.deleteOnExit();
Files.copy(warFile, tempWarFile.toPath());
withFileSystemBind(tempWarFile.getAbsolutePath(), "/app.war")

Assuming that user's home directory will not have + in it :)

@bsideup
Copy link

bsideup commented Apr 25, 2018

@felixbarny another option would be to start Tomcat and do:

tomcat.copyFileToContainer(MountableFile.forHostPath(warFilePath), "/usr/local/tomcat/webapps/ROOT.war")

The reason why it works because it will transfer the file as InputStream to docker daemon instead of telling the daemon to use one of the files on the daemon's file system :)

Useful for integration tests to ensure
the transactions have been reported

Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
This works around a bug in docker-java which encodes the plus sign to
a space when binding a file to the container.

Also restructures the integration tests into a top-level
integration-tests module as it felt a bit awkward as a apm-agent-plugins
submodule. It's not a plugin and we want to avoid packaging the tests
applications like simple-webapp into the main distribution.

Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
@bsideup
Copy link

bsideup commented Apr 26, 2018

@felixbarny snap! Apparently, there is also a bug in Testcontainers in MountableFileTest :(
I was able to reproduce it and going to fix it ASAP

@felixbarny
Copy link
Member Author

I'm thinking about deploying the wars via the tomcat manager API (https://tomcat.apache.org/tomcat-9.0-doc/manager-howto.html#Deploy_A_New_Application_Archive_(WAR)_Remotely). That way I can reuse the container across different tests. But that's something for the future... @bsideup what do you think about that approach?

- Deploy war withFileSystemBind instead of copyFileToContainer
  as that is much faster (1s vs 8s)

Signed-off-by: Felix Barnsteiner <felix.barnsteiner@elastic.co>
@bsideup
Copy link

bsideup commented Apr 30, 2018

@felixbarny we had some major issues with such approach:

  1. Tomcat's classloading leaks
  2. java agent reporting traces of another app under the test
  3. less isolated parallel tests

Also, Tomcat + deployment adds to the test execution time.
I would strongly suggest you to use Jetty + jetty-servlet like here:
https://gist.github.com/bsideup/70091d2f762b8bc7d9255260c0b5ed15
You can build it as Java app with Maven as well, but it works much faster (Tomcat-on-classpath can also be used)

@felixbarny
Copy link
Member Author

Thx for the suggestions. Whenever feasible, I'll test with a simple Groovy Jetty script. But part of what I want to verify with the integration tests is that the agent works properly with a specific application server in a specific version.

YEY, the tests are green :D

@felixbarny felixbarny changed the title WIP: Add testcontainers based integration test Add testcontainers based integration test Apr 30, 2018
@felixbarny felixbarny merged commit 0a3bafa into elastic:master Apr 30, 2018
@felixbarny felixbarny deleted the integration-test branch April 30, 2018 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add integration tests
6 participants