test failures in version 1.1.0 #170

dbyron0 · 2016-07-06T17:27:46Z

At commit 248befb, I'm seeing test failures on my local box. There's enough output that I hesitate to paste it all. I'm happy to add more to help figure out what's going on.

$ mvn clean install
<snipped...a lot>
Results :

Failed tests:   pullingNonExistentImageFailsGracefully(org.testcontainers.junit.NonExistentImagePullTest): Pulling a nonexistent container will cause an exception to be thrown: IllegalStateException was thrown instead of ContainerFetchException

Tests in error: 
  secondTest(org.testcontainers.junit.DockerComposeV2FormatTest): Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@2f48b3d2
  simpleDockerfileWorks(org.testcontainers.junit.DockerfileTest): failed to create a child event loop
  customizableImage(org.testcontainers.junit.DockerfileTest): Can't get Docker image name from org.testcontainers.junit.DockerfileTest$1@55787112
  dockerfileBuilderWorks(org.testcontainers.junit.DockerfileTest): failed to create a child event loop
  org.testcontainers.junit.DockerNetworkModeTest
  testFixedHostPortMapping(org.testcontainers.junit.FixedHostPortContainerTest): failed to create a child event loop
  org.testcontainers.junit.GenericContainerRuleTest
  testFetchStdoutWithNoLimit(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testFetchStdout(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testFetchStdoutWithTimeout(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testLogConsumer(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testToStringConsumer(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  simpleTest[alpine:3.2](org.testcontainers.junit.ParameterizedDockerfileContainerTest): failed to create a child event loop
  simpleTest[alpine:3.3](org.testcontainers.junit.ParameterizedDockerfileContainerTest): failed to create a child event loop
  testWaitUntilReady_Success(org.testcontainers.junit.wait.HostPortWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HostPortWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Success(org.testcontainers.junit.wait.HttpWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HttpWaitStrategyTest): failed to create a child event loop

Tests run: 52, Failures: 1, Errors: 18, Skipped: 3

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] TestContainers Parent POM .......................... SUCCESS [  0.850 s]
[INFO] TestContainers Core ................................ FAILURE [02:30 min]
[INFO] TestContainers :: JDBC ............................. SKIPPED
[INFO] TestContainers :: JDBC :: MySQL .................... SKIPPED
[INFO] TestContainers :: JDBC :: PostgreSQL ............... SKIPPED
[INFO] TestContainers :: Selenium ......................... SKIPPED
[INFO] TestContainers :: nginx ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:31 min
[INFO] Finished at: 2016-07-06T10:06:27-07:00
[INFO] Final Memory: 38M/524M
[INFO] ------------------------------------------------------------------------

with a stack trace that shows up a bunch:

Exception in thread "Thread-2" java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:96)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:89)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.images.builder.ImageFromDockerfile.lambda$static$22(ImageFromDockerfile.java:42)
    at java.lang.Thread.run(Thread.java:745)
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 13 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 17 more

I'm running OS X 10.10.5, using docker toolbox 1.11.2.

$ docker info
Containers: 1
Images: 449
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 613
 Dirperm1 Supported: true
Logging Driver: json-file
Kernel Version: 4.4.8-boot2docker
Operating System: Boot2Docker 1.11.1-rc1 (TCL 7.0); HEAD : 903a352 - Tue Apr 26 14:18:06 UTC 2016
CPUs: 1
Total Memory: 995.9 MiB
Name: docker-vm
ID: QAXI:YDZA:NQO6:CJ6F:222Q:FZO5:MJIB:BKUR:OCQP:WCJJ:QOUT:JYPN
Debug mode (server): true
File Descriptors: 13
Goroutines: 30
System Time: 2016-07-06T17:25:13.145332684Z
EventsListeners: 0
Init SHA1: 
Init Path: 
Docker Root Dir: /mnt/sda1/var/lib/docker
Username: ********
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

The text was updated successfully, but these errors were encountered:

ihabsoliman · 2016-07-06T19:44:19Z

Check here #163

ihabsoliman · 2016-07-06T19:54:46Z

#163 (comment)

Am runing antregos linux
Docker Info

Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 1.11.2
Storage Driver: devicemapper
 Pool Name: docker-8:2-267023-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 408.5 MB
 Data Space Total: 107.4 GB
 Data Space Available: 19.11 GB
 Metadata Space Used: 1.278 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.146 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.127 (2016-06-17)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: null host bridge
Kernel Version: 4.6.3-1-ARCH
Operating System: Antergos Linux
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 2.102 GiB
Name: isoliman
ID: VTZ4:AEDP:6BTP:KWL2:IURW:AX53:6CFY:OIZR:XPTP:JL7I:LK4A:2CUF
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/

rnorth · 2016-07-06T20:07:09Z

This is odd, and not something I've seen previously. Sorry you're experiencing this!

Obviously based on the stack trace it looks as though netty (which is quite new to docker-java and therefore testcontainers) is being constrained by a lack of available file descriptors. Quite why that is, I'm not sure yet... Some possibilities that we could try and eliminate:

Perhaps Testcontainers is mistakenly opening a vast number of docker clients
Perhaps something is going wrong in docker-java or netty
Perhaps your system file descriptor limits are low or something else is consuming a large number. Or conversely the systems I've tested on have abnormally high limits (but on my dev Mac it looks like I'm on the OS X default values)
Something else...?

Given that two of you are experiencing this on different environments it's looking like there's clearly something wrong in the libs, though - and I'm confused why this hasn't arisen before!

Just to start eliminating (3), please could you check the output of (on OS X):

$ sysctl kern.maxfiles       # I get 12288
$ sysctl kern.maxfilesperproc     # I get 10240

or on Linux:

$ sysctl fs.file-max
$ ulimit -Hn

rnorth · 2016-07-06T20:11:00Z

Just one more thing to check (though I really hope this isn't relevant):

$ java -version

I get:

java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)

dbyron0 · 2016-07-06T20:13:32Z

For me (OS X):

$ sysctl kern.maxfiles
kern.maxfiles: 12288
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 10240
$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

ihabsoliman · 2016-07-06T20:30:55Z

Docker info Mac

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 2
Server Version: 1.12.0-rc2
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 20
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: default
Default Runtime: default
Security Options: seccomp
Kernel Version: 4.4.14-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.954 GiB
Name: moby
ID: T77K:UCCE:QLZW:AUQO:HYKO:QMRC:OFAS:PJFR:Z7CG:VD5I:37QJ:SKYG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 21
 Goroutines: 28
 System Time: 2016-07-06T20:12:32.598556277Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Username: felipemendez
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8

Also

For Mac

$ sysctl kern.maxfiles
kern.maxfiles: 12288

$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 10240

For linux

fs.file-max = 218771
ulimit = 4096

rnorth · 2016-07-06T20:44:09Z

Thanks, I think we can probably disregard (3) for now. I'm wondering if (1) is correct and we're just seeing the resource limit being hit on your machines earlier than elsewhere. By the looks of it we could be closing down docker client instances after test method/class execution but don't, which could well be the cause of a resource leak.

I'll patch with a simple close() for now - will push to a branch in a bit...

rnorth · 2016-07-06T20:51:29Z

If you wouldn't mind, could you try this branch some time and see if it makes a difference?

dbyron0 · 2016-07-06T20:59:06Z

Almost the same failures here on that branch:

Results :

Failed tests:   pullingNonExistentImageFailsGracefully(org.testcontainers.junit.NonExistentImagePullTest): Pulling a nonexistent container will cause an exception to be thrown: IllegalStateException was thrown instead of ContainerFetchException

Tests in error: 
  secondTest(org.testcontainers.junit.DockerComposeV2FormatTest): failed to create a child event loop
  simpleDockerfileWorks(org.testcontainers.junit.DockerfileTest): failed to create a child event loop
  customizableImage(org.testcontainers.junit.DockerfileTest): Can't get Docker image name from org.testcontainers.junit.DockerfileTest$1@3f28bd56
  dockerfileBuilderWorks(org.testcontainers.junit.DockerfileTest): failed to create a child event loop
  org.testcontainers.junit.DockerNetworkModeTest
  testFixedHostPortMapping(org.testcontainers.junit.FixedHostPortContainerTest): failed to create a child event loop
  org.testcontainers.junit.GenericContainerRuleTest
  testFetchStdoutWithNoLimit(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testFetchStdout(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testFetchStdoutWithTimeout(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testLogConsumer(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  testToStringConsumer(org.testcontainers.junit.OutputStreamTest): failed to create a child event loop
  simpleTest[alpine:3.2](org.testcontainers.junit.ParameterizedDockerfileContainerTest): failed to create a child event loop
  simpleTest[alpine:3.3](org.testcontainers.junit.ParameterizedDockerfileContainerTest): failed to create a child event loop
  testWaitUntilReady_Success(org.testcontainers.junit.wait.HostPortWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HostPortWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Success(org.testcontainers.junit.wait.HttpWaitStrategyTest): failed to create a child event loop
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HttpWaitStrategyTest): failed to create a child event loop

Tests run: 52, Failures: 1, Errors: 18, Skipped: 3

and the same stack trace:

Exception in thread "Thread-2" java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:96)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:89)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.images.builder.ImageFromDockerfile.lambda$static$22(ImageFromDockerfile.java:42)
    at java.lang.Thread.run(Thread.java:745)
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 13 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 17 more

rnorth · 2016-07-06T21:03:54Z

😢

Then I think the problem may not be a leak, but a surge in the number of clients being used at startup. Right now each container has its own Docker Client instance; this used to be required because the Jersey implementation of docker-java was not thread-safe. Now that we're using the netty docker-java interface, apparently it is thread safe so we could go back to a single shared client. That would be less resource heavy.

I'll have a go at this tomorrow - sorry for keeping you waiting...!

dbyron0 · 2016-07-06T21:06:53Z

On the contrary, thanks for the quick response.

ihabsoliman · 2016-07-07T01:09:16Z

Your doing a great

On the contrary, thanks for the quick response.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#170 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AB9VChEO9QeMBXHCs_mmx0pswuV0Dt4iks5qTBjugaJpZM4JGUJI
.

You received this message because you are subscribed to the Google Groups
"Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to dev+unsubscribe@ihabsoliman.net.
To post to this group, send email to dev@ihabsoliman.net.
To view this discussion on the web visit
https://groups.google.com/a/ihabsoliman.net/d/msgid/dev/testcontainers/testcontainers-java/issues/170/230906859%40git.luolix.top
https://groups.google.com/a/ihabsoliman.net/d/msgid/dev/testcontainers/testcontainers-java/issues/170/230906859%40git.luolix.top?utm_medium=email&utm_source=footer
.

rnorth · 2016-07-08T06:00:54Z

Just to follow up - I've hit a problem adapting Testcontainers to use a single instance of docker client (which would solve the resource leak otherwise): docker-java/docker-java#632

valdisrigdon · 2016-07-08T15:42:08Z

I've hit this as well. It's ProxiedUnixSocketClientProviderStrategy that is opening the large number of files and not closing them.

valdisrigdon · 2016-07-08T16:03:25Z

I've hacked DockerClientFactory to only use DockerMachienClientProviderStrategy and I'm getting too many files open as well, even with docker-machine up and running.

Note that I'm not even getting to the point of runnings tests. This is just the initial instantiation of DockerClientFactory.

rnorth · 2016-07-08T16:36:28Z

Yeah - with lsof I was able to see a large number of kqueue file descriptors building up, all of which are ultimately coming from netty. On my machines and CI it's not hitting any limits, but I could see a gradual build-up that makes me uncomfortable.

Basically the problem/solution is a bit nested:

Testcontainers is still creating a new DockerClient instance per container, which used to be the right thing when using versions of docker-java based on Jersey, as these did not have guarantees about thread-safety
Having switched to docker-java's netty support (for closer tracking of recent Docker API changes), the DockerClient-per-container model is seemingly creating many more open file descriptors in total. For at least three of you, this is causing an ugly explosion 😭
I think the right solution is to follow advice given in Thread-safety and pooling docker-java/docker-java#536 re improved thread safety for the netty client, and go back to a single shared DockerClient instance or thread-local DockerClients
However, in testing both those changes, I've now hit Netty 'IllegalStateException: executor not accepting a task' docker-java/docker-java#632 - which in turn looks like Netty 4.1.0.Final : IllegalStateException: executor not accepting a task netty/netty#5313. This is supposedly fixed in netty v4.1.1, but bumping the version has not actually helped.

There's something very odd going on so I'm going to try and narrow down docker-java/docker-java#632 to something that helps identify exactly what the root cause is. I'm partly hopeful that I'll find I've done something silly in the instantiation of DockerClient..!

Hopefully we'll get there soon, but it's a bit more difficult than I'd have liked. Sorry to anyone afflicted by this.

If anybody has any thoughts/solutions/data points to add, please feel free!

valdisrigdon · 2016-07-08T16:53:16Z

I'm seeing an initial explosion of ~8000 file descriptors on just the initial creation of DockerClientFactory, even when limited to just using the DockerMachineClientProviderStrategy.

rnorth · 2016-07-08T17:05:10Z

@valdisrigdon would you mind sending the list of file descriptors (either email to rich.north@gmail.com or a gist would be fine)? This might yield more clues...

valdisrigdon · 2016-07-08T17:14:41Z

This should help.

ava    33486 valdis.rigdon  195u    IPv6 0x783d346a9536ce9b       0t0       TCP 192.168.99.1:54624->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  196u    IPv6 0x783d346a9536be7b       0t0       TCP 192.168.99.1:54625->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  197u    IPv6 0x783d346a9536d3fb       0t0       TCP 192.168.99.1:54626->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  198u    IPv6 0x783d346a9536d95b       0t0       TCP 192.168.99.1:54627->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  199u    IPv6 0x783d346a9536debb       0t0       TCP 192.168.99.1:54628->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  200u    IPv6 0x783d346a9536b91b       0t0       TCP 192.168.99.1:54629->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  201u    IPv6 0x783d346a9536b3bb       0t0       TCP 192.168.99.1:54630->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  202u    IPv6 0x783d346a9536e41b       0t0       TCP 192.168.99.1:54631->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  203u    IPv6 0x783d346a9536e97b       0t0       TCP 192.168.99.1:54632->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  204u    IPv6 0x783d346a9536eedb       0t0       TCP 192.168.99.1:54633->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  205u    IPv6 0x783d346a9536ae5b       0t0       TCP 192.168.99.1:54634->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  206u    IPv6 0x783d346a9536a8fb       0t0       TCP 192.168.99.1:54635->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  207u    IPv6 0x783d346a96d283db       0t0       TCP 192.168.99.1:54636->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  208u    IPv6 0x783d346a96d2893b       0t0       TCP 192.168.99.1:54637->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  209u    IPv6 0x783d346a96d28e9b       0t0       TCP 192.168.99.1:54638->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  210u    IPv6 0x783d346a96d293fb       0t0       TCP 192.168.99.1:54639->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  211u    IPv6 0x783d346a96d2995b       0t0       TCP 192.168.99.1:54640->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  212u    IPv6 0x783d346a96d27e7b       0t0       TCP 192.168.99.1:54641->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  213u    IPv6 0x783d346a96d2791b       0t0       TCP 192.168.99.1:54642->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  214u    IPv6 0x783d346a96d29ebb       0t0       TCP 192.168.99.1:54643->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  215u    IPv6 0x783d346a96d2a41b       0t0       TCP 192.168.99.1:54644->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon  216u    IPv6 0x783d346a96d273bb       0t0       TCP 192.168.99.1:54645->192.168.99.100:2376 (ESTABLISHED)

(snip)

java    33486 valdis.rigdon *229u    IPv6 0x783d346aa4d493bb       0t0       TCP 192.168.99.1:64661->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *230u    IPv6 0x783d346aa4d48e5b       0t0       TCP 192.168.99.1:64662->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *231u    IPv6 0x783d346aa4d488fb       0t0       TCP 192.168.99.1:64663->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *232u    IPv6 0x783d346aa4d58e9b       0t0       TCP 192.168.99.1:64664->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *233u    IPv6 0x783d346aa4d5893b       0t0       TCP 192.168.99.1:64665->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *234u    IPv6 0x783d346aa4d583db       0t0       TCP 192.168.99.1:64666->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *235u    IPv6 0x783d346aa4d593fb       0t0       TCP 192.168.99.1:64667->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *236u    IPv6 0x783d346aa4d5995b       0t0       TCP 192.168.99.1:64668->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *237u    IPv6 0x783d346aa4d59ebb       0t0       TCP 192.168.99.1:64669->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *238u    IPv6 0x783d346aa4d5a41b       0t0       TCP 192.168.99.1:64670->192.168.99.100:2376 (ESTABLISHED)
java    33486 valdis.rigdon *239u    IPv6 0x783d346aa4d57e7b       0t0       TCP 192.168.99.1:64671->192.168.99.100:2376 (ESTABLISHED)

rnorth · 2016-07-08T17:39:11Z

Yikes - that definitely doesn't look healthy. I was looking at slow growth of KQUEUE file descriptors but this is rather different.

This is what I'm getting right now (mid test on testcontainers core module):

$ lsof | grep java | grep TCP                                                                                                                         
java      40581 rnorth   36u    IPv6 0x41915f775e638211         0t0      TCP *:12918 (LISTEN)
java      40581 rnorth   70u    IPv6 0x41915f775e6f1c91         0t0      TCP *:52304 (LISTEN)
java      40581 rnorth   71u    IPv6 0x41915f7762b33cb1         0t0      TCP localhost:52088->localhost:52306 (ESTABLISHED)
java      43193 rnorth  356u    IPv6 0x41915f775e8b6731         0t0      TCP 192.168.99.1:52126->192.168.99.100:2376 (ESTABLISHED)

That's all - just 1 TCP connection to the docker daemon! This is running docker-machine (0.8.0-rc1) on OS X.

Working out how to reproduce this is going to be half the challenge, I suspect...

valdisrigdon · 2016-07-08T17:41:13Z

@rnorth I can (obviously) easily reproduce this. I can test out a branch, suggested changes, or run with some extra debug options if you need.

rnorth · 2016-07-08T18:14:04Z

Thanks @valdisrigdon - I'm afraid I'll probably have to take you up on that.
Later or this weekend I'll try and give you a smaller test project to test
a few things in isolation. Hopefully a divide and conquer approach will
help us narrow down on the cause more efficiently.

rnorth · 2016-07-09T19:35:51Z

I've put together a small test project to do some diagnostics in/around the area that seems to be problematic: https://github.com/rnorth/testcontainers-fd-debug

The idea is to narrow down or eliminate exactly where the leak is occurring.

Would you mind checking it out, having a look over it, and running, then send the output to me?

valdisrigdon · 2016-07-09T20:46:17Z

@rnorth Working on running it. Running it through the Maven project worked fine -- I didn't see the huge number of open sockets. When adapting the test to run through the IDE or a Gradle project, I'm still seeing the errors. And of course since I've run out of file descriptors, it's can't run lsof.

valdisrigdon · 2016-07-09T21:44:49Z

After converting testcontainers-fd-debug to a Gradle project, it isn't blowing up file descriptors, so it's not that. The project that I'm using testcontainers on does use Netty. Perhaps having that on the classpath is affecting it somehow?

rnorth · 2016-07-10T20:03:44Z

@valdisrigdon thanks. A conflict with another version of Netty on the classpath sounds like a plausible cause for odd behaviour, so worth looking at. However, I think @dbyron0 and @ihabsoliman were encountering this just when building testcontainers from source. Confirmation of this would be useful, though!

ihabsoliman · 2016-07-10T20:31:45Z

I am been using just including it from maven but the project am using it in
has Nett, Jersey, and Jetty in the classpath.
On Jul 10, 2016 2:03 PM, "Richard North" notifications@github.com wrote:

@valdisrigdon https://github.com/valdisrigdon thanks. A conflict with
another version of Netty on the classpath sounds like a plausible cause for
odd behaviour, so worth looking at. However, I think @dbyron0
https://github.com/dbyron0 and @ihabsoliman
https://github.com/IhabSoliman were encountering this just when
building testcontainers from source. Confirmation of this would be useful,
though!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#170 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AB9VCnGzryMb2eU8KmPQqTE4BH4IZn0qks5qUVAhgaJpZM4JGUJI
.

You received this message because you are subscribed to the Google Groups
"Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to dev+unsubscribe@ihabsoliman.net.
To post to this group, send email to dev@ihabsoliman.net.
To view this discussion on the web visit
https://groups.google.com/a/ihabsoliman.net/d/msgid/dev/testcontainers/testcontainers-java/issues/170/231607960%40git.luolix.top
https://groups.google.com/a/ihabsoliman.net/d/msgid/dev/testcontainers/testcontainers-java/issues/170/231607960%40git.luolix.top?utm_medium=email&utm_source=footer
.

…tions) to resolve as a potential cause of #170

rnorth · 2016-07-10T20:59:26Z

@ihabsoliman thanks

Re the commit I've just linked above, since I can't reproduce the issue I've not been able to derive a test where it makes a difference. However, it's one of those things that I think ought to be done anyway.

dbyron0 · 2016-07-11T00:14:17Z

Yup, I was building from source. Likely won't get to https://github.com/rnorth/testcontainers-fd-debug til tomorrow.

valdisrigdon · 2016-07-11T13:32:33Z

FWIW I don't get tons of file handles when running mvn test.

valdisrigdon · 2016-07-12T16:41:18Z

So this maybe something. I was able to use jstack to grab a thread dump of the test as it's going out of control. It looks like it's using io.netty.Bootstrap. Shouldn't that be shaded so that it's guaranteed to use the version that dockerjava wants?

"pool-2-thread-1" #24 prio=5 os_prio=31 tid=0x00007fb394141000 nid=0x6027 runnable [0x0000700000e3d000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.Net.socket0(Native Method)
    at sun.nio.ch.Net.socket(Net.java:411)
    at sun.nio.ch.Net.socket(Net.java:404)
    at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:105)
    at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:61)
    at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:80)
    at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:73)
    at sun.reflect.GeneratedConstructorAccessor13.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:38)
    at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:315)
    at io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:157)
    at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:139)
    at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:120)
    at org.testcontainers.shaded.com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.connect(DockerCmdExecFactoryImpl.java:271)
    at org.testcontainers.shaded.com.github.dockerjava.netty.DockerCmdExecFactoryImpl.connect(DockerCmdExecFactoryImpl.java:209)
    at org.testcontainers.shaded.com.github.dockerjava.netty.DockerCmdExecFactoryImpl.connect(DockerCmdExecFactoryImpl.java:202)
    at org.testcontainers.shaded.com.github.dockerjava.netty.DockerCmdExecFactoryImpl.access$000(DockerCmdExecFactoryImpl.java:150)
    at org.testcontainers.shaded.com.github.dockerjava.netty.DockerCmdExecFactoryImpl$1.getChannel(DockerCmdExecFactoryImpl.java:176)
    at org.testcontainers.shaded.com.github.dockerjava.netty.InvocationBuilder.getChannel(InvocationBuilder.java:155)
    at org.testcontainers.shaded.com.github.dockerjava.netty.InvocationBuilder.get(InvocationBuilder.java:424)
    at org.testcontainers.shaded.com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:23)
    at org.testcontainers.shaded.com.github.dockerjava.netty.exec.PingCmdExec.execute(PingCmdExec.java:10)
    at org.testcontainers.shaded.com.github.dockerjava.netty.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
    at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$ping$0(DockerClientProviderStrategy.java:101)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy$$Lambda$15/1702143276.call(Unknown Source)
    at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilSuccess$0(Unreliables.java:41)
    at org.rnorth.ducttape.unreliables.Unreliables$$Lambda$16/42544488.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

valdisrigdon · 2016-07-12T17:06:42Z

I shaded io.netty and now I can get through the sample DebugTest from the other project.

rnorth · 2016-07-12T18:34:53Z

Ah, that's great to hear. Thanks for identifying that as a cause of the
issue! I'll prioritise shading netty in to the library then. This should
fix for yourself and @ihabsoliman.

I'm a little concerned that there's an underlying/alternative root cause
which is causing the same symptoms for @dbyron0, but let's see...

…170, addresses #157.

rnorth · 2016-07-12T21:28:12Z

Shaded netty dependencies, pushed to branch.

Testing of this would be much appreciated as always!

dbyron0 · 2016-07-12T21:40:30Z

At 6f41fc5 on #170-fix-resource-leak I'm still seeing some failures. I apologize in advance as I've changed versions of some underlying docker stuff since I reported this, so it's not the best test. Here's my current docker info:

$ docker info
Containers: 22
 Running: 1
 Paused: 0
 Stopped: 21
Images: 353
Server Version: 1.12.0-rc3
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 622
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: null host overlay bridge
Kernel Version: 4.4.14-boot2docker
Operating System: Boot2Docker 1.12.0-rc3 (TCL 7.1); HEAD : 8d9ee9f - Sat Jul  2 05:02:44 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.9 MiB
Name: docker-vm
ID: QAXI:YDZA:NQO6:CJ6F:222Q:FZO5:MJIB:BKUR:OCQP:WCJJ:QOUT:JYPN
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug mode (client): false
Debug mode (server): true
 File Descriptors: 19
 Goroutines: 31
 System Time: 2016-07-12T21:37:11.886656366Z
 EventsListeners: 0
Username: tildenci
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

And the results:

Tests in error: 
  org.testcontainers.junit.GenericContainerRuleTest
  simpleTest[alpine:3.3](org.testcontainers.junit.ParameterizedDockerfileContainerTest): Can't get Docker image name from org.testcontainers.images.builder.ImageFromDockerfile@54afd745
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HostPortWaitStrategyTest): Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@70d2e40b
  testWaitUntilReady_Success(org.testcontainers.junit.wait.HttpWaitStrategyTest): Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@29ad44e3
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HttpWaitStrategyTest): failed to create a child event loop

Tests run: 54, Failures: 0, Errors: 5, Skipped: 3

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] TestContainers Parent POM .......................... SUCCESS [  0.879 s]
[INFO] TestContainers Core ................................ FAILURE [02:33 min]
[INFO] TestContainers :: JDBC ............................. SKIPPED
[INFO] TestContainers :: JDBC :: MySQL .................... SKIPPED
[INFO] TestContainers :: JDBC :: PostgreSQL ............... SKIPPED
[INFO] TestContainers :: Selenium ......................... SKIPPED
[INFO] TestContainers :: nginx ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:34 min
[INFO] Finished at: 2016-07-12T14:35:11-07:00
[INFO] Final Memory: 38M/530M
[INFO] ------------------------------------------------------------------------

same stack trace as before...

rnorth · 2016-07-12T21:49:38Z

Hi David

Thanks for running this so quickly. It's useful to know that this is only
afflicting a subset of tests now. Still, it's a shame I can't reproduce
locally as I'd really like to dig in to what's causing this!

I think the next logical step is the shared DockerClient model rather than
one per container...

Richard

dbyron0 · 2016-07-12T23:13:09Z

I wish I could help more. I'm happy to try the next thing, and the thing after that, etc.

-DB

valdisrigdon · 2016-07-14T00:49:40Z

Tests pass for me on this branch.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] TestContainers Parent POM .......................... SUCCESS [  0.201 s]
[INFO] TestContainers Core ................................ SUCCESS [03:18 min]
[INFO] TestContainers :: JDBC ............................. SUCCESS [  0.912 s]
[INFO] TestContainers :: JDBC :: MySQL .................... SUCCESS [01:40 min]
[INFO] TestContainers :: JDBC :: PostgreSQL ............... SUCCESS [ 18.066 s]
[INFO] TestContainers :: Selenium ......................... SUCCESS [ 54.935 s]
[INFO] TestContainers :: nginx ............................ SUCCESS [ 20.067 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:33 min
[INFO] Finished at: 2016-07-13T20:49:00-04:00
[INFO] Final Memory: 43M/537M
[INFO] ------------------------------------------------------------------------

ihabsoliman · 2016-07-14T05:33:45Z

Hey guys,
Tests didn't pass for me

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] TestContainers Parent POM ......................... SUCCESS [1:42.456s]
[INFO] TestContainers Core ............................... FAILURE [13:48.504s]
[INFO] TestContainers :: JDBC ............................ SKIPPED
[INFO] TestContainers :: JDBC :: MySQL ................... SKIPPED
[INFO] TestContainers :: JDBC :: PostgreSQL .............. SKIPPED
[INFO] TestContainers :: Selenium ........................ SKIPPED
[INFO] TestContainers :: nginx ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15:31.181s
[INFO] Finished at: Wed Jul 13 23:29:53 CST 2016
[INFO] Final Memory: 25M/222M
[INFO] ------------------------------------------------------------------------

Full stack here or here

ihabsoliman · 2016-07-14T05:51:38Z

2nd run

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] TestContainers Parent POM ......................... SUCCESS [0.752s]
[INFO] TestContainers Core ............................... SUCCESS [4:00.198s]
[INFO] TestContainers :: JDBC ............................ SUCCESS [1.608s]
[INFO] TestContainers :: JDBC :: MySQL ................... FAILURE [2.127s]
[INFO] TestContainers :: JDBC :: PostgreSQL .............. SKIPPED
[INFO] TestContainers :: Selenium ........................ SKIPPED
[INFO] TestContainers :: nginx ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:04.791s
[INFO] Finished at: Wed Jul 13 23:48:43 CST 2016
[INFO] Final Memory: 37M/259M
[INFO] ------------------------------------------------------------------------

Full stack here or here

rnorth · 2016-07-14T19:58:01Z

@ihabsoliman thanks - that looks like it might be a different problem though. There are timeouts while pulling images.

Testcontainers is less aggressive about retrying container startup in this release, and I think what's happened is the startup retries were masking slow pulls of containers before. Now, container startup can more easily timeout if it takes a while to pull any images.

This is a bug, and I'll raise a separate ticket to address.

Shade io.netty dependencies into Testcontainers JAR Improve cleanup of docker clients

rnorth · 2016-07-17T21:28:46Z

I'm in the process of releasing v1.1.1 with the bug fixes discussed above. @dbyron0 I'm afraid this may not work for you still - until we've figured out a way to reproduce the issue you're seeing it's going to be difficult.

dbyron0 · 2016-07-17T23:01:09Z

Just gave the head of master (d445d75) a whirl. I still see some failures, but something's different. I see a (> 5 minute) hang here:

Running org.testcontainers.junit.OutputStreamTest
15:48:37.385 DEBUG 🐳 [alpine:3.2] - Starting container: alpine:3.2
15:48:37.386 DEBUG 🐳 [alpine:3.2] - Trying to start container: alpine:3.2
15:48:37.386 DEBUG 🐳 [alpine:3.2] - Trying to start container: alpine:3.2 (attempt 1/1)
15:48:37.386 DEBUG 🐳 [alpine:3.2] - Starting container: alpine:3.2
15:48:37.386 INFO  🐳 [alpine:3.2] - Creating container for image: alpine:3.2
15:48:37.421 INFO  🐳 [alpine:3.2] - Starting container with ID: f0f802f4dfa80255c9b2ca42c40d923eb2b06696cada10686de502abcd2ead08
15:48:37.521 INFO  🐳 [alpine:3.2] - Container alpine:3.2 is starting: f0f802f4dfa80255c9b2ca42c40d923eb2b06696cada10686de502abcd2ead08
15:48:37.557 INFO  🐳 [alpine:3.2] - Container alpine:3.2 started
15:48:42.527 DEBUG org.testcontainers.containers.output.WaitingConsumer - END: null

Up to that point, here are the failures:

Running org.testcontainers.junit.DockerComposeErrorHandlingTest
15:46:50.096 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Starting container: dduportal/docker-compose:1.7.1
15:46:50.097 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Trying to start container: dduportal/docker-compose:1.7.1
15:46:50.097 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Trying to start container: dduportal/docker-compose:1.7.1 (attempt 1/1)
15:46:50.097 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Starting container: dduportal/docker-compose:1.7.1
15:46:50.098 INFO  🐳 [dduportal/docker-compose:1.7.1] - Creating container for image: dduportal/docker-compose:1.7.1
15:46:50.133 INFO  🐳 [dduportal/docker-compose:1.7.1] - Starting container with ID: 93348dc318579750a4b7d8f89cf2685fb3e2862c0100bf25f6b1cc1fb02c5311
15:46:50.254 INFO  🐳 [dduportal/docker-compose:1.7.1] - Container dduportal/docker-compose:1.7.1 is starting: 93348dc318579750a4b7d8f89cf2685fb3e2862c0100bf25f6b1cc1fb02c5311
15:46:50.571 ERROR 🐳 [dduportal/docker-compose:1.7.1] - Container did not start correctly; container log output (if any) will be fetched and logged shortly
15:46:50.588 INFO  🐳 [dduportal/docker-compose:1.7.1] - STDERR: Top level object in '/compose/invalid-compose.yml' needs to be an object not '<type 'str'>'.
15:46:50.596 ERROR 🐳 [dduportal/docker-compose:1.7.1] - Could not start container
java.lang.IllegalStateException: Container did not start correctly.
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:218) [classes/:na]
    at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:164) [classes/:na]
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) ~[duct-tape-1.0.6.jar:na]
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:162) [classes/:na]
    at org.testcontainers.containers.DockerCompose.start(DockerComposeContainer.java:289) ~[classes/:na]
    at org.testcontainers.containers.DockerComposeContainer.createServices(DockerComposeContainer.java:92) ~[classes/:na]
    at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:82) ~[classes/:na]
    at org.testcontainers.junit.DockerComposeErrorHandlingTest.lambda$simpleTest$20(DockerComposeErrorHandlingTest.java:26) ~[test-classes/:na]
    at org.rnorth.visibleassertions.VisibleAssertions.assertThrows(VisibleAssertions.java:345) ~[visible-assertions-1.0.5.jar:na]
    at org.testcontainers.junit.DockerComposeErrorHandlingTest.simpleTest(DockerComposeErrorHandlingTest.java:23) ~[test-classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) ~[junit-4.12.jar:4.12]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.12.jar:4.12]
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) ~[junit-4.12.jar:4.12]
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) ~[junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) ~[junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363) ~[junit-4.12.jar:4.12]
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) ~[na:na]
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) ~[na:na]
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) ~[na:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) ~[surefire-api-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) ~[surefire-booter-2.12.4.jar:2.12.4]

testBridgedNetworkContainer(org.testcontainers.junit.DockerNetworkModeTest)  Time elapsed: 1.881 sec  <<< FAILURE!
java.lang.AssertionError: 'bridge' network can access the internet
    at org.rnorth.visibleassertions.VisibleAssertions.fail(VisibleAssertions.java:368)
    at org.rnorth.visibleassertions.VisibleAssertions.assertTrue(VisibleAssertions.java:119)
    at org.testcontainers.junit.DockerNetworkModeTest.testBridgedNetworkContainer(DockerNetworkModeTest.java:60)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

testHostNetworkContainer(org.testcontainers.junit.DockerNetworkModeTest)  Time elapsed: 0.052 sec  <<< FAILURE!
java.lang.AssertionError: 'host' network can access the internet
    at org.rnorth.visibleassertions.VisibleAssertions.fail(VisibleAssertions.java:368)
    at org.rnorth.visibleassertions.VisibleAssertions.assertTrue(VisibleAssertions.java:119)
    at org.testcontainers.junit.DockerNetworkModeTest.testHostNetworkContainer(DockerNetworkModeTest.java:53)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:31)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Running org.testcontainers.junit.GenericContainerRuleTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.296 sec <<< FAILURE!
org.testcontainers.junit.GenericContainerRuleTest  Time elapsed: 2.296 sec  <<< ERROR!
java.lang.ExceptionInInitializerError
    at sun.misc.Unsafe.ensureClassInitialized(Native Method)
    at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
    at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    at java.lang.reflect.Field.get(Field.java:393)
    at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
    at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
    at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
    at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
    at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:103)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:96)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:70)
    at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:105)
    at org.testcontainers.junit.GenericContainerRuleTest.<clinit>(GenericContainerRuleTest.java:76)
    ... 24 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 38 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 42 more

Running org.testcontainers.junit.NonExistentImagePullTest
15:47:37.524 INFO  🐳 [richnorth/nonexistent:latest] - Pulling docker image: richnorth/nonexistent:latest. Please be patient; this may take some time but only needs to be done once.
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 59.989 sec <<< FAILURE!
pullingNonExistentImageFailsGracefully(org.testcontainers.junit.NonExistentImagePullTest)  Time elapsed: 59.988 sec  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 60000 milliseconds
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
    at com.github.dockerjava.core.async.ResultCallbackTemplate.awaitCompletion(ResultCallbackTemplate.java:90)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:81)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:21)
    at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
    at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:687)
    at org.testcontainers.containers.GenericContainer.setDockerImageName(GenericContainer.java:677)
    at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:140)
    at org.testcontainers.junit.NonExistentImagePullTest.lambda$pullingNonExistentImageFailsGracefully$19(NonExistentImagePullTest.java:19)
    at org.testcontainers.junit.NonExistentImagePullTest$$Lambda$79/2035787687.call(Unknown Source)
    at org.rnorth.visibleassertions.VisibleAssertions.assertThrows(VisibleAssertions.java:320)
    at org.testcontainers.junit.NonExistentImagePullTest.pullingNonExistentImageFailsGracefully(NonExistentImagePullTest.java:17)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
    at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.lang.Thread.run(Thread.java:745)

dbyron0 · 2016-07-17T23:26:38Z

Things got better after I ran docker-machine restart docker-vm. No more hang, and fewer failures. Here's what remains:

Results :

Tests in error: 
  org.testcontainers.junit.GenericContainerRuleTest
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HostPortWaitStrategyTest): Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@fac80
  testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HttpWaitStrategyTest): Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@133e019b

Tests run: 54, Failures: 0, Errors: 3, Skipped: 3

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] TestContainers Parent POM .......................... SUCCESS [  0.790 s]
[INFO] TestContainers Core ................................ FAILURE [02:37 min]
[INFO] TestContainers :: JDBC ............................. SKIPPED
[INFO] TestContainers :: JDBC :: MySQL .................... SKIPPED
[INFO] TestContainers :: JDBC :: PostgreSQL ............... SKIPPED
[INFO] TestContainers :: Selenium ......................... SKIPPED
[INFO] TestContainers :: nginx ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:38 min
[INFO] Finished at: 2016-07-17T16:22:40-07:00
[INFO] Final Memory: 38M/532M
[INFO] ------------------------------------------------------------------------

Running org.testcontainers.junit.DockerComposeErrorHandlingTest
16:21:13.320 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Starting container: dduportal/docker-compose:1.7.1
16:21:13.320 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Trying to start container: dduportal/docker-compose:1.7.1
16:21:13.320 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Trying to start container: dduportal/docker-compose:1.7.1 (attempt 1/1)
16:21:13.321 DEBUG 🐳 [dduportal/docker-compose:1.7.1] - Starting container: dduportal/docker-compose:1.7.1
16:21:13.321 INFO  🐳 [dduportal/docker-compose:1.7.1] - Creating container for image: dduportal/docker-compose:1.7.1
16:21:13.348 INFO  🐳 [dduportal/docker-compose:1.7.1] - Starting container with ID: 0880764c67ca8e630b0957657d40da031957969ec6697832c2b2f12242e6235a
16:21:13.452 INFO  🐳 [dduportal/docker-compose:1.7.1] - Container dduportal/docker-compose:1.7.1 is starting: 0880764c67ca8e630b0957657d40da031957969ec6697832c2b2f12242e6235a
16:21:13.760 ERROR 🐳 [dduportal/docker-compose:1.7.1] - Container did not start correctly; container log output (if any) will be fetched and logged shortly
16:21:13.775 INFO  🐳 [dduportal/docker-compose:1.7.1] - STDERR: Top level object in '/compose/invalid-compose.yml' needs to be an object not '<type 'str'>'.
16:21:13.780 ERROR 🐳 [dduportal/docker-compose:1.7.1] - Could not start container
java.lang.IllegalStateException: Container did not start correctly.
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:218) [classes/:na]
    at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:164) [classes/:na]
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) ~[duct-tape-1.0.6.jar:na]
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:162) [classes/:na]
    at org.testcontainers.containers.DockerCompose.start(DockerComposeContainer.java:289) ~[classes/:na]
    at org.testcontainers.containers.DockerComposeContainer.createServices(DockerComposeContainer.java:92) ~[classes/:na]
    at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:82) ~[classes/:na]
    at org.testcontainers.junit.DockerComposeErrorHandlingTest.lambda$simpleTest$20(DockerComposeErrorHandlingTest.java:26) ~[test-classes/:na]
    at org.rnorth.visibleassertions.VisibleAssertions.assertThrows(VisibleAssertions.java:345) ~[visible-assertions-1.0.5.jar:na]
    at org.testcontainers.junit.DockerComposeErrorHandlingTest.simpleTest(DockerComposeErrorHandlingTest.java:23) ~[test-classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) ~[junit-4.12.jar:4.12]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.12.jar:4.12]
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) ~[junit-4.12.jar:4.12]
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) ~[junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) ~[junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) ~[junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363) ~[junit-4.12.jar:4.12]
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) ~[na:na]
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) ~[na:na]
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) ~[na:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) ~[surefire-api-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) ~[surefire-booter-2.12.4.jar:2.12.4]
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) ~[surefire-booter-2.12.4.jar:2.12.4]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 sec

Running org.testcontainers.junit.GenericContainerRuleTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.26 sec <<< FAILURE!
org.testcontainers.junit.GenericContainerRuleTest  Time elapsed: 2.26 sec  <<< ERROR!
java.lang.ExceptionInInitializerError
    at sun.misc.Unsafe.ensureClassInitialized(Native Method)
    at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
    at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    at java.lang.reflect.Field.get(Field.java:393)
    at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
    at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
    at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
    at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
    at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@25f9407e
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:689)
    at org.testcontainers.containers.GenericContainer.setDockerImageName(GenericContainer.java:677)
    at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:140)
    at org.testcontainers.junit.GenericContainerRuleTest.<clinit>(GenericContainerRuleTest.java:61)
    ... 24 more
Caused by: java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:103)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:96)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:43)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:21)
    at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
    at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:687)
    ... 27 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 43 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 47 more

Running org.testcontainers.junit.wait.HostPortWaitStrategyTest
16:22:32.067 DEBUG 🐳 [alpine:latest] - Starting container: alpine:latest
16:22:32.067 DEBUG 🐳 [alpine:latest] - Trying to start container: alpine:latest
16:22:32.068 DEBUG 🐳 [alpine:latest] - Trying to start container: alpine:latest (attempt 1/1)
16:22:32.068 DEBUG 🐳 [alpine:latest] - Starting container: alpine:latest
16:22:32.068 INFO  🐳 [alpine:latest] - Creating container for image: alpine:latest
16:22:32.115 INFO  🐳 [alpine:latest] - Starting container with ID: 006985d4bb2e8a44220b170ecd065d6d5753de01a315615c49347d402c03fc5f
16:22:32.242 INFO  🐳 [alpine:latest] - Container alpine:latest is starting: 006985d4bb2e8a44220b170ecd065d6d5753de01a315615c49347d402c03fc5f
16:22:33.100 INFO  🐳 [alpine:latest] - Container alpine:latest started
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.295 sec <<< FAILURE!
testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HostPortWaitStrategyTest)  Time elapsed: 2.239 sec  <<< ERROR!
org.testcontainers.containers.ContainerFetchException: Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@fac80
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:689)
    at org.testcontainers.containers.GenericContainer.setDockerImageName(GenericContainer.java:677)
    at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:140)
    at org.testcontainers.junit.wait.AbstractWaitStrategyTest.startContainerWithCommand(AbstractWaitStrategyTest.java:57)
    at org.testcontainers.junit.wait.AbstractWaitStrategyTest.waitUntilReadyAndTimeout(AbstractWaitStrategyTest.java:85)
    at org.testcontainers.junit.wait.HostPortWaitStrategyTest.testWaitUntilReady_Timeout(HostPortWaitStrategyTest.java:30)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:103)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:96)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:43)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:21)
    at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
    at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:687)
    ... 35 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 51 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 55 more

Running org.testcontainers.junit.wait.HttpWaitStrategyTest
16:22:35.363 DEBUG 🐳 [alpine:latest] - Starting container: alpine:latest
16:22:35.363 DEBUG 🐳 [alpine:latest] - Trying to start container: alpine:latest
16:22:35.363 DEBUG 🐳 [alpine:latest] - Trying to start container: alpine:latest (attempt 1/1)
16:22:35.364 DEBUG 🐳 [alpine:latest] - Starting container: alpine:latest
16:22:35.364 INFO  🐳 [alpine:latest] - Creating container for image: alpine:latest
16:22:35.389 INFO  🐳 [alpine:latest] - Starting container with ID: 934b956bc104ae5513c212f30184f264d3798247ed0538265622d5cef04f4e14
16:22:35.504 INFO  🐳 [alpine:latest] - Container alpine:latest is starting: 934b956bc104ae5513c212f30184f264d3798247ed0538265622d5cef04f4e14
16:22:35.549 INFO  🐳 [alpine:latest] - Waiting for 3 seconds for URL: http://192.168.99.100:32785/
16:22:35.563 INFO  🐳 [alpine:latest] - Container alpine:latest started
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.454 sec <<< FAILURE!
testWaitUntilReady_Timeout(org.testcontainers.junit.wait.HttpWaitStrategyTest)  Time elapsed: 2.233 sec  <<< ERROR!
org.testcontainers.containers.ContainerFetchException: Can't get Docker image name from org.testcontainers.images.RemoteDockerImage@133e019b
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:689)
    at org.testcontainers.containers.GenericContainer.setDockerImageName(GenericContainer.java:677)
    at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:140)
    at org.testcontainers.junit.wait.AbstractWaitStrategyTest.startContainerWithCommand(AbstractWaitStrategyTest.java:57)
    at org.testcontainers.junit.wait.AbstractWaitStrategyTest.waitUntilReadyAndTimeout(AbstractWaitStrategyTest.java:85)
    at org.testcontainers.junit.wait.HttpWaitStrategyTest.testWaitUntilReady_Timeout(HttpWaitStrategyTest.java:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.IllegalStateException: failed to create a child event loop
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:48)
    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:57)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:67)
    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:54)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl$InetSocketInitializer.init(DockerCmdExecFactoryImpl.java:241)
    at com.github.dockerjava.netty.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:197)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:159)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClientForConfig(DockerClientProviderStrategy.java:103)
    at org.testcontainers.dockerclient.DockerClientProviderStrategy.getClient(DockerClientProviderStrategy.java:96)
    at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:85)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:43)
    at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:21)
    at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
    at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
    at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:687)
    ... 35 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
    ... 51 more
Caused by: java.io.IOException: Too many open files
    at sun.nio.ch.IOUtil.makePipe(Native Method)
    at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
    at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
    ... 55 more

dbyron0 · 2016-07-20T15:14:54Z

Same as before with a86827b, effectively 1.1.2.

rnorth · 2016-07-20T19:16:03Z

David
Thanks for re-testing. I'm afraid it could be a while until we have an
answer. It might be a challenge to schedule but perhaps we could arrange a
slack or Skype conversation to do some deeper debugging. Please let me know
what you think. You can sign up for Testcontainers slack at
slack.testcontainers.org ...
Thanks

Richard

rnorth · 2016-08-16T20:12:07Z

As far as we've been able to tell the changes in 1.1.4 should fix this. Will close. Thanks to all for your patience and efforts in diagnosis.

rnorth mentioned this issue Jul 6, 2016

Docker 1.12 support: "starting container with HostConfig was deprecated since v1.10 and removed in v1.12" #163

Closed

rnorth added the type/bug label Jul 6, 2016

rnorth mentioned this issue Jul 8, 2016

Netty 'IllegalStateException: executor not accepting a task' docker-java/docker-java#632

Closed

rnorth added a commit that referenced this issue Jul 10, 2016

Introduce rate limiter for docker daemon pinging (when testing connec…

fcea783

…tions) to resolve as a potential cause of #170

rnorth added a commit that referenced this issue Jul 12, 2016

Shade io.netty dependencies to avoid conflicts. May partially improve #…

6f41fc5

…170, addresses #157.

This was referenced Jul 14, 2016

Pulling of images and Selenium startup failures more likely to cause container startup timeout in v1.1.0 #172

Closed

Shade io.netty dependencies into Testcontainers JAR #175

Merged

rnorth added a commit that referenced this issue Jul 16, 2016

Merge pull request #175 from testcontainers/#170-fix-resource-leak

a3c0fee

Shade io.netty dependencies into Testcontainers JAR Improve cleanup of docker clients

rnorth closed this as completed Aug 16, 2016

test failures in version 1.1.0 #170

test failures in version 1.1.0 #170

Comments

dbyron0 commented Jul 6, 2016

ihabsoliman commented Jul 6, 2016

ihabsoliman commented Jul 6, 2016

rnorth commented Jul 6, 2016

rnorth commented Jul 6, 2016

dbyron0 commented Jul 6, 2016

ihabsoliman commented Jul 6, 2016

rnorth commented Jul 6, 2016

rnorth commented Jul 6, 2016

dbyron0 commented Jul 6, 2016

rnorth commented Jul 6, 2016

dbyron0 commented Jul 6, 2016

ihabsoliman commented Jul 7, 2016

rnorth commented Jul 8, 2016

valdisrigdon commented Jul 8, 2016

valdisrigdon commented Jul 8, 2016 • edited Loading

rnorth commented Jul 8, 2016

valdisrigdon commented Jul 8, 2016

rnorth commented Jul 8, 2016

valdisrigdon commented Jul 8, 2016

rnorth commented Jul 8, 2016 • edited Loading

valdisrigdon commented Jul 8, 2016

rnorth commented Jul 8, 2016 • edited Loading

rnorth commented Jul 9, 2016

valdisrigdon commented Jul 9, 2016

valdisrigdon commented Jul 9, 2016

rnorth commented Jul 10, 2016

ihabsoliman commented Jul 10, 2016

rnorth commented Jul 10, 2016

dbyron0 commented Jul 11, 2016

valdisrigdon commented Jul 11, 2016

valdisrigdon commented Jul 12, 2016

valdisrigdon commented Jul 12, 2016

rnorth commented Jul 12, 2016 • edited Loading

rnorth commented Jul 12, 2016

dbyron0 commented Jul 12, 2016

rnorth commented Jul 12, 2016

Richard

dbyron0 commented Jul 12, 2016

valdisrigdon commented Jul 14, 2016

ihabsoliman commented Jul 14, 2016

ihabsoliman commented Jul 14, 2016

rnorth commented Jul 14, 2016

rnorth commented Jul 17, 2016

dbyron0 commented Jul 17, 2016 • edited Loading

dbyron0 commented Jul 17, 2016

dbyron0 commented Jul 20, 2016

rnorth commented Jul 20, 2016

Richard

rnorth commented Aug 16, 2016

valdisrigdon commented Jul 8, 2016 •

edited

Loading

rnorth commented Jul 8, 2016 •

edited

Loading

rnorth commented Jul 8, 2016 •

edited

Loading

rnorth commented Jul 12, 2016 •

edited

Loading

dbyron0 commented Jul 17, 2016 •

edited

Loading