"Can not connect to Ryuk" on CircleCI 2.0 with Docker executor

[rafaelalmeida]

Overview

I'm trying to run a build with Testcontainers on CircleCI 2.0 with the Docker executor (not the Machine executor, as recommended by the docs, see below). The build fails with java.lang.IllegalStateException: Can not connect to Ryuk.

My question: Is there a chance this will work?

On using CircleCI Docker executor vs Machine executor

The Testcontainers docs suggest using the machine executor type in CircleCI instead of docker: https://www.testcontainers.org/ci/ci.html#circleci-20

I have not tested this, but I assume it would work.

However, I am trying to avoid using the machine executor, since it has some limitations with startup time and custom images, and also "may require additional fees in a future pricing update" (source).

The docker executor (aka "the new one") does support using Docker itself in the build (to build images, etc), by using the "Remote Docker" feature, calling the setup_remote_docker command in the config file.

Full log of my investigation

The Docker runner in CircleCI communicates the Remote Docker host location to the running processes in the build by setting the usual env variables ($DOCKER_HOST, etc).

Testcontainers recognizes this configuration and reports as follows:

2018-12-12 21:30:51,892 INFO  o.t.dockerclient.DockerClientProviderStrategy  - Found Docker environment with Environment variables, system properties and defaults. Resolved: 
    dockerHost=tcp://35.237.241.162:2376
    apiVersion='{UNKNOWN_VERSION}'
    registryUrl='https://index.docker.io/v1/'
    registryUsername='circleci'
    registryPassword='null'
    registryEmail='null'
    dockerConfig='DefaultDockerClientConfig[dockerHost=tcp://35.237.241.162:2376,registryUsername=circleci,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/home/circleci/.docker,sslConfig=LocalDirectorySSLConfig{dockerCertPath=/tmp/docker-certs117526089},apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]''

It seems that Testcontainers is indeed able to work with this remote Docker host, since it reports having downloaded the Ruyk image:

2018-12-12 21:30:53,320 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Status: Downloaded newer image for quay.io/testcontainers/ryuk:0.2.2,progressDetail=<null>,progress=<null>,id=<null>,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]

It also shows lots of ResponseItem.ProgressDetail logs before that while downloading the image.

I can check that it indeed download the image by SSH'ing into the build container as CircleCI allows and running docker images:

circleci@1fc8c94e899d:~$ docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
quay.io/testcontainers/ryuk   0.2.2               527073eb32d1        10 months ago       9.31MB

According to the full log (reproduced below), at time 21:30:53,605, dockerjava issues a start container command, then almost immediately (21:30:53,924) tries to inspect it. Then it hangs for the timeout period, 30 seconds, and spits out the SEVERE error at 21:31:23.

I've tried increasing the Ryuk timeout by setting ryuk.container.timeout to 120 seconds as suggested in #581. The timeout did change, because the build hanged for 2 minutes before closing with the same exception.

It seems the start command never ran, because nothing is listed when I run docker ps -a inside the build container.

I have also tried to debug Ruyk independently, running the command below as suggested by @jmaicher in #572:

docker run -p8080:8080 -v /var/run/docker.sock:/var/run/docker.sock --name ryuk -P -d quay.io/testcontainers/ryuk:0.2.2

It started, but then quits after waiting for the initial connection:

2018-12-12T21:46:29.814640890Z 2018/12/12 21:46:29 Starting on port 8080...
2018-12-12T21:47:29.818668885Z panic: Timed out waiting for the initial connection
2018-12-12T21:47:29.818704221Z
2018-12-12T21:47:29.818708974Z goroutine 1 [running]:
2018-12-12T21:47:29.818717702Z main.main()
2018-12-12T21:47:29.818721719Z 	/go/src/github.com/bsideup/moby-ryuk/main.go:92 +0xe1e

I believe it says that it was able to the Docker Engine, since if I start without mapping the socket it exits immediately with panic: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?. It does not say if it's indeed reachable, since I believe Ryuk listens for messages sent by the ResourceReaper class in Testcontainers, which wasn't running at the time.

I have also tried to check for port connectivity, and indeed CircleCI seems to close ports. I ran the nginx image mapping port 80 to 30080, and tried curl -v 35.237.241.162:30080/, which just hangs there. So, Ryuk working in this scenario may be a problem even if the container did start.

Full log and stack trace

2018-12-12 21:30:51,061 DEBUG o.t.utility.TestcontainersConfiguration        - Testcontainers configuration overrides will be loaded from file:/home/circleci/exampleproject/target/test-classes/testcontainers.properties
2018-12-12 21:30:51,063 DEBUG o.t.utility.TestcontainersConfiguration        - Testcontainers configuration overrides will be loaded from file:/home/circleci/exampleproject/target/test-classes/testcontainers.properties
2018-12-12 21:30:51,063 DEBUG o.t.utility.TestcontainersConfiguration        - Testcontainers configuration overrides will be loaded from file:/home/circleci/.testcontainers.properties
2018-12-12 21:30:51,063 DEBUG o.t.utility.TestcontainersConfiguration        - Testcontainers configuration overrides loaded from TestcontainersConfiguration(properties={checks.disable=false})
2018-12-12 21:30:51,137 INFO  o.t.dockerclient.DockerClientProviderStrategy  - Will use 'okhttp' transport
2018-12-12 21:30:51,406 DEBUG o.t.dockerclient.DockerClientProviderStrategy  - Pinging docker daemon...
2018-12-12 21:30:51,607 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: org.testcontainers.dockerclient.transport.okhttp.OkHttpDockerCmdExecFactory$1@15de4639
2018-12-12 21:30:51,881 INFO  ronmentAndSystemPropertyClientProviderStrategy - Found docker client settings from environment
2018-12-12 21:30:51,892 INFO  o.t.dockerclient.DockerClientProviderStrategy  - Found Docker environment with Environment variables, system properties and defaults. Resolved: 
    dockerHost=tcp://35.237.241.162:2376
    apiVersion='{UNKNOWN_VERSION}'
    registryUrl='https://index.docker.io/v1/'
    registryUsername='circleci'
    registryPassword='null'
    registryEmail='null'
    dockerConfig='DefaultDockerClientConfig[dockerHost=tcp://35.237.241.162:2376,registryUsername=circleci,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/home/circleci/.docker,sslConfig=LocalDirectorySSLConfig{dockerCertPath=/tmp/docker-certs117526089},apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'

2018-12-12 21:30:51,895 INFO  org.testcontainers.DockerClientFactory         - Docker host IP address is 35.237.241.162
2018-12-12 21:30:51,897 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: com.github.dockerjava.core.exec.InfoCmdExec@5c669da8
2018-12-12 21:30:52,032 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: com.github.dockerjava.core.exec.VersionCmdExec@14fc1f0
2018-12-12 21:30:52,049 INFO  org.testcontainers.DockerClientFactory         - Connected to docker: 
  Server Version: 17.09.0-ce
  API Version: 1.32
  Operating System: Ubuntu 16.04.3 LTS
  Total Memory: 7477 MB
2018-12-12 21:30:52,051 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: ListImagesCmdImpl[imageNameFilter=quay.io/testcontainers/ryuk:0.2.2,showAll=false,filters=com.github.dockerjava.core.util.FiltersBuilder@0,execution=com.github.dockerjava.core.exec.ListImagesCmdExec@66c61024]
2018-12-12 21:30:52,078 DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: quay.io/testcontainers/ryuk:0.2.2
2018-12-12 21:30:52,078 DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/circleci/.docker/config.json (does not exist) and commandPathPrefix: 
2018-12-12 21:30:52,079 WARN  org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config (dockerImageName: quay.io/testcontainers/ryuk:0.2.2, configFile: /home/circleci/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/circleci/.docker/config.json (No such file or directory)
2018-12-12 21:30:52,079 DEBUG o.t.d.auth.AuthDelegatingDockerClientConfig    - Effective auth config [null]
2018-12-12 21:30:52,479 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pulling from testcontainers/ryuk,progressDetail=<null>,progress=<null>,id=0.2.2,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,480 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pulling fs layer,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,480 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pulling fs layer,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,480 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pulling fs layer,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,674 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Downloading,progressDetail=ResponseItem.ProgressDetail[current=30864,total=1776708,start=<null>],progress=[>                                                  ]  30.86kB/1.777MB,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,676 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Downloading,progressDetail=ResponseItem.ProgressDetail[current=20728,total=2065537,start=<null>],progress=[>                                                  ]  20.73kB/2.066MB,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,709 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Downloading,progressDetail=ResponseItem.ProgressDetail[current=3610,total=308109,start=<null>],progress=[>                                                  ]   3.61kB/308.1kB,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,775 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Downloading,progressDetail=ResponseItem.ProgressDetail[current=620688,total=1776708,start=<null>],progress=[=================>                                 ]  620.7kB/1.777MB,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,777 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Downloading,progressDetail=ResponseItem.ProgressDetail[current=545016,total=2065537,start=<null>],progress=[=============>                                     ]    545kB/2.066MB,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,790 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Download complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,811 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Verifying Checksum,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,811 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Download complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,826 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Verifying Checksum,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,826 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Download complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,827 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=32768,total=2065537,start=<null>],progress=[>                                                  ]  32.77kB/2.066MB,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,927 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=1245184,total=2065537,start=<null>],progress=[==============================>                    ]  1.245MB/2.066MB,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:52,981 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=2065537,total=2065537,start=<null>],progress=[==================================================>]  2.066MB/2.066MB,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,015 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pull complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=ff3a5c916c92,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,019 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=32768,total=308109,start=<null>],progress=[=====>                                             ]  32.77kB/308.1kB,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,109 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=308109,total=308109,start=<null>],progress=[==================================================>]  308.1kB/308.1kB,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,111 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=308109,total=308109,start=<null>],progress=[==================================================>]  308.1kB/308.1kB,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,146 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pull complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=c9ea39c9283a,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,150 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=32768,total=1776708,start=<null>],progress=[>                                                  ]  32.77kB/1.777MB,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,252 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=1277952,total=1776708,start=<null>],progress=[===================================>               ]  1.278MB/1.777MB,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,278 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Extracting,progressDetail=ResponseItem.ProgressDetail[current=1776708,total=1776708,start=<null>],progress=[==================================================>]  1.777MB/1.777MB,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,309 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Pull complete,progressDetail=ResponseItem.ProgressDetail[current=<null>,total=<null>,start=<null>],progress=<null>,id=f95d3189d433,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,316 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Digest: sha256:3d53f437c84a355eafccfef4a0fe3270c49893bd2ac80b5ef57698c1a9102fec,progressDetail=<null>,progress=<null>,id=<null>,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,320 DEBUG c.g.d.core.command.PullImageResultCallback     - PullResponseItem[stream=<null>,status=Status: Downloaded newer image for quay.io/testcontainers/ryuk:0.2.2,progressDetail=<null>,progress=<null>,id=<null>,from=<null>,time=<null>,errorDetail=<null>,error=<null>,aux=<null>]
2018-12-12 21:30:53,326 DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: quay.io/testcontainers/ryuk:0.2.2
2018-12-12 21:30:53,326 DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/circleci/.docker/config.json (does not exist) and commandPathPrefix: 
2018-12-12 21:30:53,326 WARN  org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config (dockerImageName: quay.io/testcontainers/ryuk:0.2.2, configFile: /home/circleci/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/circleci/.docker/config.json (No such file or directory)
2018-12-12 21:30:53,326 DEBUG o.t.d.auth.AuthDelegatingDockerClientConfig    - Effective auth config [null]
2018-12-12 21:30:53,340 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: com.github.dockerjava.core.command.CreateContainerCmdImpl@7f132176[name=testcontainers-ryuk-150ffa6c-872e-4ca0-abd5-cd3b2bdea1e4,hostName=<null>,domainName=<null>,user=<null>,attachStdin=<null>,attachStdout=<null>,attachStderr=<null>,portSpecs=<null>,tty=<null>,stdinOpen=<null>,stdInOnce=<null>,env=<null>,cmd=<null>,entrypoint=<null>,image=quay.io/testcontainers/ryuk:0.2.2,volumes=com.github.dockerjava.api.model.Volumes@8ad6665,workingDir=<null>,macAddress=<null>,networkDisabled=<null>,exposedPorts=com.github.dockerjava.api.model.ExposedPorts@30af5b6b,stopSignal=<null>,hostConfig=com.github.dockerjava.api.model.HostConfig@19835e64[binds=com.github.dockerjava.api.model.Binds@42039326,blkioWeight=<null>,blkioWeightDevice=<null>,blkioDeviceReadBps=<null>,blkioDeviceReadIOps=<null>,blkioDeviceWriteBps=<null>,blkioDeviceWriteIOps=<null>,memorySwappiness=<null>,capAdd=<null>,capDrop=<null>,containerIDFile=<null>,cpuPeriod=<null>,cpuShares=<null>,cpuQuota=<null>,cpusetCpus=<null>,cpusetMems=<null>,devices=<null>,diskQuota=<null>,dns=<null>,dnsSearch=<null>,extraHosts=<null>,links=<null>,logConfig=<null>,lxcConf=<null>,memory=<null>,memorySwap=<null>,memoryReservation=<null>,kernelMemory=<null>,networkMode=<null>,oomKillDisable=<null>,autoRemove=true,oomScoreAdj=<null>,portBindings=<null>,privileged=false,publishAllPorts=true,readonlyRootfs=<null>,restartPolicy=<null>,ulimits=<null>,volumesFrom=<null>,pidMode=<null>,securityOpts=<null>,cgroupParent=<null>,volumeDriver=<null>,shmSize=<null>,pidsLimit=<null>,runtime=<null>,tmpFs=<null>],labels={org.testcontainers=true},networkingConfig=<null>,ipv4Address=<null>,ipv6Address=<null>,aliases=<null>,authConfig=<null>,execution=com.github.dockerjava.core.exec.CreateContainerCmdExec@33308786]
2018-12-12 21:30:53,605 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: 734ce3894816f33b7645d9da779fb9ccf5ba1d75d04fb03717681f3282681ee3,com.github.dockerjava.core.exec.StartContainerCmdExec@487db668
2018-12-12 21:30:53,923 DEBUG c.g.dockerjava.core.command.AbstrDockerCmd     - Cmd: 734ce3894816f33b7645d9da779fb9ccf5ba1d75d04fb03717681f3282681ee3,false,com.github.dockerjava.core.exec.InspectContainerCmdExec@43f82e78
2018-12-12 21:30:53,924 DEBUG c.g.d.core.exec.InspectContainerCmdExec        - GET: OkHttpWebTarget(okHttpClient=org.testcontainers.shaded.okhttp3.OkHttpClient@e54303, baseUrl=https://35.237.241.162:2376/, path=[/containers/734ce3894816f33b7645d9da779fb9ccf5ba1d75d04fb03717681f3282681ee3/json], queryParams={})
Dec 12, 2018 9:31:23 PM org.springframework.test.context.TestContextManager prepareTestInstance
SEVERE: Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener@442675e1] to prepare test instance [com.example.test.integration.business.service.BasicIT@32193bea]
java.lang.ExceptionInInitializerError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:102)
	at org.springframework.test.context.support.AbstractContextLoader.invokeApplicationContextInitializers(AbstractContextLoader.java:165)
	at org.springframework.test.context.support.AbstractContextLoader.prepareContext(AbstractContextLoader.java:138)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:122)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:108)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:251)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116)
	at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:228)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:287)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:289)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:247)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.lang.IllegalStateException: Can not connect to Ryuk
	at org.testcontainers.utility.ResourceReaper.start(ResourceReaper.java:148)
	at org.testcontainers.utility.ResourceReaper.start(ResourceReaper.java:64)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:124)
	at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:170)
	at org.testcontainers.containers.JdbcDatabaseContainer.<init>(JdbcDatabaseContainer.java:45)
	at org.testcontainers.containers.MySQLContainer.<init>(MySQLContainer.java:29)
	at com.example.test.integration.mysql.MySqlInitializer.<clinit>(MySqlInitializer.java:38)
	... 41 more

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 33.191 s <<< FAILURE! - in com.example.test.integration.business.service.BasicIT
[ERROR] setupScenario(com.example.test.integration.business.service.BasicIT)  Time elapsed: 0 s  <<< ERROR!
java.lang.ExceptionInInitializerError
Caused by: java.lang.IllegalStateException: Can not connect to Ryuk

[vmassol]

I'm having exactly the same issue but using a Jenkins Docker agent. Any progress on your side @rafaelalmeida ? Thx

[kiview]

Worst case, you can try to disable Ryuk to get CI working: #1181

[vmassol]

@kiview Good idea. This is something I'll test tomorrow and which maybe could provide more info about the problem but in the end we'll need Ruyk, especially on a CI agent. I'll report on the chat at https://testcontainers.slack.com/archives/C1SUBPZK6/p1549555885186300

[vmassol]

FTR I got this working, see #1277

[rafaelalmeida]

@vmassol Nope, didn't get it working here. In #1277 it seems you got this working by opening the remote Docker port in iptables, but this does not seem to be an option in CircleCI.

We ended up adding code to use the Testcontainers integrations in local builds (to make it easy for developers to run tests), but allow usage of a provisioned database (controlled by environment variables).

[BalmungSan]

Effectively the problem is that the containers initiated inside CircleCI are not accessible from the main container.
If any one is interested in this, it would be good to vote for this to be implemented directly from CircleCI.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

[stale]

This issue has been automatically closed due to inactivity. We apologise if this is still an active problem for you, and would ask you to re-open the issue if this is the case.