Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for WaitStrategy on DockerComposeContainer - alternative approach #600

Merged
merged 10 commits into from
Mar 19, 2018
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ All notable changes to this project will be documented in this file.
- Abstracted and changed database init script functionality to support use of SQL-like scripts with non-JDBC connections. ([\#551](https://github.com/testcontainers/testcontainers-java/pull/551))
- Added `JdbcDatabaseContainer(Future)` constructor. ([\#543](https://github.com/testcontainers/testcontainers-java/issues/543))
- Mark DockerMachineClientProviderStrategy as not persistable ([\#593](https://github.com/testcontainers/testcontainers-java/pull/593))
- Added overloaded `withExposedService()` methods to `DockerComposeContainer` to allow user to define `WaitStrategy` for compose containers. ([\#174](https://github.com/testcontainers/testcontainers-java/issues/174) and [\#515](https://github.com/testcontainers/testcontainers-java/issues/515))
- Deprecated `WaitStrategy` and implementations in favour of classes with same names in `org.testcontainers.containers.strategy`
- Added `ContainerState` interface representing the state of a started container
- Added `WaitStrategyTarget` interface which is the target of the new `WaitStrategy`

## [1.6.0] - 2018-01-28

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package org.testcontainers.containers;

import com.github.dockerjava.api.command.InspectContainerResponse;
import com.github.dockerjava.api.model.Container;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import org.slf4j.Logger;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.wait.strategy.WaitStrategyTarget;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;


/**
* Class to provide a wait strategy target for services started through docker-compose
*/
@EqualsAndHashCode
class ComposeServiceWaitStrategyTarget implements WaitStrategyTarget {

private final Container container;
private final GenericContainer proxyContainer;
@Getter
private final Logger logger;
private Map<Integer, Integer> mappedPorts = new HashMap<>();
@Getter
private List<Integer> exposedPorts = new ArrayList<>();
private InspectContainerResponse containerInfo;

ComposeServiceWaitStrategyTarget(Container container, GenericContainer proxyContainer,
Logger logger, Map<Integer, Integer> mappedPorts) {
this.container = container;

this.proxyContainer = proxyContainer;
this.logger = logger;

if (mappedPorts != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add @NonNull to mappedPorts argument and use this.mappedPorts = new HashMap(mappedPorts)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A ComposeServiceWaitStrategyTarget is created for every service returned by listChildContainers() in DockerComposeContainer, which can return services with no exposed ports.

for example:

redis:
  image: redis
db:
  image: orchardup/mysql

and a test with

new DockerComposeContainer(new File("src/test/resources/compose-test.yml"))
            .withExposedService("redis_1", REDIS_PORT, Wait.forListeningPort());

Only the redis container has exposed/mapped ports, so the ComposeServiceWaitStrategyTarget for the db service would have a null value for mappedPorts.

I'll change the behaviour to only create a ComposeServiceWaitStrategyTarget for services that have been explicitly exposed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should never accept null as a valid collection argument's value, that was the motivation :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the behaviour to only create a ComposeServiceWaitStrategyTarget for services that have been explicitly exposed.

Now I think about use cases where waiting strategy doesn't use the ports at all but checks the logs for instance.
Current API doesn't allow that without exposing a port, right?

Should we maybe add waitingFor("redis_1", Wait.forLogMessage()) method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add that method, but then we get into the problem of a ComposeServiceWaitStrategyTarget with no exposed ports, so the constructor would have to accept a null or empty value for mappedPorts, which would mean returning a empty list from getExposedPorts.

That goes against this idea:

We should never accept null as a valid collection argument's value, that was the motivation :)

It also means you could use DockerComposeContainer like this:

new DockerComposeContainer(new File("src/test/resources/compose-test.yml"))
            .withExposedService("redis_1", REDIS_PORT, Wait.forListeningPort());

or like this:

new DockerComposeContainer(new File("src/test/resources/compose-test.yml"))
            .withExposedService("redis_1", REDIS_PORT)
            .waitingFor("redis_1", Wait.forListeningPort());

Is that slightly confusing for the user?

Copy link
Member

@bsideup bsideup Mar 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but then we get into the problem of a ComposeServiceWaitStrategyTarget with no exposed ports, so the constructor would have to accept a null or empty value for mappedPorts, which would mean returning a empty list from getExposedPorts

Sounds absolutely fine to me - no exposed ports means an empty set.

Second example IMO doesn't look confusing, and makes even more sense because we're actually waiting for a service and not for the exposed port :)

Just .withExposedService("redis_1", REDIS_PORT, Wait.forListeningPort()) is a good alias/shortcut for it

this.mappedPorts.putAll(mappedPorts);
this.exposedPorts.addAll(this.mappedPorts.keySet());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you store exposedPorts when it's just an alias to this.mappedPorts.keySet()? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the getExposedPorts() method from the ContainerState interface returns Set<Integer>, while this.mappedPorts.keySet() returns List<Integer>.

I didn't want to create a new Set every time the getter is called.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keySet() returns Set as far as I remember :) Maybe you meant that getExposedPorts returns List<Integer> and not Set<Integer>?

Also, maybe we should make getExposedPortNumbers return Set<Integer>, use it where possible, and add getExposedPorts to ComposeServiceWaitStrategyTarget with getExposedPortNumbers().stream.collect(toList()) implementation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry, obviously keySet() returns a Set! :-)

getExposedPortNumbers().stream.collect(toList())

Isn't that functionally the same as new ArrayList<>(getExposedPortNumbers());?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is :) Both are equally fine to me since we're talking about collections with just a few elements

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add getExposedPorts to ComposeServiceWaitStrategyTarget with getExposedPortNumbers().stream.collect(toList()) implementation?

Ah, I've just realised why this won't work. getExposedPortNumbers() in ContainerState is a default method that calls getExposedPort()!

default Set<Integer> getExposedPortNumbers() {
        return getExposedPorts().stream()
            .map(this::getMappedPort)
            .collect(Collectors.toSet());
    }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just override (implement) both inComposeServiceWaitStrategyTarget , getExposedPortNumbers and getExposedPorts

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I didn't notice that! Yes, sure!

P.S. I checked the implementation and it seems it is only used in WaitStrategyTarget ::getLivenessCheckPortNumbers.

Let's just remove this method and use it's implementation in WaitStrategyTarget.
WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Despite the similar names, getExposedPorts() and getExposedPortNumbers() do different things.

I think getExposedPortNumbers() is confusingly named, because what it actually does is return the port that the service port has been mapped to.

e.g. .withExposedService("redis_1", 6379).
getExposedPorts() returns 6379 and getExposedPortNumbers() would return e.g. 32679

I took the name from a private method in GenericContainer, but it should maybe be renamed in ContainerState to getMappedPorts(). WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I deleted the original comment, because I was starting to confuse myself.
Yes, it's only used in WaitStrategyTarget ::getLivenessCheckPortNumbers
I agree, it can definitely go!

}
}

/**
* {@inheritDoc}
*/
@Override
public Integer getMappedPort(int originalPort) {
return this.proxyContainer.getMappedPort(this.mappedPorts.get(originalPort));
}

/**
* {@inheritDoc}
*/
@Override
public String getContainerId() {
return this.container.getId();
}

@Override
public InspectContainerResponse getContainerInfo() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this method and use:

@Getter(lazy=true)
private InspectContainerResponse containerInfo = DockerClientFactory.instance().client().inspectContainerCmd(getContainerId()).exec();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will also make it thread safe

if(containerInfo == null) {
containerInfo = DockerClientFactory.instance().client().inspectContainerCmd(getContainerId()).exec();
}
return containerInfo;
}
}
83 changes: 19 additions & 64 deletions core/src/main/java/org/testcontainers/containers/Container.java
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
package org.testcontainers.containers;

import com.github.dockerjava.api.DockerClient;
import com.github.dockerjava.api.command.InspectContainerResponse;
import com.github.dockerjava.api.model.Bind;
import com.github.dockerjava.api.model.Info;
import lombok.NonNull;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.output.OutputFrame;
import org.testcontainers.containers.startupcheck.StartupCheckStrategy;
import org.testcontainers.containers.traits.LinkableContainer;
import org.testcontainers.containers.wait.Wait;
import org.testcontainers.containers.wait.WaitStrategy;
import org.testcontainers.containers.wait.strategy.WaitStrategy;
import org.testcontainers.utility.LogUtils;
import org.testcontainers.utility.MountableFile;

import java.io.IOException;
Expand All @@ -22,7 +22,7 @@
import java.util.function.Consumer;
import java.util.function.Function;

public interface Container<SELF extends Container<SELF>> extends LinkableContainer {
public interface Container<SELF extends Container<SELF>> extends LinkableContainer, ContainerState {

/**
* @return a reference to this container instance, cast to the expected generic type.
Expand All @@ -32,6 +32,11 @@ default SELF self() {
return (SELF) this;
}

@Override
default String getContainerName() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change is needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HostPortWaitStrategy uses getContainerName(), which was previously protected visibility in GenericContainer.
It's only used for a debug statement, so it could probably be removed if you think this method shouldn't be added.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the method was protected, this is a breaking change anyway (making it public)
Let's just not expose this method and use getContainerInfo().getName() in HostPortWaitStrategy, or maybe even remove it since the name is not that helpful anyway (we assign random names)

return ContainerState.super.getContainerName();
}

/**
* Class to hold results from a "docker exec" command. Note that, due to the limitations of the
* docker API, there's no easy way to get the result code from the process we ran.
Expand Down Expand Up @@ -131,7 +136,7 @@ default void addFileSystemBind(final String hostPath, final String containerPath
/**
* Specify the {@link WaitStrategy} to use to determine if the container is ready.
*
* @see Wait#defaultWaitStrategy()
* @see org.testcontainers.containers.wait.strategy.Wait#defaultWaitStrategy()
* @param waitStrategy the WaitStrategy to use
* @return this
*/
Expand Down Expand Up @@ -283,7 +288,7 @@ default SELF withClasspathResourceMapping(final String resourcePath, final Strin

/**
* Set the duration of waiting time until container treated as started.
* @see WaitStrategy#waitUntilReady(GenericContainer)
* @see WaitStrategy#waitUntilReady(org.testcontainers.containers.wait.strategy.WaitStrategyTarget)
*
* @param startupTimeout timeout
* @return this
Expand All @@ -297,13 +302,6 @@ default SELF withClasspathResourceMapping(final String resourcePath, final Strin
*/
SELF withPrivilegedMode(boolean mode);

/**
* Get the IP address that this container may be reached on (may not be the local machine).
*
* @return an IP address
*/
String getContainerIpAddress();

/**
* Only consider a container to have successfully started if it has been running for this duration. The default
* value is null; if that's the value, ignore this check.
Expand All @@ -327,33 +325,6 @@ default SELF withClasspathResourceMapping(final String resourcePath, final Strin
*/
SELF withWorkingDirectory(String workDir);

/**
* @return is the container currently running?
*/
Boolean isRunning();

/**
* Get the actual mapped port for a first port exposed by the container.
*
* @return the port that the exposed port is mapped to
* @throws IllegalStateException if there are no exposed ports
*/
default Integer getFirstMappedPort() {
return getExposedPorts()
.stream()
.findFirst()
.map(this::getMappedPort)
.orElseThrow(() -> new IllegalStateException("Container doesn't expose any ports"));
}

/**
* Get the actual mapped port for a given port exposed by the container.
*
* @param originalPort the original TCP port that is exposed
* @return the port that the exposed port is mapped to, or null if it is not exposed
*/
Integer getMappedPort(int originalPort);

/**
* <b>Resolve</b> Docker image and set it.
*
Expand Down Expand Up @@ -386,7 +357,9 @@ default Integer getFirstMappedPort() {
*
* @param consumer consumer that the frames should be sent to
*/
void followOutput(Consumer<OutputFrame> consumer);
default void followOutput(Consumer<OutputFrame> consumer) {
LogUtils.followOutput(DockerClientFactory.instance().client(), getContainerId(), consumer);
}

/**
* Follow container output, sending each frame (usually, line) to a consumer. This method allows Stdout and/or stderr
Expand All @@ -395,7 +368,9 @@ default Integer getFirstMappedPort() {
* @param consumer consumer that the frames should be sent to
* @param types types that should be followed (one or both of STDOUT, STDERR)
*/
void followOutput(Consumer<OutputFrame> consumer, OutputFrame.OutputType... types);
default void followOutput(Consumer<OutputFrame> consumer, OutputFrame.OutputType... types) {
LogUtils.followOutput(DockerClientFactory.instance().client(), getContainerId(), consumer, types);
}


/**
Expand All @@ -419,22 +394,15 @@ default Integer getFirstMappedPort() {
* Run a command inside a running container, as though using "docker exec", and interpreting
* the output as UTF8.
* <p>
* @see #execInContainer(Charset, String...)
* @see ExecInContainerPattern#execInContainer(com.github.dockerjava.api.command.InspectContainerResponse, org.slf4j.Logger, String...)
*/
ExecResult execInContainer(String... command)
throws UnsupportedOperationException, IOException, InterruptedException;

/**
* Run a command inside a running container, as though using "docker exec".
* <p>
* This functionality is not available on a docker daemon running the older "lxc" execution driver. At
* the time of writing, CircleCI was using this driver.
* @param outputCharset the character set used to interpret the output.
* @param command the parts of the command to run
* @return the result of execution
* @throws IOException if there's an issue communicating with Docker
* @throws InterruptedException if the thread waiting for the response is interrupted
* @throws UnsupportedOperationException if the docker daemon you're connecting to doesn't support "exec".
* @see ExecInContainerPattern#execInContainer(com.github.dockerjava.api.command.InspectContainerResponse, Charset, org.slf4j.Logger, String...)
*/
ExecResult execInContainer(Charset outputCharset, String... command)
throws UnsupportedOperationException, IOException, InterruptedException;
Expand All @@ -460,8 +428,6 @@ ExecResult execInContainer(Charset outputCharset, String... command)
*/
void copyFileFromContainer(String containerPath, String destinationPath) throws IOException, InterruptedException;

List<Integer> getExposedPorts();

List<String> getPortBindings();

List<String> getExtraHosts();
Expand Down Expand Up @@ -496,17 +462,6 @@ ExecResult execInContainer(Charset outputCharset, String... command)
@Deprecated
Info getDockerDaemonInfo();

String getContainerId();

String getContainerName();

/**
*
* @deprecated please use {@code org.testcontainers.DockerClientFactory.instance().client().inspectContainerCmd(container.getContainerId()).exec()}
*/
@Deprecated
InspectContainerResponse getContainerInfo();

void setExposedPorts(List<Integer> exposedPorts);

void setPortBindings(List<String> portBindings);
Expand Down
115 changes: 115 additions & 0 deletions core/src/main/java/org/testcontainers/containers/ContainerState.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
package org.testcontainers.containers;

import com.github.dockerjava.api.command.InspectContainerResponse;
import com.github.dockerjava.api.exception.DockerException;
import com.github.dockerjava.api.model.ExposedPort;
import com.github.dockerjava.api.model.PortBinding;
import com.github.dockerjava.api.model.Ports;
import org.testcontainers.DockerClientFactory;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

public interface ContainerState {

/**
* Get the IP address that this container may be reached on (may not be the local machine).
*
* @return an IP address
*/
default String getContainerIpAddress() {
return DockerClientFactory.instance().dockerHostIpAddress();
}

/**
* @return is the container currently running?
*/
default Boolean isRunning() {
try {
return getContainerId() != null && DockerClientFactory.instance().client().inspectContainerCmd(getContainerId()).exec().getState().getRunning();
} catch (DockerException e) {
return false;
}
}

/**
* Get the actual mapped port for a first port exposed by the container.
*
* @return the port that the exposed port is mapped to
* @throws IllegalStateException if there are no exposed ports
*/
default Integer getFirstMappedPort() {
return getExposedPorts()
.stream()
.findFirst()
.map(this::getMappedPort)
.orElseThrow(() -> new IllegalStateException("Container doesn't expose any ports"));
}

/**
* Get the actual mapped port for a given port exposed by the container.
*
* @param originalPort the original TCP port that is exposed
* @return the port that the exposed port is mapped to, or null if it is not exposed
*/
Integer getMappedPort(int originalPort);

/**
* @return the exposed ports
*/
List<Integer> getExposedPorts();

/**
* @return the container exposed port numbers mapped to ports exposed on the docker host
*/
default List<Integer> getExposedPortNumbers() {
return getExposedPorts().stream()
.map(this::getMappedPort)
.collect(Collectors.toList());
}

/**
* @return the port bindings
*/
default List<String> getPortBindings() {
List<String> portBindings = new ArrayList<>();
final Ports hostPortBindings = this.getContainerInfo().getHostConfig().getPortBindings();
for (Map.Entry<ExposedPort, Ports.Binding[]> binding : hostPortBindings.getBindings().entrySet()) {
for (Ports.Binding portBinding : binding.getValue()) {
portBindings.add(String.format("%s:%s", portBinding.toString(), binding.getKey()));
}
}
return portBindings;
}

/**
* @return the bound port numbers
*/
default List<Integer> getBoundPortNumbers() {
return getPortBindings().stream()
.map(PortBinding::parse)
.map(PortBinding::getBinding)
.map(Ports.Binding::getHostPortSpec)
.map(Integer::valueOf)
.collect(Collectors.toList());
}

/**
* @return the id of the container
*/
String getContainerId();

/**
* @return the name of the container
*/
default String getContainerName() {
return getContainerInfo().getName();
}

/**
* @return the container info
*/
InspectContainerResponse getContainerInfo();
}
Loading