Skip to content

Commit

Permalink
conf: useDockerComposeV2 defaults to true (#412)
Browse files Browse the repository at this point in the history
* conf: useDockerComposeV2 defaults to true

* more version formats handled

* duration striped from cache key

* also error output read from containers captured
  • Loading branch information
augi authored Aug 9, 2023
1 parent 20b1700 commit 178316b
Show file tree
Hide file tree
Showing 10 changed files with 37 additions and 25 deletions.
18 changes: 10 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# gradle-docker-compose-plugin [![Build](https://github.com/avast/gradle-docker-compose-plugin/actions/workflows/build.yml/badge.svg)](https://github.com/avast/gradle-docker-compose-plugin/actions/workflows/build.yml) [![Version](https://badgen.net/maven/v/maven-central/com.avast.gradle/gradle-docker-compose-plugin/)](https://repo1.maven.org/maven2/com/avast/gradle/gradle-docker-compose-plugin/)

Simplifies usage of [Docker Compose](https://www.docker.com/docker-compose) for local development and integration testing in [Gradle](https://gradle.org/) environment.
Simplifies usage of [Docker Compose](https://docs.docker.com/compose/) for local development and integration testing in [Gradle](https://gradle.org/) environment.

`composeUp` task starts the application and waits till all containers become [healthy](https://docs.docker.com/engine/reference/builder/#healthcheck) and all exposed TCP ports are open (so till the application is ready). It reads assigned host and ports of particular containers and stores them into `dockerCompose.servicesInfos` property.

Expand Down Expand Up @@ -44,19 +44,21 @@ dockerCompose.isRequiredBy(test)
* Please note that in Docker Compose v2, the suffix contains `-` instead of `_`

## Why to use Docker Compose?
1. I want to be able to run my application on my computer, and it must work for my colleagues as well. Just execute `docker-compose up` and I'm done - e.g. the database is running.
2. I want to be able to test my application on my computer - I don't wanna wait till my application is deployed into dev/testing environment and acceptance/end2end tests get executed. I want to execute these tests on my computer - it means execute `docker-compose up` before these tests.
1. I want to be able to run my application on my computer, and it must work for my colleagues as well. Just execute `docker compose up` and I'm done - e.g. the database is running.
2. I want to be able to test my application on my computer - I don't wanna wait till my application is deployed into dev/testing environment and acceptance/end2end tests get executed. I want to execute these tests on my computer - it means execute `docker compose up` before these tests.

## Why this plugin?
You could easily ensure that `docker-compose up` is called before your tests but there are few gotchas that this plugin solves:
You could easily ensure that `docker compose up` is called before your tests but there are few gotchas that this plugin solves:

1. If you execute `docker-compose up -d` (_detached_) then this command returns immediately and your application is probably not able to serve requests at this time. This plugin waits till all containers become [healthy](https://docs.docker.com/engine/reference/builder/#healthcheck) and all exported TCP ports of all services are open.
1. If you execute `docker compose up -d` (_detached_) then this command returns immediately and your application is probably not able to serve requests at this time. This plugin waits till all containers become [healthy](https://docs.docker.com/engine/reference/builder/#healthcheck) and all exported TCP ports of all services are open.
- If waiting for healthy state or open TCP ports timeouts (default is 15 minutes) then it prints log of related service.
2. It's recommended not to assign fixed values of exposed ports in `docker-compose.yml` (i.e. `8888:80`) because it can cause ports collision on integration servers. If you don't assign a fixed value for exposed port (use just `80`) then the port is exposed as a random free port. This plugin reads assigned ports (and even IP addresses of containers) and stores them into `dockerCompose.servicesInfo` map.
3. There are minor differences when using Linux containers on Linux, Windows and Mac, and when using Windows Containers. This plugin handles these differences for you so you have the same experience in all environments.

# Usage
The plugin must be applied on project that contains `docker-compose.yml` file. It supposes that [Docker Engine](https://www.docker.com/docker-engine) and [Docker Compose](https://www.docker.com/docker-compose) are installed and available in `PATH`.
The plugin must be applied on project that contains `docker-compose.yml` file. It supposes that [Docker Engine](https://docs.docker.com/engine/) and [Docker Compose](https://docs.docker.com/compose/) are installed and available in `PATH`.

> Starting from plugin version _0.17.0_, _useDockerComposeV2_ property defaults to _true_, so the new `docker compose` (instead of deprecated `docker-compose` is used).
> Starting from plugin version _0.10.0_, Gradle 4.9 or newer is required (because it uses [Task Configuration Avoidance API](https://docs.gradle.org/current/userguide/task_configuration_avoidance.html)).
Expand Down Expand Up @@ -115,9 +117,9 @@ dockerCompose {
projectName = 'my-project' // allow to set custom docker-compose project name (defaults to a stable name derived from absolute path of the project and nested settings name), set to null to Docker Compose default (directory name)
projectNamePrefix = 'my_prefix_' // allow to set custom prefix of docker-compose project name, the final project name has nested configuration name appended
executable = '/path/to/docker-compose' // allow to set the path of the docker-compose executable (useful if not present in PATH). Not used if useDockerComposeV2 is set to true.
executable = '/path/to/docker-compose' // allow to set the base Docker Compose command (useful if not present in PATH). Defaults to `docker-compose`. Ignored if useDockerComposeV2 is set to true.
useDockerComposeV2 = true // Use Docker Compose V2 instead of Docker Compose V1, default is true. If set to true, `dockerExecutable compose` is used for execution, so executable property is ignored.
dockerExecutable = '/path/to/docker' // allow to set the path of the docker executable (useful if not present in PATH)
useDockerComposeV2 = true // Use Docker Compose V2 instead of Docker Compose V1. All invocations will be done using `docker compose` instead of `docker-compose`. Default is false.
dockerComposeWorkingDirectory = project.file('/path/where/docker-compose/is/invoked/from')
dockerComposeStopTimeout = java.time.Duration.ofSeconds(20) // time before docker-compose sends SIGTERM to the running containers after the composeDown task has been started
environment.put 'BACKEND_ADDRESS', '192.168.1.100' // environment variables to be used when calling 'docker-compose', e.g. for substitution in compose file
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ abstract class ComposeExecutor implements BuildService<Parameters>, AutoCloseabl
e.environment = System.getenv() + parameters.environment.get()

def finalArgs = []
finalArgs.addAll(getDockerComposeBinaryArgs())
finalArgs.addAll(getDockerComposeBaseCommand())
finalArgs.addAll(parameters.composeAdditionalArgs.get())
if (noAnsi) {
if (version >= VersionNumber.parse('1.28.0')) {
Expand Down Expand Up @@ -138,7 +138,7 @@ abstract class ComposeExecutor implements BuildService<Parameters>, AutoCloseabl
}

Map<String,Iterable<String>> getContainerIds(List<String> serviceNames) {
// `docker-compose ps -q serviceName` returns an exit code of 1 when the service
// `docker compose ps -q serviceName` returns an exit code of 1 when the service
// doesn't exist. To guard against this, check the service list first.
def services = execute('ps', '--services').readLines()
def result = [:]
Expand Down Expand Up @@ -277,15 +277,15 @@ abstract class ComposeExecutor implements BuildService<Parameters>, AutoCloseabl
boolean isScaleSupported() {
def v = version
if (v < VersionNumber.parse('1.13.0') && parameters.scale) {
throw new UnsupportedOperationException("docker-compose version $v doesn't support --scale option")
throw new UnsupportedOperationException("Docker Compose version $v doesn't support --scale option")
}
!parameters.scale.get().isEmpty()
}

// Determines whether to use docker-compose (V1) or docker compose (V2)
List<String> getDockerComposeBinaryArgs() {
List<String> getDockerComposeBaseCommand() {
parameters.useDockerComposeV2.get()
? [parameters.dockerExecutable.get(), "compose"]
: [parameters.executable.get()]
: Arrays.asList(parameters.executable.get().split("\\s+")) // split on spaces
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ abstract class ComposeSettings {
String nestedName

abstract Property<String> getExecutable()
abstract Property<String> getDockerExecutable()
abstract Property<Boolean> getUseDockerComposeV2()
abstract Property<String> getDockerExecutable()
abstract MapProperty<String, Object> getEnvironment()

abstract DirectoryProperty getDockerComposeWorkingDirectory()
Expand Down Expand Up @@ -147,8 +147,8 @@ abstract class ComposeSettings {
captureContainersOutput.set(false)

executable.set('docker-compose')
useDockerComposeV2.set(true)
dockerExecutable.set('docker')
useDockerComposeV2.set(false)

dockerComposeStopTimeout.set(Duration.ofSeconds(10))

Expand Down Expand Up @@ -204,8 +204,8 @@ abstract class ComposeSettings {
r.projectNamePrefix = this.projectNamePrefix

r.executable.set(this.executable.get())
r.dockerExecutable.set(this.dockerExecutable.get())
r.useDockerComposeV2.set(this.useDockerComposeV2.get())
r.dockerExecutable.set(this.dockerExecutable.get())
r.environment.set(new HashMap<String, Object>(this.environment.get()))

r.dockerComposeWorkingDirectory.set(this.dockerComposeWorkingDirectory.getOrNull())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ class DockerExecutor {
def finalArgs = [settings.dockerExecutable.get()]
finalArgs.addAll(args)
e.commandLine finalArgs
e.errorOutput = os
e.standardOutput = os
e.ignoreExitValue = true
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ abstract class ServiceInfoCache implements BuildService<Parameters> {
if (cachedState == currentState) {
return deserialized.collectEntries { k, v -> [k, deserializeServiceInfo(v)] }
} else {
logger.lifecycle("Current and cached states differs, cannot use the cached service infos.")
logger.lifecycle("Current and cached states are different, cannot use the cached service infos.")
logger.info("Cached state:\n$cachedState\nCurrent state:\n$currentState")
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import org.gradle.api.provider.MapProperty
import org.gradle.api.provider.Property
import org.gradle.api.tasks.Internal
import org.gradle.api.tasks.TaskAction
import org.yaml.snakeyaml.Yaml

import java.time.Duration
import java.time.Instant
Expand Down Expand Up @@ -210,7 +211,14 @@ abstract class ComposeUp extends DefaultTask {

@Internal
protected def getStateForCache() {
composeExecutor.get().execute('ps') + composeExecutor.get().execute('config') + startedServices.get().join(',')
String processesAsString = composeExecutor.get().execute('ps', '--format', 'json')
// Status field contains something like "Up 8 seconds", so we have to strip the duration.
Object[] processes = new Yaml().load(processesAsString)
List<Object> transformed = processes.collect {
if (it.Status.startsWith('Up ')) it.Status = 'Up'
it
}
transformed.join('\t') + composeExecutor.get().execute('config') + startedServices.get().join(',')
}

protected Iterable<ServiceInfo> loadServicesInfo(Iterable<String> servicesNames) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ boolean hasDigit() {
}

boolean hasSpecifierSeparator() {
return pos < str.length() && str.charAt(pos) == '-';
return pos < str.length() && (str.charAt(pos) == '-' || str.charAt(pos) == '+');
}

boolean isSeparatorAndDigit() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ class ComposeExecutorTest extends Specification {
}

@Unroll
def "getDockerComposeBinaryArgs returns correct values when useDockerComposeV2 is #useDockerComposeV2" () {
def "getDockerComposeBaseCommand returns correct values when useDockerComposeV2 is #useDockerComposeV2" () {
def f = Fixture.withHelloWorld()
f.project.plugins.apply 'java'

Expand All @@ -64,7 +64,7 @@ class ComposeExecutorTest extends Specification {
f.project.plugins.apply 'docker-compose'

when:
def actual = ComposeExecutor.getInstance(f.project, f.project.dockerCompose).get().getDockerComposeBinaryArgs()
def actual = ComposeExecutor.getInstance(f.project, f.project.dockerCompose).get().getDockerComposeBaseCommand()

then:
expectedDockerComposeBinaryArgs.size() == actual.size()
Expand All @@ -77,6 +77,6 @@ class ComposeExecutorTest extends Specification {
useDockerComposeV2 | expectedDockerComposeBinaryArgs
true | ["docker", "compose"]
false | ["docker-compose"]
null | ["docker-compose"]
null | ["docker", "compose"]
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,14 @@ class DockerExecutorTest extends Specification {
}

def "reads container logs"() {
def f = Fixture.withHelloWorld()
def f = Fixture.withNginx()
f.project.tasks.composeBuild.build()
f.project.tasks.composeUp.up()
String containerId = f.extension.servicesInfos.hello.firstContainer.containerId
String containerId = f.extension.servicesInfos.web.firstContainer.containerId
when:
String output = f.extension.dockerExecutor.getContainerLogs(containerId)
then:
output.contains('Hello from Docker')
output.contains('nginx')
cleanup:
f.project.tasks.composeDown.down()
f.close()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ class VersionNumberTest extends Specification {
"1.1.0" | "1.2.0" | -1
"1.28.0" | "1.16.0" | 1
"2.20.2-desktop.1" | "2.20.2" | 0
"2.20.2+azure-1" | "2.20.2" | 0
}

def "handles non parseable versions as UNKNOWN"() {
Expand Down

0 comments on commit 178316b

Please sign in to comment.