add property to start ryuk in privileged mode #894

atomfrede · 2018-10-02T18:54:35Z

I would like to add some documentation but the ryuk customizations are not documented at all, so not sure where to add id (as we had the a similar problem a few weeks ago).

closes #871

closes testcontainers#871

kiview

LGTM, you can add docs in docs/usage/properties.md, but we can do it of course in another PR and try to cover all the Ryuk properties.

kiview · 2018-10-03T08:44:07Z

Have you manually verified that it works? We are simply calling a method from docker-java, so I suppose yes of course.

atomfrede · 2018-10-03T12:02:10Z

A few weeks ago we had a problem with one of our jenkins agents and it look like a se linux problem and non privileged mode (it wasn't) and we changed the code and hardcoded the privileged flag, so setting the privileged flag works with docker-java.

kiview · 2018-10-04T08:50:20Z

Just retriggered all the builds, seems there was a problem with Gradle plugin resolution :(

atomfrede · 2018-10-04T08:59:44Z

Thx.

…

On Thu, Oct 4, 2018, 10:50 Kevin Wittek ***@***.***> wrote: Just retriggered all the builds, seems there was a problem with Gradle plugin resolution :( — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#894 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMaiQqNrBFiauQn_BoeHY5mR0sE0FNBks5uhcvUgaJpZM4XEqAo> .

kiview · 2018-10-04T09:05:18Z

Let's wait if this solves the problem for @kamkie in #871 and then we'll merge 🙂

kamkie · 2018-10-04T09:30:42Z

@kiview is there maven snapshot repository with that or should build this branch from sources?

bsideup · 2018-10-04T09:31:13Z

@kamkie see "Jitpack" section here:
https://www.testcontainers.org/usage.html#maven-dependencies

kiview · 2018-10-04T09:51:28Z

This version should work:

	<dependency>
	    <groupId>com.github.testcontainers</groupId>
	    <artifactId>testcontainers-java</artifactId>
	    <version>5bac09b</version>
	</dependency>

bsideup · 2018-10-04T09:59:50Z

@kiview

	<dependency>
	    <groupId>com.github.testcontainers.testcontainers-java</groupId>
	    <artifactId>testcontainers</artifactId>
	    <version>5bac09b</version>
	</dependency>

kamkie · 2018-10-04T10:00:25Z

maybe it shoiuld be https://github.com/atomfrede/testcontainers-java/commits/871-ryuk-as-priviliged

Downloading: https://dl.bintray.com/testcontainers/releases/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.pom
Downloading: https://jitpack.io/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.pom
Downloading: https://repo.maven.apache.org/maven2/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.pom
[WARNING] The POM for com.github.testcontainers:testcontainers-java:jar:5bac09b is missing, no dependency information available
Downloading: https://dl.bintray.com/testcontainers/releases/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.jar
Downloading: https://jitpack.io/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.jar
Downloading: https://repo.maven.apache.org/maven2/com/github/testcontainers/testcontainers-java/5bac09b/testcontainers-java-5bac09b.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

kiview · 2018-10-04T10:00:27Z

Whoops, copied from Jitpack 😓

kamkie · 2018-10-04T10:27:08Z

using

         <dependency>
            <groupId>com.github.atomfrede.testcontainers-java</groupId>
            <artifactId>postgresql</artifactId>
            <version>871-ryuk-as-priviliged-SNAPSHOT</version>
        </dependency>

it runs fine on windows but hangs on openshift

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
10:23:55.056 [main] WARN  org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config (dockerImageName: alpine:3.5, configFile: /root/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /root/.docker/config.json (No such file or directory)

my hack was working on version 1.9.0-rc1

String ryukContainerId = client.createContainerCmd(ryukImage).withHostConfig(new HostConfig() {
                @JsonProperty("AutoRemove")
                boolean autoRemove = true;
            }).withExposedPorts(new ExposedPort(8080)).withPublishAllPorts(true).withName("testcontainers-ryuk-" + DockerClientFactory.SESSION_ID).withLabels(Collections.singletonMap(DockerClientFactory.TESTCONTAINERS_LABEL, "true")).withBinds(binds).withPrivileged(true).exec().getId();

kiview · 2018-10-04T10:33:09Z

Thanks for testing, did you set the testcontainers.properties value?
It is non-privileged by default.
See https://github.com/testcontainers/testcontainers-java/pull/894/files#diff-3138625d72a2414ae0b3ee588d93ceca

kamkie · 2018-10-04T10:34:28Z

I assumed that java system property will work System.setProperty("ryuk.container.privileged", "true");

kamkie · 2018-10-04T10:38:54Z

no difference using testcontainers.properties file

kiview · 2018-10-04T10:42:10Z

Can you share your pom.xml please?
My best guess is something messed up with Jitpack.

kamkie · 2018-10-04T10:49:18Z

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>pl.com.example</groupId>
    <artifactId>example-backend</artifactId>
    <version>1.0.0-SNAPSHOT</version>


    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.4.RELEASE</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <java.version>1.8</java.version>

        <liquibase.db.url>jdbc:postgresql://localhost:5432/db</liquibase.db.url>

        <postgresql.version>42.2.2</postgresql.version>
        <feign.version>10.0.1</feign.version>
        <hibernate-search.version>5.9.2.Final</hibernate-search.version>
        <lombok.version>1.18.2</lombok.version>
        <junit.jupiter.version>5.3.0</junit.jupiter.version>
        <mockito.version>2.21.0</mockito.version>
        <liquibase.version>3.6.2</liquibase.version>
    </properties>

    <prerequisites>
        <maven>3.3.9</maven>
    </prerequisites>

    <dependencies>
        <!-- https://mvnrepository.com/artifact/org.hibernate/hibernate-search-engine -->
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-search-engine</artifactId>
            <version>${hibernate-search.version}</version>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-search-orm</artifactId>
            <version>${hibernate-search.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.8</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.liquibase</groupId>
            <artifactId>liquibase-core</artifactId>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-ldap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-ldap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-java8</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.directory.api</groupId>
            <artifactId>api-all</artifactId>
            <version>2.0.0.AM2</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>

        <dependency>
            <groupId>io.github.openfeign</groupId>
            <artifactId>feign-okhttp</artifactId>
            <version>${feign.version}</version>
        </dependency>
        <dependency>
            <groupId>io.github.openfeign</groupId>
            <artifactId>feign-gson</artifactId>
            <version>${feign.version}</version>
        </dependency>
        <dependency>
            <groupId>io.github.openfeign</groupId>
            <artifactId>feign-jackson</artifactId>
            <version>${feign.version}</version>
        </dependency>
        <dependency>
            <groupId>io.github.openfeign</groupId>
            <artifactId>feign-slf4j</artifactId>
            <version>${feign.version}</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-envers</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-mail</artifactId>
        </dependency>

        <dependency>
            <groupId>net.coobird</groupId>
            <artifactId>thumbnailator</artifactId>
            <version>0.4.8</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-parameter-names</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-jdk8</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-jsr310</artifactId>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-hibernate5</artifactId>
        </dependency>
        <dependency>
            <groupId>org.docx4j</groupId>
            <artifactId>docx4j</artifactId>
            <version>6.0.1</version>
            <exclusions>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>2.4.0-b180830.0359</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.j256.simplemagic/simplemagic -->
        <dependency>
            <groupId>com.j256.simplemagic</groupId>
            <artifactId>simplemagic</artifactId>
            <version>1.14</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <!-- https://mvnrepository.com/artifact/net.sf.supercsv/super-csv -->
        <dependency>
            <groupId>net.sf.supercsv</groupId>
            <artifactId>super-csv</artifactId>
            <version>2.4.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.github.ben-manes.caffeine/caffeine -->
        <dependency>
            <groupId>com.github.ben-manes.caffeine</groupId>
            <artifactId>caffeine</artifactId>
        </dependency>

        <!-- TESTS -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.unboundid</groupId>
            <artifactId>unboundid-ldapsdk</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.icegreen</groupId>
            <artifactId>greenmail</artifactId>
            <version>1.5.8</version>
            <scope>test</scope>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.testcontainers/postgresql -->
        <dependency>
            <groupId>com.github.atomfrede.testcontainers-java</groupId>
            <artifactId>postgresql</artifactId>
            <version>871-ryuk-as-priviliged-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.mockito/mockito-junit-jupiter -->
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-junit-jupiter</artifactId>
            <version>${mockito.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>build-info</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>pl.project13.maven</groupId>
                <artifactId>git-commit-id-plugin</artifactId>
                <version>2.2.5</version>
                <configuration>
                    <gitDescribe>
                        <tags>true</tags>
                        <forceLongFormat>true</forceLongFormat>
                    </gitDescribe>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <debug>true</debug>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.liquibase</groupId>
                <artifactId>liquibase-maven-plugin</artifactId>
                <version>${liquibase.version}</version>
                <configuration>
                    <url>${liquibase.db.url}</url>
                    <driver>org.postgresql.Driver</driver>
                    <username>dev</username>
                    <password>dev</password>
                    <changeLogFile>db/changelog/db-changelog.xml</changeLogFile>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>analyze</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>findbugs-maven-plugin</artifactId>
                        <version>3.0.5</version>
                        <configuration>
                            <effort>Max</effort>
                            <threshold>Low</threshold>
                            <xmlOutput>true</xmlOutput>
                            <findbugsXmlOutputDirectory>${project.build.directory}/findbugs</findbugsXmlOutputDirectory>
                            <fork>false</fork>
                        </configuration>
                        <executions>
                            <execution>
                                <phase>verify</phase>
                                <goals>
                                    <goal>check</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>

                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-pmd-plugin</artifactId>
                        <version>3.10.0</version>
                        <configuration>
                            <targetJdk>${java.version}</targetJdk>
                            <minimumTokens>20</minimumTokens>
                            <failOnViolation>true</failOnViolation>
                            <verbose>true</verbose>
                        </configuration>
                        <executions>
                            <execution>
                                <phase>verify</phase>
                                <goals>
                                    <goal>check</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>

                    <plugin>
                        <groupId>org.jacoco</groupId>
                        <artifactId>jacoco-maven-plugin</artifactId>
                        <version>0.8.2</version>
                        <executions>
                            <execution>
                                <id>default-prepare-agent</id>
                                <goals>
                                    <goal>prepare-agent</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>default-report</id>
                                <phase>prepare-package</phase>
                                <goals>
                                    <goal>report</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>default-check</id>
                                <goals>
                                    <goal>check</goal>
                                </goals>
                                <configuration>
     
                                    <rules>
                                        <rule>
                                            <element>BUNDLE</element>
                                            <limits>
                                                <limit>
                                                    <counter>INSTRUCTION</counter>
                                                    <value>COVEREDRATIO</value>
                                                    <minimum>0.6</minimum>
                                                </limit>
                                            </limits>
                                        </rule>
                                    </rules>
                                    <haltOnFailure>true</haltOnFailure>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <repositories>
        <repository>
            <id>bintray-testcontainers-maven</id>
            <url>https://dl.bintray.com/testcontainers/releases</url>
        </repository>
        <repository>
            <id>jitpack.io</id>
            <url>https://jitpack.io</url>
        </repository>
    </repositories>
</project>

kiview · 2018-10-04T10:52:26Z

Looks fine I think.

@bsideup
Will it works as expected, if this Jitpack is used (postgresql module)

        <dependency>
            <groupId>com.github.atomfrede.testcontainers-java</groupId>
            <artifactId>postgresql</artifactId>
            <version>871-ryuk-as-priviliged-SNAPSHOT</version>
        </dependency>

but the change was in core?

kamkie · 2018-10-04T10:57:23Z

] +- com.github.atomfrede.testcontainers-java:postgresql:jar:871-ryuk-as-priviliged-1.9.1-g5bac09b-6:compile
[INFO] |  \- com.github.atomfrede.testcontainers-java:jdbc:jar:871-ryuk-as-priviliged-1.9.1-g5bac09b-6:compile
[INFO] |     \- com.github.atomfrede.testcontainers-java:database-commons:jar:871-ryuk-as-priviliged-1.9.1-g5bac09b-6:compile
[INFO] |        \- com.github.atomfrede.testcontainers-java:testcontainers:jar:871-ryuk-as-priviliged-1.9.1-g5bac09b-6:compile

this also looks ok

kamkie · 2018-10-08T08:59:06Z

@kiview there is something wrong
no breakpoint in ResourceReaper catches
but breakpoints in PostgresContainer are working ok

kamkie · 2018-10-10T09:37:32Z

@atomfrede I can confirm that this works for me
I had problem with that test-containers was looking in wrong place for config.json

kiview · 2018-10-10T09:51:42Z

Awesome, l'll merge then.

Closes #871

add property to start ryuk in privileged mode

e70f6f4

closes testcontainers#871

atomfrede requested review from bsideup, kiview and rnorth as code owners October 2, 2018 18:54

atomfrede mentioned this pull request Oct 2, 2018

add missing argument to log statement #895

Merged

kiview mentioned this pull request Oct 3, 2018

ability to run ResourceReaper as privileged container #871

Closed

kiview approved these changes Oct 3, 2018

View reviewed changes

add documentation about ryuk image and new privileged property

5bac09b

rnorth mentioned this pull request Oct 4, 2018

Gradle plugin resolution fails intermittently during CI builds #898

Closed

kiview merged commit 16326e2 into testcontainers:master Oct 10, 2018

kiview added this to the next milestone Oct 10, 2018

dbyron0 referenced this pull request in locationlabs/testcontainers-java Oct 10, 2018

Add property to start ryuk in privileged mode (#894)

5d9d249

Closes #871

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add property to start ryuk in privileged mode #894

add property to start ryuk in privileged mode #894

atomfrede commented Oct 2, 2018

kiview left a comment

kiview commented Oct 3, 2018

atomfrede commented Oct 3, 2018

kiview commented Oct 4, 2018

atomfrede commented Oct 4, 2018 via email

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

bsideup commented Oct 4, 2018

kiview commented Oct 4, 2018

bsideup commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kamkie commented Oct 8, 2018

kamkie commented Oct 10, 2018

kiview commented Oct 10, 2018

add property to start ryuk in privileged mode #894

add property to start ryuk in privileged mode #894

Conversation

atomfrede commented Oct 2, 2018

kiview left a comment

Choose a reason for hiding this comment

kiview commented Oct 3, 2018

atomfrede commented Oct 3, 2018

kiview commented Oct 4, 2018

atomfrede commented Oct 4, 2018 via email

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

bsideup commented Oct 4, 2018

kiview commented Oct 4, 2018

bsideup commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kiview commented Oct 4, 2018

kamkie commented Oct 4, 2018

kamkie commented Oct 8, 2018

kamkie commented Oct 10, 2018

kiview commented Oct 10, 2018