[Bug]: Two custom containers can't connect on docker using testcontainers

[dbraun1991]

Module

Core

Testcontainers version

1.19.8

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host Arch

ARM

Docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

=== First ===
Java code from the testcontainers library behaves differently.

• MacOS on "MacBook Pro from 2023" with "Apple M2 Pro Silicon" fails
• Gitlab Pipeline with linux runner executes successfully

=== Second ===
The bookingEndpoint gets an IP that is not sufficient for connecting.
Neither of the following ...

• http://localhost:Port (obviously wrong)
• http://authEndpoint:Port (not rerouted by docker)
• http://HostIpAddress:Port (Given IP not leading to the authEndpoint)
  ... leads to a connection.
  Needed solution for:
  bookingEndpoint.withEnv("AUTH_URL", authEndpointBaseURL);

=== Third ===
Executing the JARs without Testcontainers locally -> works as a charm, just as in gitlab pipeline

    // Static definitions which are lifted before all tests

    // Define the NATS & MySql
    @ClassRule
    public static GenericContainer<?> natsContainer =
            new GenericContainer<>(
                    DockerImageName.parse("nats:alpine3.19"))
                    .withExposedPorts(4222) // Default MQTT port
                    .withNetwork(network)
                    .withNetworkAliases("natsmb")
                    .waitingFor(Wait.forListeningPort());

    @ClassRule
    public static MySQLContainer<?> mysqlContainer =
            new MySQLContainer<>("mysql:8.2")
                    .withDatabaseName("testdb")
                    .withUsername("test")
                    .withPassword("test")
                    // Adjust the service network
                    .withNetwork(network)
                    .withNetworkAliases("mysqldb");

    // Define the container for the AuthEndpoint using the built image
    @ClassRule
    public static GenericContainer<?> authEndpoint =
            new GenericContainer<>(authEndpointDockerImage)
                    .withExposedPorts(authEndpointPort)
                    // Adjust the service network
                    .withNetwork(network)
                    .withNetworkAliases("authEndpoint")

                    // Adjust the 'infrastructure'-connections
                    .withEnv("DB_URL", "jdbc:mysql://mysqldb:3306/testdb")
                    .withEnv("DB_USERNAME", "test")
                    .withEnv("DB_PASSWORD", "test")
                    .withEnv("NATS_URL", "nats://natsmb:4222")
                    .withEnv("INSTANCE_INDICATOR", "AUTH")
                    .withEnv("INITIAL_DELAY", "130000")
                    .withEnv("FIXED_INTERVAL", "12000000")

                    // Add a wait strategy for the AuthEndpoint, ensuring the mysqlContainer is up first
                    .dependsOn(mysqlContainer, mysqlContainer)
                    // AuthEndpoint is ready => actuator is open for connection:
                    .waitingFor(Wait.forHttp("/actuator").forStatusCode(200));

    // Define the container for BookingEndpoint using the built image
    @ClassRule
    public static GenericContainer<?> bookingEndpoint =
            new GenericContainer<>(bookingEndpointDockerImage)
                    .withExposedPorts(bookingEndpointPort)
                    // Adjust the service network
                    .withNetwork(network)
                    .withNetworkAliases("bookingEndpoint")

                    // Adjust the 'infrastructure'-connections
                    .withEnv("DB_URL", "jdbc:mysql://mysqldb:3306/testdb")
                    .withEnv("DB_USERNAME", "test")
                    .withEnv("DB_PASSWORD", "test")
                    .withEnv("NATS_URL", "nats://natsmb:4222")
                    .withEnv("INSTANCE_INDICATOR", "BOOKING")
                    .withEnv("AUTH_URL", "")
                    .withEnv("INITIAL_DELAY", "1000")
                    .withEnv("FIXED_INTERVAL", "5000")

                    // Add a wait strategy for the BookingEndpoint, ensuring the AuthEndpoint is up first
                    .dependsOn(authEndpoint)
                    // BookingEndpoint is ready => actuator is open for connection:
                    .waitingFor(Wait.forHttp("/actuator").forStatusCode(200));


    static {
        OffsetDateTime start = OffsetDateTime.now();
        logger.info(String.format("Setup start at: %s", start));

        logger.info("==============================================");
        logger.info("=====     Starting the message broker    =====");
        logger.info("==============================================");
        // Start the message broker
        natsContainer.start();
        // === Set system properties ===
        // --> MessageQueue
        String natsURL = String.format("nats://%s:%s",
                natsContainer.getHost(),
                natsContainer.getMappedPort(4222));
        System.setProperty("NATS_URL", natsURL);

        // Start the database
        logger.info("==============================================");
        logger.info("=====        Starting the database       =====");
        logger.info("==============================================");
        mysqlContainer.start();
        // === Set system properties ===
        // --> Database
        System.setProperty("DB_URL", mysqlContainer.getJdbcUrl());
        System.setProperty("DB_USERNAME", mysqlContainer.getUsername());
        System.setProperty("DB_PASSWORD", mysqlContainer.getPassword());

        // Start the AuthEndpoint
        logger.info("==============================================");
        logger.info("=====     Starting the Auth-Endpoint     =====");
        logger.info("==============================================");
        authEndpoint.start();
        authEndpointBaseURL =
                "http://" + getHostIp(authEndpoint) + ":" + authEndpoint.getMappedPort(authEndpointPort);
        // authEndpointBaseURL2 = "http://authEndpoint:" + authEndpoint.getMappedPort(authEndpointPort);

        bookingEndpoint.withEnv("AUTH_URL", authEndpointBaseURL);
        logger.info("AuthUrl given to BookingEndpoint definition via ENV:   " + authEndpointBaseURL);

        bookingEndpoint.start();

        // === Set general system properties ===
        System.setProperty("INSTANCE_INDICATOR", "VALIDATOR");

        Duration duration = Duration.between(start, OffsetDateTime.now());
        logger.info(String.format("Setup took %s seconds", duration.getSeconds()));

        // Known startup duration is ~30 seconds locally and ~60 seconds in pipeline.
        // It shall be known if Environment startup-time drastically changes.
        // assertTrue(duration.getSeconds() < 90);
    }

    private static String getHostIp(GenericContainer<?> authEndpoint) {

        if (authEndpoint.getHost().equals("localhost")) {
            try {
                // Get IP from Host-Machine 
                return InetAddress.getLocalHost().getHostAddress();
            } catch (UnknownHostException e) {
                throw new RuntimeException(e);
            }
        }
        return authEndpoint.getHost();
    }

Relevant log output

Custom Log Output:

• I/O error on GET request for "http://RecentIP:TempPort/v1/publickey": null
• I/O error on GET request for "http://RecentIP:TempPort/actuator": null

Additional Information

https://stackoverflow.com/questions/78352477/testcontainers-container-gethost-is-behaving-differently-locally-vs-gitlab-pi

[eddumelendez]

Hi, few notes:

• Don't use start() if already using JUnit 4 integration with @ClassRule
• why authEndpointBaseURL is calling getHostIp(...)? Instead, use AUTH_URL with value authEndpoint:<port>. You are already using a similar approach with the other containers, so, I don't understand why here it is different.

Looking at the code again, you probably don't want to mark bookingEndpoint with @ClassRule because it will start with an empty AUTH_URL. So, starting it manually would make sense to set the custom value. You can debug and see if the right env vars have been set correctly.

[dbraun1991]

Hi @eddumelendez :)

First off:
The code works like a charm in my gitlab pipeline. 🥇
Still it fails locally on my MAC.

Nevertheless, you are right. I am using JUnit4 & Junit5 aside, mixing them 'in wild manner'.
I'll take a look to fix that.

[dbraun1991]

There are 4 approaches:

First:

• authEndpoint.getIpAddress() is deprecated

Second:

• pass authEndpoint:<MappedPort>. This is - in my case not converted by docker => equally: leads to a not in use IP.

Third:

• authEndpoint.getHost() returns 'localhost'
• passing localhost:<MappedPort> makes the booking endpoint search for auth in its own container.

Fourth:

• Both services run in docker on the same host. Consequently ...

if (authEndpoint.getHost().equals("localhost")) {
            try {
                return InetAddress.getLocalHost().getHostAddress();

Works in the pipeline, but not locally.

---

Hopefully this explains my approach better :)