[Bug]: Ryuk container created, then it is removed before it starts

[RiverPhillips]

Testcontainers version

0.31.0

Using the latest Testcontainers version?

Yes

Host OS

Mac OS

Host arch

ARM

Go version

1.22

Docker version

Client: Docker Engine - Community
 Version:           26.1.4
 API version:       1.45
 Go version:        go1.22.4
 Git commit:        5650f9b102
 Built:             Wed Jun  5 10:47:13 2024
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:47 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client: Docker Engine - Community
 Version:    26.1.4
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.15.0
    Path:     /Users/river/.docker/cli-plugins/docker-buildx
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-31-generic
 Operating System: Ubuntu 24.04 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 1.91GiB
 Name: colima
 ID: 5a6217f8-7cca-498b-91f6-d4070390a0e4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

Any attempt to run a test using test containers appears to create the Ryuk Container, return an ID. Then it is removed while waiting for the container to start.

Relevant log output

024/06/18 16:26:45 🐳 Creating container for image testcontainers/ryuk:0.7.0
2024/06/18 16:26:45 ✅ Container created: 4879564c4f90
2024/06/18 16:26:45 🐳 Starting container: 4879564c4f90
2024/06/18 16:26:45 ✅ Container started: 4879564c4f90
2024/06/18 16:26:45 ⏳ Waiting for container id 4879564c4f90 image: testcontainers/ryuk:0.7.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
2024/06/18 16:26:45 failed accessing container logs: Error response from daemon: No such container: 4879564c4f9093fffa6b43a4a75d48e63a71672bc391fd3ce69951bb13152643

Additional information

I attempted to run test tests with main and observed the same failures so I don't believe this fixes it. #2477

[ash2k]

Experiencing the same issue. macOS 14.5 Sonoma, Rancher Desktop 1.14.2. Testcontainers 36573d0 (a few commits after 0.31.0).

Client: Docker Engine - Community
 Version:    26.1.4
 Context:    rancher-desktop
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.1
    Path:     /Users/mike/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.1
    Path:     /Users/mike/.docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 25.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 4e1fe7492b9df85914c389d1f15a3ceedbb280ac
 runc version: 51d5e94601ceffbbd85688df1c928ecccbfa4685
 init version:
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.30-0-virt
 Operating System: Alpine Linux v3.19
 OSType: linux
 Architecture: aarch64
 CPUs: 9
 Total Memory: 15.6GiB
 Name: lima-rancher-desktop
 ID: d04c0365-7614-447f-960d-353ba1523b4f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

It worked yesterday, not sure what I did to break it.

[ash2k]

Set TESTCONTAINERS_RYUK_DISABLED to true and things work again.

[RiverPhillips]

Disabling Ryuk isn't really a solution, we'd like to continue to use it.

[mdelapenya]

Hi 👋 are you by the way using the compose module? If so please take a look at #2563 (comment)

If not, I'd like to see the ryuk logs if possible, so we can determine why it's being removed. Is it a timeout issue? You can check the ryuk configuration here: https://golang.testcontainers.org/features/configuration/#customizing-ryuk-the-resource-reaper

[RiverPhillips]

Hi @mdelapenya we're not using compose.

Ryuk doesn't appear to log anything in this case. Upon further inspection when I run go test from the command line it does work, but running it from VS code's test runner, or Goland's test runner it fails.

[mdelapenya]

Mmm that's even weirder. Can you verify that the IDE sets the proper go settings for testing, such as "go.testTimeout" (I configured it to 600s)? I wonder why the ryuk container dies in VSCode but not in your terminal.

Can you provide a repro repository?

[RiverPhillips]

Mmm that's even weirder. Can you verify that the IDE sets the proper go settings for testing, such as "go.testTimeout" (I configured it to 600s)? I wonder why the ryuk container dies in VSCode but not in your terminal.

The test timeouts look good, it fails almost instantly, it's not taking a long time.

Can you provide a repro repository?

I have this issue for any project using testcontainers

[stevenh]

This should be fixed by testcontainers/moby-ryuk#121 but we need a release and then testcontainers-go updated to use the new release.

Try cloning the moby-ryuk repo and running the following in it to replace the image that testcontainers-go uses to see if it does fix:

docker build -f linux/Dockerfile -t testcontainers/ryuk:0.7.0 .