Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: deadlock in container.Host() #2897

Closed
vikstrous2 opened this issue Nov 28, 2024 · 0 comments · Fixed by #2900
Closed

[Bug]: deadlock in container.Host() #2897

vikstrous2 opened this issue Nov 28, 2024 · 0 comments · Fixed by #2900
Labels
bug An issue with the library

Comments

@vikstrous2
Copy link

Testcontainers version

v0.34.1-0.20241126184225-db433916fa6a

Using the latest Testcontainers version?

Yes

Host OS

linux

Host arch

x86

Go version

go version go1.23.3 linux/amd64

Docker version

Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:08:17 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.26
  GitCommit:        3dd1e886e55dd695541fdcd67420c2888645a495
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx

Server:
 Containers: 21
  Running: 20
  Paused: 0
  Stopped: 1
 Images: 396
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.112+
 Operating System: Debian GNU/Linux 11 (bullseye) (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.35GiB
 Name: viktor-main
 ID: 74745a36-d417-4e03-89f2-52e442230a26
 Docker Root Dir: /home/.docker_data
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

deadlock calling container.Host() from a test running in a container

Relevant log output

Stacktrace:

goroutine 7 [sync.Mutex.Lock]:
sync.runtime_SemacquireMutex(0x412e98?, 0x68?, 0x1?)
        /usr/local/go/src/runtime/sema.go:95 +0x25
sync.(*Mutex).lockSlow(0xc00011a410)
        /usr/local/go/src/sync/mutex.go:173 +0x15d
sync.(*Mutex).Lock(...)
        /usr/local/go/src/sync/mutex.go:92
github.com/testcontainers/testcontainers-go.(*DockerProvider).ensureDefaultNetwork(0xc00011a370, {0x9dbaa0, 0xc000142fc0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1625 +0x8a
github.com/testcontainers/testcontainers-go.(*DockerProvider).GetGatewayIP(0xc00011a370, {0x9dbaa0, 0xc000142fc0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1597 +0x3a
github.com/testcontainers/testcontainers-go.(*DockerProvider).daemonHostLocked(0xc00011a370, {0x9dbaa0, 0xc000142fc0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1501 +0x205
github.com/testcontainers/testcontainers-go.(*DockerProvider).DaemonHost(0xcb1a20?, {0x9dbaa0?, 0xc000142fc0?})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1475 +0xc5
github.com/testcontainers/testcontainers-go.(*DockerContainer).Host(0x9db9f8?, {0x9dbaa0?, 0xc000142fc0?})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:165 +0x25
github.com/testcontainers/testcontainers-go/wait.(*HostPortStrategy).WaitUntilReady(0xc00039ecc0, {0x9db9f8?, 0xc00011d3e0?}, {0x9ddad8, 0xc00017a0e0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/wait/host_port.go:107 +0xbe
github.com/testcontainers/testcontainers-go.init.func3.2({0x9db9f8, 0xc00011d3e0}, {0x9e0230, 0xc00017a0e0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/lifecycle.go:289 +0x1a9
github.com/testcontainers/testcontainers-go.(*DockerContainer).applyLifecycleHooks.containerHookFn.func1({0x9e0230, 0xc00017a0e0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/lifecycle.go:472 +0xd6
github.com/testcontainers/testcontainers-go.(*DockerContainer).applyLifecycleHooks(0xc00017a0e0, {0x9db9f8, 0xc00011d3e0}, 0x1, 0x9587b8)
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/lifecycle.go:413 +0x1fe
github.com/testcontainers/testcontainers-go.(*DockerContainer).startedHook(...)
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/lifecycle.go:351
github.com/testcontainers/testcontainers-go.(*DockerContainer).Start(0xc00017a0e0, {0x9db9f8, 0xc00011d3e0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:237 +0x189
github.com/testcontainers/testcontainers-go.(*DockerProvider).RunContainer(_, {_, _}, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...}, ...})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1448 +0x95
github.com/testcontainers/testcontainers-go.(*reaperSpawner).newReaper(0xcb01d0, {0x9db9f8, 0xc00011d3e0}, {0xc0001e4f00, 0x40}, {0x9d96f8, 0xc00011a370})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/reaper.go:419 +0x74b
github.com/testcontainers/testcontainers-go.(*reaperSpawner).reuseOrCreate(0xcb01d0, {0x9db9f8, 0xc00011d3e0}, {0xc0001e4f00, 0x40}, {0x9d96f8, 0xc00011a370})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/reaper.go:329 +0x16c
github.com/testcontainers/testcontainers-go.(*reaperSpawner).reaper.(*reaperSpawner).retryLocked.func1()
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/reaper.go:281 +0x68
github.com/cenkalti/backoff/v4.doRetryNotify[...](0xc000414ef0?, {0x7c8025b4a0d8, 0xc00013c5a0}, 0x0, {0x0, 0x0?})
        /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:88 +0x12b
github.com/cenkalti/backoff/v4.RetryNotifyWithData[...](...)
        /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:54
github.com/cenkalti/backoff/v4.RetryWithData[...](0x899380?, {0x7c8025b4a0d8?, 0xc00013c5a0?})
        /go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:43 +0x2b
github.com/testcontainers/testcontainers-go.(*reaperSpawner).reaper(0xcb01d0, {0x9db9f8, 0xc00011d3e0}, {0xc0001e4f00, 0x40}, {0x9d96f8, 0xc00011a370})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/reaper.go:270 +0x2fe
github.com/testcontainers/testcontainers-go.(*DockerContainer).connectReaper(0xc00017a000, {0x9db950, 0xcd31e0})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:898 +0x118
github.com/testcontainers/testcontainers-go.(*DockerProvider).CreateContainer(_, {_, _}, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...}, ...})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/docker.go:1250 +0x18ec
github.com/testcontainers/testcontainers-go.GenericContainer({_, _}, {{{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...})
        /go/pkg/mod/github.com/testcontainers/testcontainers-go@v0.34.1-0.20241126184225-db433916fa6a/generic.go:74 +0x308
<redacted>

Additional information

package settlementnetwork

import (
	"context"
	"testing"

	"github.com/testcontainers/testcontainers-go"
)

func TestBlah(t *testing.T) {
	req := testcontainers.ContainerRequest{
		Name:  "example",
		Image: "alpine",
	}
	container, err := testcontainers.GenericContainer(context.Background(), testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started:          true,
	})
	if err != nil {
		t.Fatal(err)
	}
	container.Host(context.Background())
}

To repro, run the test in a container:

docker run --rm -it -w $(pwd) -v $(pwd):$(pwd) -v /var/run:/var/run golang
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug An issue with the library
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: avoid double lock in DockerProvider.DaemonHost() vikstrous/testcontainers-go
1 participant
@vikstrous2