[Enhancement]: Dockerfile with inline scripts

[Steven-Harris]

Testcontainers version

3.9.0

Using the latest Testcontainers version?

Yes

Host OS

Windows 11

Host arch

x86

.NET version

8.0.108

Docker version

Client:
 Version:           27.2.0
 API version:       1.45 (downgraded from 1.47)
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:17:17 2024
 OS/Arch:           windows/amd64
 Context:           default

Server:
 Engine:
  Version:          26.1.3
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.22.5
  Git commit:       8e96db1c328d0467b015768e42a62c0f834970bb
  Built:            Sun Jul  7 17:34:20 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.17
  GitCommit:        3a4de459a68952ffb703bbe7f2290861a75b6b67
 runc:
  Version:          1.1.12
  GitCommit:        51d5e94601ceffbbd85688df1c928ecccbfa4685
 docker-init:
  Version:          0.19.0
  GitCommit:

Docker info

Client:
 Version:    27.2.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     C:\Users\<redacted>\.docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2
    Path:     C:\Users\<redacted>\.docker\cli-plugins\docker-compose.exe

Server:
 Containers: 6
  Running: 1
  Paused: 0
  Stopped: 5
 Images: 52
 Server Version: 26.1.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3a4de459a68952ffb703bbe7f2290861a75b6b67
 runc version: 51d5e94601ceffbbd85688df1c928ecccbfa4685
 init version:
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Rancher Desktop WSL Distribution
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 15.47GiB
 Name: G9WG5S3
 ID: aa23d5ea-e978-4661-a673-02dd7790e5e8
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

Dockerfiles with inline bash scripts don't build properly. I'll be changing line 7 of the Dockerfile for these examples in the repo listed below. For all these instances, I verified that docker-compose could spin up the Dockerfile before trying it with TestContainers.

If the docker file contains COPY <<-"EOF" start.sh, testcontainers is throwing COPY failed: no source files were specified.

I've tried converting the COPY command into a run command, which gets me a bit further. The image builds, but it never starts. Using RUN cat <<"EOF" > start.sh yields, System.TimeoutException: The operation has timed out.

I can work around the issue by moving the inline script into its own bash file and pulling it into the Dockerfile instead.

Relevant log output

COPY failed: no source files were specified

OR 

System.TimeoutException: The operation has timed out

Additional information

https://github.com/Steven-Harris/testcontainers-inline-script-issue

[HofmeisterAn]

I don't think we can support this. AFAIK Heredoc is a BuildKit feature that is not available (or only partially available) via the Docker Engine API - sorry. Instead, you can use Testcontainer's WithResourceMapping or WithStartupCallback feature (or build it with the CLI and pass the image name to your test). Does this issue belong to: #1244 too?

[Steven-Harris]

Ok, thanks @HofmeisterAn. Although, shouldn't it be working when using a simple cat command? e.g. RUN cat <<"EOF" > start.sh?

I've been monitoring #1244 and looked in the Wiremock.net repo. Their dockerfile doesn't look anything like mine. Although maybe there is a modification being made that I can't see. It's hard to tell, since they didn't provide the Dockerfile in the issue.

[HofmeisterAn]

I am not sure, but I do not think it will work. If you disable BuildKit using $env:DOCKER_BUILDKIT=0, it will not work with the CLI either. You can find more information about the Heredoc syntax and its support here. If you would like to use Testcontainers' build capabilities, I believe the only options available are the two approaches mentioned above.

[DanielHabenicht]

I want to add my voice to this request. If a dockerfile contains the following:

RUN cat <<EOF > /usr/config/entrypoint.sh
#!/bin/bash

# some setup logic....

/opt/mssql/bin/sqlservr
EOF

RUN chmod +x /usr/config/entrypoint.sh

ENTRYPOINT [ "/usr/config/entrypoint.sh" ]

It will build without issues (even with testcontainers) but on start of the container it fails with exec /usr/config/entrypoint.sh: exec format error as entrypoint.sh is empty.

So maybe adding a warning or error early in order to not fail with a unrelated error lates would be good.

[HofmeisterAn]

As long as the Docker Engine API does not support BuildKit, there is nothing we can do (at least not without support from Docker). Probably the best we can do is to forward the feature request to extend the Docker Engine API to Docker.