WITH DOCKER using compose gives "Project name not found"

[lalithsuresh]

What went wrong?

A previously working use of WITH DOCKER with --compose in our Earthfile has suddenly begun failing in our CI runs with a project name must not be empty error:

> earthly --verbose +repro
            frontend | Running command: docker info
            frontend | Running command: docker info --format={{.SecurityOptions}}
            frontend | Running command: docker info --format={{.DockerRootDir}}
            frontend | docker-shell frontend initialized.
 Init 🚀
————————————————————————————————————————————————————————————————————————————————

            frontend | Running command: docker container inspect earthly-buildkitd
           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
            frontend | Running command: docker container inspect earthly-buildkitd
            frontend | Running command: docker image inspect earthly/buildkitd:v0.7.6
           buildkitd | Comparing running container "earthly-buildkitd" image ("sha256:d7e54c656f0dea2e41f3996ab69c3091f68a9c21971233e1a6fcaa497ba8faf4") with available image "earthly/buildkitd:v0.7.6" ("sha256:d7e54c656f0dea2e41f3996ab69c3091f68a9c21971233e1a6fcaa497ba8faf4")
            frontend | Running command: docker container inspect earthly-buildkitd
           buildkitd | Settings hashes match ("f108a55ba94437cf"), no restart required
           buildkitd | Version github.com/earthly/buildkit v0.7.6 0da024053105e3807a23ee7225954d822fabda00
           buildkitd | Platforms: linux/arm64 (native) linux/amd64 linux/amd64/v2 linux/riscv64 linux/ppc64le linux/s390x linux/386 linux/mips64le linux/mips64 linux/arm/v7 linux/arm/v6
           buildkitd | Utilization: 0 other builds, 0/20 op load
           buildkitd | GC stats: 527 MB cache, avg GC duration 0s, all-time GC duration 0s, last GC duration 0s, last cleared 0 B

 Build 🔧
————————————————————————————————————————————————————————————————————————————————

Logbus: setting organization "" and project "" at 2023-08-03T20:13:54.454656-07:00
 earthly/dind:alpine | --> Load metadata earthly/dind:alpine linux/arm64
 earthly/dind:alpine | Completed in 450.867583ms
             context | --> local context .
             context | Completed in 98.25µs
              +repro | --> FROM earthly/dind:alpine
              +repro | [----------] 100% resolve docker.io/earthly/dind:alpine@sha256:c156e7cbe19303a3ac1afafaeaeca3d46ec8eb49037a826618b1cfdbe4cb80a4
              +repro | Completed in 22.057041ms
             context | [          ]   0% transferring .:
             context | sent 1 file stat)
             context | [----------] 100% transferring .:
             context | Completed in 9.824459ms
              +repro | *cached* --> COPY docker-compose-x.yml .
              +repro | Completed in 0s
              +repro | *cached* --> WITH DOCKER (install deps)
              +repro | Completed in 24.792µs
              +repro | --> WITH DOCKER (docker-compose config)
              +repro | project name must not be empty
              +repro | ERROR Earthfile line 9:8
              +repro |       The command
              +repro |           WITH DOCKER (docker-compose config)
              +repro |       did not complete successfully. Exit code 15
              +repro | Overriding args used:
              +repro | Completed in 90.1195ms
earthfile2llb immediate error: Earthfile line 9:8 with docker run: read compose config file: process "/bin/sh -c EARTHLY_START_COMPOSE=\"true\" EARTHLY_COMPOSE_FILES=\"docker-compose-x.yml\" EARTHLY_COMPOSE_SERVICES=\"db\" /var/earthly/dockerd-wrapper.sh get-compose-config" did not complete successfully: exit code: 15
in		+repro
earthfile2llb group error: <nil>

============================= ❌ FAILURE [Build 🔧] ==============================

Repeating the output of the command that caused the failure
              +repro *failed* | --> WITH DOCKER (docker-compose config)
              +repro *failed* | project name must not be empty
              +repro *failed* | ERROR Earthfile line 9:8
              +repro *failed* |       The command
              +repro *failed* |           WITH DOCKER (docker-compose config)
              +repro *failed* |       did not complete successfully. Exit code 15
              +repro *failed* | Overriding args used:
Summary of timing information
Note that the times do not include the expansion of commands like BUILD, FROM, COPY (artifact).
 earthly/dind:alpine | () 450.867583ms
              +repro | () 112.201333ms
             context | () 9.922709ms
===============================================================
Total       	572.991625ms
Total (real)	715.46375ms
Share your logs with an Earthly account (experimental)! Register for one at https://ci.earthly.dev.
Error: build target: build main: failed to solve: Earthfile line 9:8 with docker run: read compose config file: process "/bin/sh -c EARTHLY_START_COMPOSE=\"true\" EARTHLY_COMPOSE_FILES=\"docker-compose-x.yml\" EARTHLY_COMPOSE_SERVICES=\"db\" /var/earthly/dockerd-wrapper.sh get-compose-config" did not complete successfully: exit code: 15
in		+repro

This seems sufficient to reproduce it.

Earthfile

VERSION 0.7

repro:
    FROM earthly/dind:alpine
    COPY docker-compose-x.yml .
    WITH DOCKER --pull postgres \
                --compose docker-compose-x.yml \
                --service db
        RUN sleep 5
    END

Docker compose file:

services:
  db:
    image: postgres
    restart: always
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

What should have happened?

This snippet should not fail with that error. It's been working fine for us for a while now, and suddenly began failing today.

The only thing I can tell might have changed is the new earthly/dind:alpine release from today.

I have not yet found a workaround.

Other Helpful Information

Earthly version: ➜ earthly version v0.7.6 78d2f5be89c8ada1048ed7130ed319948bf79912 darwin/arm64; macOS 13.2 homebrew

Docker: Docker version 24.0.2, build cb74dfc

[lalithsuresh]

A workaround is to add a WORKDIR in the target itself. I suspect it's related to this: #1885?

repro:
    FROM earthly/dind:alpine
    WORKDIR /root
    COPY docker-compose-x.yml .
    WITH DOCKER --pull postgres \
                --compose docker-compose-x.yml \
                --service db
        RUN sleep 5
    END

[EamonKeane]

Another workaround is to add:

name: myproject 
services:

[alexcb]

the earthly/dind:alpine was updated in a1ad930 (it has since been rolled back via docker tag earthly/dind@sha256:9b09804c68a2c68195b71e61a09616657b8f0a5ca01b4e029f710c7ca2396dc6 earthly/dind:alpine but only for amd64).

we're working on an update to earthly to fix this in #3163

[alexcb]

a fix has been released under v0.7.15

If you continue to experience difficulties (while using v0.7.15), try pining the dind image to: earthly/dind:alpine-3.18 or earthly/dind:ubuntu-23.04

[Smile-zjk]

Another workaround is to add:

name: myproject 
services:

@EamonKeane It’s worked for me,3q

[alexcb]

For anyone revisiting this issue, we ended up pushed a new image to the legacy earthly/dind:alpine tag, based on a rewrite of the dind-alpine target under 8187526. This new image fixed this regression for all versions of earthly.

Users should still use the newer image tags, which allow them to pin to specific OS and docker versions.