[bug?] docker-compose.yml using build unable to access image from previous service-build #2024

typoworx-de · 2023-08-29T10:04:20Z

Contributing guidelines

I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

... the documentation does not mention anything about my problem
... there are no open or closed issues that are related to my problem

Description

I'm using docker-compose.yml to setup two services. Let's name them service-a and service-b for simplicity.

service-a builds an image that's required to build the dockerfile for service-b by using FROM service-a.

FROM registry.local.docker/service-a:local AS release

2nd try using additional-contexts & base-image

FROM base-image AS release

Expected behaviour

I would expect that docker-compose build for service-b will load the docker-host image tagged as registry.local.docker/service-a into the buildx build for service-b making the image available.

Actual behaviour

Running docker-compose build will break-off trying to build the service-b with error:

failed to solve: registry.local.docker/service-a:latest: registry.local.docker/service-a: not found

The image is available on docker-host. This has been checked using docker image ls | grep service-a which lists the service-a image correctly tagged as registry.local.docker/service-a.

Even trying to add build.additional_contexts for base-image and using base-image instead of the image-name itself won't work.

Buildx version

github.com/docker/buildx v0.11.2 9872040

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.11.2)
  compose: Docker Compose (Docker Inc., v2.20.0)
  scan: Docker Scan (Docker Inc., v0.8.0)
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-compose-v2.20.3-linux-x86_64" is not valid: plugin candidate "compose-v2.20.3-linux-x86_64" did not match "^[a-z][a-z0-9]*$"

Server:
 Containers: 27
  Running: 21
  Paused: 0
  Stopped: 6
 Images: 127
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: local
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: sysbox-runc crun io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: crun
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.15.89-xanmod1
 Operating System: Ubuntu 16.04.7 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 62.6GiB
 Name: gabriel-XMG
 ID: PZ6V:VDD5:IKCG:L27W:Q3BS:VO7L:GVQU:TELK:LD6F:5H3W:TEYI:7UKC
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: typoworx
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://registry-api.php-stack.docker:443/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 192.168.165.0/24, Size: 24

Builders list

NAME/NODE    DRIVER/ENDPOINT             STATUS  BUILDKIT        PLATFORMS
zen_haibt *  docker-container                                    
  zen_haibt0 unix:///var/run/docker.sock running v0.12.2         linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default      docker                                              
  default    default                     running v0.8.2+244e8cde linux/amd64, linux/386

Configuration

I've made a reproducer basing on simple alpine image (without any real service):
https://github.com/Gabriel-Kaufmann/reproducer-docker-compose-buildx-issue

Build logs

#1 [service-a internal] load build definition from Dockerfile.service-a
#1 transferring dockerfile: 69B done
#1 DONE 0.6s

#2 [service-a internal] load metadata for docker.io/library/alpine:latest
#2 DONE 1.4s

#3 [service-a internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.7s

#4 [service-a 1/1] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a
#4 resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a
#4 resolve docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a 0.3s done
#4 DONE 0.8s

#5 [service-a] exporting to docker image format
#5 exporting layers done
#5 exporting manifest sha256:6c1479c44138ae52eb46b9824a2cf7ab4755835b9c6135fbef50c19ec7f53dd3
#5 exporting manifest sha256:6c1479c44138ae52eb46b9824a2cf7ab4755835b9c6135fbef50c19ec7f53dd3 0.1s done
#5 exporting config sha256:c0881693c02c4d90f827e18e7260011ce1f1f6745832dcbff55c8e2e9caf8cde
#5 exporting config sha256:c0881693c02c4d90f827e18e7260011ce1f1f6745832dcbff55c8e2e9caf8cde 0.1s done
#5 ...

#4 [service-a 1/1] FROM docker.io/library/alpine:latest@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a
#4 sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 3.40MB / 3.40MB 0.8s done
#4 DONE 2.0s

#5 [service-a] exporting to docker image format
#5 sending tarball
#5 sending tarball 0.3s done
#5 DONE 2.0s

#6 [service-a service-a] importing to docker
#6 DONE 0.0s

#7 [service-b internal] load build definition from Dockerfile.service-b
#7 transferring dockerfile: 156B done
#7 DONE 0.2s

#8 [service-b context base-image] load metadata for registry.php-stack.docker/service-a:local
#8 ERROR: registry.php-stack.docker/service-a:local: not found
------
 > [service-b context base-image] load metadata for registry.php-stack.docker/service-a:local:
------

Additional info

I've tested using local docker registry:v2 image and build still fails. In my opinion build should work with and without external registry also trying to provide local docker images from host (where buildx runs).

service-a image is loaded into my docker image-storage! But funny fact... it's always shown with "age 3 weeks" even when I've been running a fresh build only 1 minute ago (and deleted the old image before):

docker image ls | grep 'registry.php-stack.docker/service'
registry.php-stack.docker/service-a       local     c0881693c02c      3 weeks ago    7.34MB

And finally regardeless of adding --push --pull to docker-compose build and also providing x-bake load/push attributes in docker-compose.yml the registry is NOT pushed to my local registry. It's only available in my docker host image-storage.

This should take care of that the image(s) are provided to buildx (regardeless if registry or docker image-storage), but it doesn't work as documentated and expected.

 additional_contexts:
  base-image: docker-image://${REGISTRY:-registry.local.docker}/service-a:local

The text was updated successfully, but these errors were encountered:

typoworx-de · 2023-08-30T10:57:08Z

Just for notice. As my build and work got stuck on this issue I've merged all service Dockerfiles into one using different targets to specify which image should be used for service-a and service-b.

This build seems to work now by referencing the internal stage (AS xyz) instead of the same stage being build as base-image. But this isn't what I would expect to get this working in a more clean manner. This is simply a workaround. Neither referencing using an external registry, nor using additional_contexts worked for me as they should do.

crazy-max · 2023-10-20T13:10:38Z

Thanks for your report and sorry for the delay!

docker image ls | grep 'registry.php-stack.docker/service'
registry.php-stack.docker/service-a       local     c0881693c02c      3 weeks ago    7.34MB

NAME/NODE    DRIVER/ENDPOINT             STATUS  BUILDKIT        PLATFORMS
zen_haibt *  docker-container                                    
  zen_haibt0 unix:///var/run/docker.sock running v0.12.2         linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default      docker                                              
  default    default                     running v0.8.2+244e8cde linux/amd64, linux/386

A container builder (docker-container driver) can't access the docker store.

Even trying to add build.additional_contexts for base-image and using base-image instead of the image-name itself won't work.

services:
  service-a:
    image: ${REGISTRY:-registry.local.docker}/service-a:local
    build:
      context: docker/
      dockerfile: Dockerfile.service-a
      target: release
      x-bake:
        output: type=registry
        load: true
        push: true

  service-b:
    image: ${REGISTRY:-registry.local.docker}/service-b:local
    depends_on:
      - service-a
    build:
      context: docker/
      dockerfile: Dockerfile.service-b
      target: release
      additional_contexts:
        base-image: docker-image://${REGISTRY:-registry.local.docker}/service-a:local
      args:
        # only required for out-commented try using direct-image reference
        REGISTRY: "${REGISTRY:-registry.local.docker}"
      x-bake:
        output: type=registry
        load: true
        push: true

The way you're using the named context is racy and will not work. When using a named context you want to use the build result of the target. In your case you want to use build result of service-a target in service-b for the base-image stage. Would look like this:

services:
  service-a:
    image: ${REGISTRY:-registry.local.docker}/service-a:local
    build:
      context: docker/
      dockerfile: Dockerfile.service-a
      target: release

  service-b:
    image: ${REGISTRY:-registry.local.docker}/service-b:local
    depends_on:
      - service-a
    build:
      context: docker/
      dockerfile: Dockerfile.service-b
      target: release
      additional_contexts:
        base-image: "target:service-a"

But it doesn't seem to work either:

$ docker compose build
#0 building with "default" instance using docker driver

#1 [service-a internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [service-a internal] load build definition from Dockerfile.service-a
#2 transferring dockerfile: 69B 0.0s done
#2 DONE 0.0s

#3 [service-a internal] load metadata for docker.io/library/alpine:latest
#3 DONE 0.0s

#4 [service-a 1/1] FROM docker.io/library/alpine
#4 CACHED

#5 [service-a] exporting to image
#5 exporting layers done
#5 writing image sha256:d9d23fabe826a490f1ddf3cef69c2b8bbf8f009b2ee18fa282664bf0fdde6a75 done
#5 naming to registry.php-stack.docker/service-a:local done
#5 DONE 0.0s
failed to get build context base-image: stat /home/crazymax/buildx/.dev/2024/target:service-a: no such file or directory

Looks like compose try to solve the absolute path of the named context which is wrong (cc @glours @milas).

If you're using compose to just build, I would suggest to switch to bake and an HCL definition. Tried your repro with Bake and works for me:

# docker-bake.hcl
group "default" {
  targets = ["service-b"]
}

target "service-a" {
  context = "./docker"
  dockerfile = "Dockerfile.service-a"
  target = "release"
}

target "service-b" {
  context = "./docker"
  dockerfile = "Dockerfile.service-b"
  target = "release"
  contexts = {
    base-image = "target:service-a"
  }
}

$ docker buildx bake
#0 building with "default" instance using docker driver

#1 [service-a internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [service-a internal] load build definition from Dockerfile.service-a
#2 transferring dockerfile: 69B 0.0s done
#2 DONE 0.0s

#3 [service-a internal] load metadata for docker.io/library/alpine:latest
#3 DONE 0.0s

#4 [service-a 1/1] FROM docker.io/library/alpine
#4 CACHED

#5 [service-a] exporting to image
#5 exporting layers done
#5 writing image sha256:1cad2846fc76fdde7217fb06db38fbcdccaed71baccc30a7e8ec70f095ea3d11 done
#5 DONE 0.0s

#6 [service-b internal] load build definition from Dockerfile.service-b
#6 transferring dockerfile: 156B 0.0s done
#6 DONE 0.1s

#7 [service-b internal] load .dockerignore
#7 transferring context: 2B done
#7 DONE 0.0s

#4 [service-b 1/1] FROM docker.io/library/alpine
#4 CACHED

#8 [service-b] exporting to image
#8 exporting layers done
#8 writing image sha256:1cad2846fc76fdde7217fb06db38fbcdccaed71baccc30a7e8ec70f095ea3d11 done
#8 DONE 0.0s

More info: https://docs.docker.com/build/bake/build-contexts/

typoworx-de added the status/triage label Aug 29, 2023

typoworx-de changed the title ~~[bug?] docker-compose.yml using build unable to b~~ [bug?] docker-compose.yml using build unable to access image from previous service-build Aug 29, 2023

typoworx-de mentioned this issue Aug 29, 2023

docker-compose using buildkit fails to fetch local images (already existing or build by another service) moby/buildkit#4162

Open

crazy-max added area/bake/compose and removed status/triage labels Oct 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug?] docker-compose.yml using build unable to access image from previous service-build #2024

[bug?] docker-compose.yml using build unable to access image from previous service-build #2024

typoworx-de commented Aug 29, 2023 •

edited

Loading

typoworx-de commented Aug 30, 2023

crazy-max commented Oct 20, 2023

[bug?] docker-compose.yml using build unable to access image from previous service-build #2024

[bug?] docker-compose.yml using build unable to access image from previous service-build #2024

Comments

typoworx-de commented Aug 29, 2023 • edited Loading

Contributing guidelines

I've found a bug and checked that ...

Description

Expected behaviour

Actual behaviour

Buildx version

Docker info

Builders list

Configuration

Build logs

Additional info

typoworx-de commented Aug 30, 2023

crazy-max commented Oct 20, 2023

typoworx-de commented Aug 29, 2023 •

edited

Loading