Failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[gintsmurans]

Hi!

Description of the issue

I recently updated docker and after update it seems that Dockerfile cannot use local images anymore. I have this instance:

Which is later used by another service in Dockerfile using FROM services_develop:staging. Prior update it was working just fine, now only option seems to be to create a local registry. Is this really intended?

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

/!\ If docker-compose version reports v2.0...` you're using the wrong repository, please report issues on Compose-CLI

Output of docker version

Client:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:23 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:55:36 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config
(Make sure to add the relevant -f and other flags)

networks:
  default:
    driver: bridge
  pm_default:
    external: true
    name: pm_default
services:
  build:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/build
    depends_on:
      develop:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_build:staging
    networks:
      default: null
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  develop:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/develop
    depends_on:
      redisdb:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_develop:staging
    networks:
      default: null
      pm_default: null
    ports:
    - published: 4100
      target: 4100
    - published: 4200
      target: 4110
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  redisdb:
    image: redis:alpine
version: '3.9'

Steps to reproduce the issue

1. docker-compose up --build build

Observed result

Fails to run

Expected result

Container is run

Stacktrace / full error message

#2 [services_build:staging internal] load build definition from Dockerfile
#2 sha256:904e5663af5547b640ab36250d9303488863d83cb6ca4c0111b17700a843eec9
#2 transferring dockerfile: 337B done
#2 DONE 0.0s

#1 [services_develop:staging internal] load build definition from Dockerfile
#1 sha256:d8973824e1cc852c8a3ba858f1c282f2943a049f4c86b2cc88e4ecfb20912cb3
#1 transferring dockerfile: 860B done
#1 DONE 0.0s

#3 [services_develop:staging internal] load .dockerignore
#3 sha256:a577a2d1f67337b83d0055cf11cf3d65ec16157e002120d74886257da7875479
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 ...

#5 [services_build:staging internal] load .dockerignore
#5 sha256:a3a79252ae83d732c535c559ffb437f9be6ae841462892d5b0c084a22f5ebf2b
#5 transferring context: 2B done
#5 DONE 0.0s

#6 [services_build:staging internal] load metadata for docker.io/library/services_develop:staging
#6 sha256:0aabca4baec97ad783f7b602174718f61033d1eb5568a3e962ea3e70e39858bb
#6 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 CANCELED
------
 > [services_build:staging internal] load metadata for docker.io/library/services_develop:staging:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Additional information

Macbook Pro M1, MacOS 11.4.

[MartinFalatic]

I'm using Docker Desktop 3.5.2 on macOS 11.5.1 (Intel hardware).

I just ran into this today (recently cleared out my local images so I was rebuilding from scratch). It kept looking on docker.io for what were clearly local images I was in the process of trying to build, then failing with:

...
------
 > [my_internal_app_name_tests internal] load metadata for docker.io/my_internal_app_name/app:latest:
------
------
 > [my_internal_app_name/app internal] load build context:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Under Experimental Features "Use Docker Compose V2" was selected. I deselected that, restarted Docker, and after that docker-compose worked properly as it always has. (Apparently docker-compose disable-v2 will have the same effect, but you probably still should restart Docker afterward.)

The man page at https://docs.docker.com/compose/cli-command/ states that V2 is in beta and that they will gradually turn this option on automatically for Docker Desktop users. I don't recall opting into that myself.

I think this has been set for a while now but I didn't notice this sooner because I'd already built my app at least once before and that local image was already present - til I wiped everything today, that is.

[gintsmurans]

Looks like I already had V2 disabled, then I tried to use image when enabled then again when disabled - nothing changes:

#3 [internal] load metadata for docker.io/library/external_services_develop:latest
#3 sha256:796a096830b8eb3bfae3531721a1dee705af9d239f65cd75ce0bee93aa3490f9
#3 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/library/external_services_develop:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
Service 'build' failed to build : Build failed

It still tries to load local image from docker.io.

[gintsmurans]

This seems to be related to me having M1 Macbook. I am running same build command on windows WSL just fine.

[gintsmurans]

After reading comments from @ndeloof on the #8538 here is what I have found:

1. Firstly turns out depends_on are parallel, thats why you first need to build all dependencies and only then can use built images. Correct me if I am wrong.
2. docker-compose and docker compose both works on intel macbook and also on windows wsl.
3. docker compose build doesn't work on M1 macbook.
4. docker-compose build works on M1 macbook only when DOCKER_BUILDKIT environment variable is set to 0
5. docker-compose up does not work on M1 even with DOCKER_BUILDKIT set to 0, so you first need to build image and then "up it" without --build flags.

[ndeloof]

This is indeed a buildkit limitation. cc @tonistiigi

[jonycodes]

I'm also hitting this on windows all the sudden, docker desktop 4.0.1, engine 20.10.8 compose 1.29.2. Is there a work around?

[MartinFalatic]

I'm also hitting this on windows all the sudden, docker desktop 4.0.1, engine 20.10.8 compose 1.29.2. Is there a work around?

Hit me too on a new machine even though I know I didn't opt in to their broken docker-compose 2.0 beta. Sure enough, I'd been opted in at some point without my consent. Unchecked the option in the dashboard and it worked fine.

[ndeloof]

This issue depends on docker/buildx#447

applies both on Docker Compose V2 and V1 as long as Buildkit is enabled (which is set by default on Docker Desktop)

[GF-Huang]

1>#1 [internal] load build definition from Dockerfile
1>#1 sha256:956bc10a1f2e1ceb1467f46c35bd8f35881c8bce89860af10d947bf29232184a
1>#1 transferring dockerfile: 32B 0.0s done
1>#1 DONE 0.0s
1>#2 [internal] load .dockerignore
1>#2 sha256:39e216f61cef12c49ac62239816a4769fd6ed8537fb3a48af81544abad0525d7
1>#2 transferring context: 34B done
1>#2 DONE 0.0s
1>#3 [internal] load metadata for docker.io/microsoft/dotnet:2.2-aspnetcore-runtime
1>#3 sha256:2209054a124f7bc1424fa6735d588f36916bcec272f45a5e3e54f7288e65d73e
1>#3 ...
1>#4 [auth] microsoft/dotnet:pull token for registry-1.docker.io
1>#4 sha256:209aca23d26d7a46457af8b422359c9c5f58113ed548183047a1fee47c3b5738
1>#4 DONE 0.0s
1>#3 [internal] load metadata for docker.io/microsoft/dotnet:2.2-aspnetcore-runtime
1>#3 sha256:2209054a124f7bc1424fa6735d588f36916bcec272f45a5e3e54f7288e65d73e
1>#3 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[PeterLaComb-Availity]

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine "buildkit" feature works for now:

"features": {
"buildkit": false
}

[Karanjot-singh]

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine "buildkit" feature works for now:

"features": { "buildkit": false }

I can confirm this works for MacBook air M1. Thanks!

[Microhive]

"features": { "buildkit": false }

Can also confirm this works on my Macbook Pro M1.

[gavin-shaw]

This worked for me too thank you.

In case anyone is confused like I was briefly, there is a "features" config node in both devcontainer.json and your local Docker configuration.

You must change your local Docker configuration, not devcontainer.json:

[wishma2013]

"features": { "buildkit": false }
Can also confirm this works on my Macbook Pro of macOS Big Sur 11.5.2.

[JoshuaSchlichting]

"features": { "buildkit": false } appears to cause my Docker daemon to hang as warned.

MacBook Pro (14-inch, 2021) with Apple M1 Pro chip
macOS Monterey Version 12.2

[MarcStoecker]

50/50 for the "features": { "buildkit": false } workaround for me:

• works for Win/x64 on my main machine
• not working for my Mac/M1 (daemon hangs)

[Artem-Haholkin-deepsee]

For me worked on m1 chip add arch
FROM --platform=linux/amd64 ubunutu:16.04

[alex-oswald]

Worked for me on MacBook Pro with M1 pro chip. Thanks.

[vosechu]

Ahhhhhhh, @Artem-Haholkin-deepsee got me working. I built a local image using --platform=linux/amd64, but then forgot to put that --platform on the next image in the chain. So, since I'm on an M1, it tried to find an arm variant (which I hadn't built), and failed. Presumably, it then tried to check in with dockerhub, and for whatever reason got a 403.

[521xueweihan]

with CLI also can use：export DOCKER_BUILDKIT=0 to close buildkit.

[D1no]

Just had the same issue when setting up visual studio code remote dev container on my M1 mac — weird as I believe it worked before. Docker Desktop v4.9.0

"features": { "buildkit": false }
Changed as seen below in the docker desktop app, and works now.

[jjpinto]

Windows 10 Enterprise
Docker desktop -> Settings -> Docker engine (v20.10.16)
"features": { "buildkit": false } -

Docker hangs - had to uninstall it.

[rajeshatverbtech]

None of these worked for me on MacBook Pro 2021 with M1 Max
The daemon does not hang, but the issue persists.

[seadub]

Tried setting to false to no avail.

System:
MacBook Pro18,1
Apple M1 Pro
Docker Engine v20.10.17

Config:

{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "debug": false,
  "experimental": false,
  "features": {
    "buildkit": false
  },
  "insecure-registries": [
    "..."
  ],
  "registry-mirrors": []
}

[umama931]

Also a M1 user, None of the solution above helped me either, still getting failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed is this a docker desktop issue?

[perymerdeka]

Also a M1 user, None of the solution above helped me either, still getting failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed is this a docker desktop issue?

this is a new issue of docker desktop, on M1, I try to build it on my windows machine is work well

[abegascia]

A solution for MacOS Monterrey 12.3.1

Set buildkit to false in Docker Desktop app

And run in terminal this commands:

• export DOCKER_BUILDKIT=0
• export COMPOSE_DOCKER_CLI_BUILD=0

It works fine to me.

[PatelTheKanu]

Disabling buildkit as shown above/below worked for me:

"features": {
    "buildkit": false
  }

System:
Macbook pro 2021
M1 Max
Monterey 12.6

[enyakucera]

It helped me to solve same problem on Docker v20.10.21 on ThinkPad/Win10 :-)

[leoliuowi]

it also happned to me. i used "docker-compose" instead of "docker compose" to walk around this issue. this error only happens if I am building for the first time, if i am doing a rebuild, it won't happen. I believe this is a bug

[laurazard]

I'm closing this as it seems to be a BuildKit issue. As mentioned before, to run Compose V2 without BuildKit (with the classic builder), you can set DOCKER_BUILDKIT=0. Please file an issue with https://github.com/moby/buildkit

[jeff-h]

Just a reminder to anyone coming here with this error: this could also be a legitimate case of the wrong tag in a custom image's FROM. At some point, perhaps between docker-compose build and docker compose build, it seems the generated default tag has changed; previously it would use an underscore but now a hyphen between the directory and the service name.

eg docker_my-service has become docker-my-service.

[midnightcodr]

export DOCKER_BUILDKIT=0 from @521xueweihan did the trick! Many thanks. btw Setting

"features": {
    "buildkit": false
  }

in daemon.json doesn't seem to work for me.

[avishayhajbi]

works for me as well
thanks !!

[surak]

+1 to export DOCKER_BUILDKIT=0 on an INTEL Mac with MacOS Ventura 13.2.1. This only happened after an update on docker.

The

features": {
    "buildkit": false

did nothing for me either.

Closing the issue here looks like a lazy thing to do when it's caused by this very own software.

[cgunnels]

For me, I was updating FROM php:7.3-apache to FROM php:8.1-apache. No suggested fixes work. I guess I won't be updating.

[HomeOfTheWizard]

windows 10 home + docker desktop 4.17
suggested solution don't work.

[leandrosena]

add --platform=linux/amd64 works for me

[vhartikainen]

v20.10.24 Docker Engine on Mac M1:
Setting the buildkit to false in configuration -> no help.
export DOCKER_BUILDKIT=0 -> works.

[dss010101]

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine "buildkit" feature works for now:

"features": { "buildkit": false }

setting this and im prompted with the following when trying to rebuild the container. it doesnt seem to want to use my docker file specified in the devcontainer.json..and instead prompts me to select an image