Support GCP Artifact Registry

[K4pper]

Is your feature request related to a problem? Please describe.

Currently you cannot authenticate against the google artifact registry with the config.json file in watchtower.
The reason for watchtower failing authentication is because of the expected strings in the www-authenticate header. Currently watchtower expects both a realm and a service string but the artifact registry does not send any service string when sending a HTTP GET request.
example below:

This is not an issue when creating a config.json and authenticating against the artifact registry manually.

Describe the solution you'd like

Watchtower should be able to authenticate against the google artifact registry using the config.json file by not expecting a service string.

Describe alternatives you've considered

An alternative would be to build a container that have gcloud installed, this however would be a very bloated container compared to making the config.json file work

Additional context

No response

[github-actions]

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

[piksel]

Are you sure this doesn't work? The code you are thinking of is only used for HEAD requests (which I don't even know if GCP support?). It should just fall back to doing a pull as normal if it cannot perform a HEAD request. And, when pulling it's not watchtower that does the authentication, it's the docker daemon itself.

[K4pper]

@piksel
When i look into the watchtower logs it gives me the following:

So it seems that it does try a normal pull but that fails and says authentication failed.
i have tried creating an ubuntu container with docker in docker (DIND) and tried using the exact same config.json file within that container and it works, so it is only when using watchtower it seems to have an authentication error.

Hope this gives more insight into the problem (:

[Kerwood]

Can confirm, Watchtower does not work on Google Artifact Registry.
Is it possible to force Watchtower use the Docker daemon to do the pull instead of injecting credentials into Watchtower?

[piksel]

Like I said, it's always the docker daemon that does the pull and authentication. The error even contains Error response from daemon: .
That being said, something is clearly different. How are you authenticating against the repo?

[K4pper]

We authenticate with the repo by following the steps in the watchtower docs: https://containrrr.dev/watchtower/private-registries/

Where it says to authenticate with GCloud you would base64 encode the value of your service account key and insert that into a config.json, then mount that into the container.

However when we do this watchtower errors out. When i try to authenticate manually it pulls the image just fine.

[piksel]

Hm, it says that you should base64 encode it as _json_key:<JSON>, where <JSON> is the full contents of a key file in JSON format. Is this what you are doing?

This is the GCP docs for reference:
https://cloud.google.com/artifact-registry/docs/docker/authentication#json-key

[K4pper]

Yes the whole thing is base64 encoded as described in the docs

[piksel]

What do you mean by:

This is not an issue when creating a config.json and authenticating against the artifact registry manually.

Are you saying that using the same config.json as you mount in the container can be used to authenticate with the docker cli, without using any credentials helpers?

[K4pper]

Yes exactly, when i use the same config.json to authenticate with the docker CLI it works as expected

[simskij]

Yes exactly, when i use the same config.json to authenticate with the docker CLI it works as expected

Can you please post a redacted version of your config.json? This doesn't add up for me.

[K4pper]

@simskij Yeah sure i have posted it below:

{
    "auths": {
        "europe-west3-docker.pkg.dev": {
            "auth": "BASE64-encoded-value"
        }
    }
}

[Kerwood]

@simskij Yeah sure i have posted it below:

{
    "auths": {
        "europe-west3-docker.pkg.dev": {
            "auth": "BASE64-encoded-value"
        }
    }
}

And the BASE64-encoded-value is generated with below command ?

bash echo -n "_json_key:$(cat gcloudauth.json)" | base64 -w0

[K4pper]

@Kerwood
Yes I used that exact command

[simskij]

@simskij Yeah sure i have posted it below:

{
    "auths": {
        "europe-west3-docker.pkg.dev": {
            "auth": "BASE64-encoded-value"
        }
    }
}

And the BASE64-encoded-value is generated with below command ?

bash echo -n "_json_key:$(cat gcloudauth.json)" | base64 -w0

And in your container config, have you prefixed your container image names with the registry they will be fetched from (and does it match europe-west3-docker.pkg.dev)?