unauthorized pull after v2.3.0 upgrade

[andreas-p]

I'm running Harbor 2.2.2 as Pull-Through Registry with a little modification:

nginx configuration is modified, so that the server on 8443 will respond to the registry name only (setting server_name), and adding another server configuration for the server_names registry-1.docker.io quay.io k8s.gcr.io for the /v2/ endpoint. In addition, the URI is rewritten with rewrite ^/v2/(.*) /v2/$host/$1 break;, which redirects the upstream host requests to the local public registry projects. Added another connect proxy and redirect docker to it, and now docker will pull transparently through Harbor.

This works with 2.2.2, but the new 2.3.0 will provoke an error when pulling from those proxied upstream registries: Error response from daemon: unauthorized: authorize header needed to send HEAD to repository: authorize header needed to send HEAD to repository. Only the proxy 401 is logged, nothing from core.
What's going wrong here?

[sixeela]

Hello,

Probably caused by breaking change:

Breaking Changes
The API to GET artifact under public project such as GET /v2/$public_project/$repo/manifests/$tag, will receive a 401 if the request does not carry "Authorization" header, more details see:
#14711
#14768

[andreas-p]

@sixeIa Indeed this seems to describe the problem.
So how to pull anonymously from a public project with another proxy in front?

[andreas-p]

I was able to hot-fix this for my setup by adding proxy_set_header Authorization "none"; in my rewriting proxy.

[Edgar097]

Hi, I am having the same issue
based on this issue #8082
$ docker pull dockerhubcache.com/library/ubuntu:latest
Error response from daemon: unauthorized: authorize header needed to send HEAD to repository: authorize header needed to send HEAD to repository

I tried a bunch of workarounds but they didn't work.
These are my nginx.conf and default.conf.
Any thoughts?

nginx.conf

http {
    #include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  'Uri= $new_uri, $args, New= $new_args   --- $remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  65;
    #gzip  on;

    # Map for proxy cache
    map $host $harbor_project {
      default 0;  # $harbor_project = 0
      ~^dockerhubcache.com$ cache;  # if ($host== dockerhubcache.com)  $harbor_project = cache   // dockerhubcache.com/"cache"/repo/image
      ~^(?<project>.+).dockerhubcache.com$ $project;   # if($host == project.dockerhubcache.com)  $harbor_project = "project"
  }

  map $request_uri $new_uri {
    ~^/v2/(.+)$ /v2/$harbor_project/$1;   # if ($request_uri== /v2/project_1)  $new_uri = /v2/$harbor_project/project_1
  }
  map $args $new_args {
    ~^(?<prefix2>.*scope=repository%3A)(?<suffix2>.*)$ ${prefix2}${harbor_project}%2F$suffix2;   # if ($args == (AnythingOrNothing)scope=repository%3A(AorN)) $new_args =
                                                                                                    #  (AorN)scope=repository%3A$harbor_project%2F(AorN)
  }
  map $upstream_http_www_authenticate $new_header {
    ~^(?<prefix1>.*https://).*(?<suffix1>/service/token.*)$ $prefix1$host$suffix1;  # if ($upstream_http_www_authenticate== (AorN)https://(AorN)/service/token(AorN)) $new_header = 
                                                                                        #  (AorN)https://$host/service/token(AorN)
  }

    include /etc/nginx/conf.d/*.conf;
}

default.conf

server {
    listen       80;
    server_name  dockerhubcache.com;

    # dockerhubcache.com

    # access_log  /var/log/nginx/host.access.log  main;

    # Rewrite for proxy cache
    if ($harbor_project != 0) {
      rewrite ^/v2/(.+)$ $new_uri; #  $new_uri = /v2/(A)
      set $args $new_args;   # $args = (AorN)scope=repository%3A$harbor_project%2F(AorN)
    }
    
    location /v2/ {
        proxy_hide_header Www-Authenticate;
        add_header Www-Authenticate $new_header always;
        rewrite ^/v2/(.+)$ https://<cache-registry>.com/v2/$1;  # https://cache-registry.com/v2/(A)
    }
}

[hcnhcn012]

Hello,

Probably caused by breaking change:

Breaking Changes
The API to GET artifact under public project such as GET /v2/$public_project/$repo/manifests/$tag, will receive a 401 if the request does not carry "Authorization" header, more details see:
#14711
#14768

@sixeela hello, I've also met this problem under harbor version 2.9.6, it causes pullImageBackOff in my cluster, so i want to know how to solve it by kubernetes configurations becuase i can not mannually add Authorization header one node by one node