ACME responses are blocked by auth plugins

[alexmbird]

I'm trying to use the ACME plugin to automatically provision TLS certs from LetsEncrypt and have run into a clash with the basic auth plugin. Presumably this will affect other kinds of auth too.

The setup is a simple one:

• A Kubernetes Ingress resource (path /) that is translated to a route by the Kong ingress controller. This is for the Kong admin service, which of course should not be exposed without auth. As the path is / I think this satisfies the requirement that /.well-known/acme-challenge/... fall inside a route served by Kong.
• A KongPlugin containing an instance of the basic auth plugin with a single KongConsumer setup. I've tested this and auth is being applied correctly.
• A KongClusterPlugin setting up the acme plugin. As it is a cluster-level plugin this will apply to all routes.

As expected the acme plugin tries to get a cert from LetsEncrypt. But LE's calls to http://my.domain/.well-known/acme-challenge/... are met with a 401 response.

To verify there isn't some other problem with the setup I've tried again with basic auth disabled; in that case the acme plugin is able to get a cert without any problem.

I believe what's wrong is that the ACME plugin runs with a lower priority (999) than basic-auth's (1001). Unfortunately at the present time Kong gives no way for the user to adjust plugin priorities. I cannot think of any reason why a user would want their ACME responses protected by auth, so please can the priority be adjusted so that the acme plugin runs first?

[fffonion]

I believe what's wrong is that the ACME plugin runs with a lower priority (999) than basic-auth's (1001).

Yeah that's good point, we can address that. cc @gszr

In the meanwhile, there's a work around for this. Since Kong only maps a request to a single route, as long as your basic auth plugin is not applied globally, you can actually create a dummy route (similar to https://github.com/Kong/kong-plugin-acme#enable-the-plugin). So that the challenge request will map to the dummy route, not to / where basic-auth is applied.

[alexmbird]

Thanks @fffonion. I've verified the workaround functions but in our use case it poses a challenge: when created via the Kong ingress controller every route needs its own Ingress resource. I'm planning to host many different domains on this Kong instance so I'll have to create an extra ingress for each of them.

I did try creating a single one with a wildcard hostname, but it seems Kong's Nginx prioritizes matches with non-wildcard hostnames so that didn't work.

[fffonion]

@alexmbird In that case, and if i understand correctly, you can set regex_priority for the single wildcard hostname route higher, so that it will always got matched before other routes.

[alexmbird]

I've given regex_priority a go but it doesn't appear to have an effect. Perhaps it applies to regexes and not wildcard-based hostnames. I tried it alongside regex hostnames as well but the Kubernetes Ingress resource lacks support for them.

In case it helps here's the Kubernetes resource I'm using. Pointed to kong2-general-admin backend as a hack for now.

$ kubectl -n kong2 get -o yaml ing/acme-challenge-responses
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    konghq.com/regex-priority: "100"
    kubernetes.io/ingress.class: kong2-general
  name: acme-challenge-responses
  namespace: kong2
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: kong2-general-admin
          servicePort: 8444
        path: /.well-known/acme-challenge