Add ability to pass X-Forwarded-* headers unchanged to upstream

[dzhlobo]

Problem description

In our app we generate full urls based on Host or X-Forwarded-Host and X-Forwarded-Proto headers. This is kind of default way in Django apps and there are no much flexibility present to control it, especially when a url is generated not in our code but in 3rd party library code.

On the ingress we have a "private" domain name because the app is behind a CloudFront CDN so the "public" domain is configured on CDN. This is what happens with clients requests:

1. Original request comes to CDN with Host: example.com. It sets X-Forwarded-Host and passes request to nginx ingress with following headers:

Host: api.services.example.com
X-Forwarded-Host: example.com

2. nginx ingress in its turn rewrites X-Forwarded-Host and passes request to upstream with following headers:

Host: api.services.example.com
X-Forwarded-Host: api.services.example.com
X-Forwarded-Proto: https

You can notice that X-Forwarded-Host from CDN gets lost and therefore urls generated by the app looks like https://api.services.example.com/*.

Solution I'd like to have

I'm thinking about a couple of options.

Option 1.
Add an annotation config key for ingress resource to tell nginx to pass particular headers to upstream without changes, something like this:

nginx.org/proxy-set-headers-from-downstream: "X-Forwarded-Host,ABC"

Then in location block in nginx config we would add configuration like this:

location / {
  proxy_set_header "X-Forwarded-Host" $http_x_forwarded_host;
  proxy_set_header "ABC" $http_abc;
}

I don't 100% like it because it is a custom one and not very intuitive due to different semantics comparing to other annotations that maps 1 to 1 to nginx directives (e.g. nginx.org/proxy-pass-headers adds proxy_pass_headers in config while there are no proxy_set_headers_from_downstream directive in nginx).

Option 2.
Add an annotation that allows to add proxy_set_header to location block:

nginx.org/proxy-set-header.x-forwarded-host: "${http_x_forwarded_host}"
nginx.org/proxy-set-header.abc: "${http_abc}"

I don't 100% like it because of dynamic annotation keys (wouldn't it be hard to parse in controller?). It seems we currently have no dynamic annotation keys.

Alternatives I've considered

We ended up using VirtualHost resource instead of Ingress with following configuration:

routes:
  - path: "/"
    action:
      proxy:
        upstream: "api"
        requestHeaders:
          set:
          - name: "X-Forwarded-Host"
            value: "${http_x_forwarded_host}"

It works as we need but we had to use VirtualServer custom resource while I would prefer to have Ingress.

[github-actions]

Hi @dzhlobo thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

[brianehlert]

Removed "proposal" to ensure we discuss this and it does not get missed.