fix: perform https redirect prior to ext_authz calls

[LanceEa]

Description

Fixes a regression with https_redirection when an AuthService is applied. This was introduced in v3+ due to upgrading Envoy where a behavior change was introduced for the ext_authz http filter. This restores the previous behavior by disabling ext_authz calls on a per route basis when an AuthService is applied.

Before change:

{
    "match": {
      "case_sensitive": true,
      "prefix": "/backend/",
      "runtime_fraction": {
        "default_value": {
          "denominator": "HUNDRED",
          "numerator": 100
        },
        "runtime_key": "routing.traffic_shift.cluster_quote_default_default"
      }
    },
    "redirect": {
      "https_redirect": true
    }

After change:

{
    "match": {
      "case_sensitive": true,
      "prefix": "/backend/",
      "runtime_fraction": {
        "default_value": {
          "denominator": "HUNDRED",
          "numerator": 100
        },
        "runtime_key": "routing.traffic_shift.cluster_quote_default_default"
      }
    },
    "redirect": {
      "https_redirect": true
    },
    "typed_per_filter_config": {
      "envoy.filters.http.ext_authz": {
        "@type": "type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute",
        "disabled": true
      }
    }
  }

IMPORTANT: this is only applied to the https_redirect routes that are added by default for all Host which has requestPolicy.insecure.action = Redirect as the default action.

Related Issues

#4620

Testing

Manual Verification:

• reproduced in a cluster with 3.2.0
• tested dev build and verified https redirection occurs
• visually inspected the envoy.json to ensure expected config was generated

Automtates Test:

• Added two new Unit Test to verify generated envoy config from diagd
• Added new KAT test to verify 301 redirect for non-tls request

Checklist

• I made sure to update CHANGELOG.md.
• This is unlikely to impact how Ambassador performs at scale.
• My change is adequately tested.
• I updated DEVELOPING.md with any any special dev tricks I had to use to work on this code efficiently.
• The changes in this PR have been reviewed for security concerns and adherence to security best practices.

[LanceEa]

@LukeShu @ddymko - I have the KAT testing passing now so this is good to go for a review.

As I was working on the KAT tests I added these two commits for CI improvements:

• d31ab7f
• 902bed4

I'm happy to PR them separately if they are sticking points, in favor of landing the fix commit.

[ddymko]

@LukeShu @ddymko - I have the KAT testing passing now so this is good to go for a review.

As I was working on the KAT tests I added these two commits for CI improvements:

* [d31ab7f](https://github.com/emissary-ingress/emissary/commit/d31ab7f408ba43b70661e89cfb872ecf4e651483)

* [902bed4](https://github.com/emissary-ingress/emissary/commit/902bed428da6796fd50d34730f7a9500b4dd8b4f)

I'm happy to PR them separately if they are sticking points, in favor of landing the fix commit.

I don't have a strong preference for this specific occurrence. They are small enough we can keep them here or if we want a cleaner history, then break them out into CI improvements PR.

[LanceEa]

I don't have a strong preference for this specific occurrence. They are small enough we can keep them here or if we want a cleaner history, then break them out into CI improvements PR.

Ok, I will leave it then since they are separate commits already and are not mixed with the fix commit.

[LukeShu]

When possible, I've been preferring f-strings to the old self.format magic.

[LukeShu]

There's the redirect-Mapping issue discussed today that still needs fixed. But the changes here lgtm.

[LanceEa]

There's the redirect-Mapping issue discussed today that still needs fixed. But the changes here lgtm.

Cool, I will go ahead and land this so it can be in the next RC and will get to work on the fix for the mapping redirect issue as well.