Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: New integration guide for Envoy Gateway #1412

Merged
merged 4 commits into from
May 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions docs/content/guides/proxies/envoy_gateway.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
title: "Envoy Gateway Integration"
date: 2024-05-02T13:02:43+02:00
draft: false
weight: 13
menu:
guides:
parent: "API Gateways & Proxies"
description: This guide explains how to integrate heimdall with Envoy Gateway.
---

:toc:

https://gateway.envoyproxy.io[Envoy Gateway] is an open source project for managing https://www.envoyproxy.io/[Envoy Proxy] as a Kubernetes-based application gateway by making use of the https://gateway-api.sigs.k8s.io/[Gateway API] resources.

== Prerequisites

* A kubernetes cluster
* Deployed Envoy Gateway (See https://gateway.envoyproxy.io/v1.0.1/install/[here] for installation options)
* Deployed https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.GatewayClass[`GatewayClass`] resource that matches Envoy Gateway's configured `controllerName` (typically `gateway.envoyproxy.io/gatewayclass-controller`), as well as a deployed https://gateway-api.sigs.k8s.io/api-types/gateway[`Gateway`] resource.
* heimdall installed and operated in link:{{< relref "/docs/concepts/operating_modes.adoc#_decision_mode" >}}[Decision Operation Mode].

== Integration Options

Technically, the integration happens the same way as with link:{{< relref "/guides/proxies/envoy.adoc" >}}[Envoy] itself by making use of the https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto.html[External Authorization] filter, and can be done in two ways:

* either via HTTP
* or via gRPC (recommended)

In both cases, the filter calls an external gRPC or HTTP service (here heimdall) to check whether an incoming HTTP request is authorized or not. If heimdall responses with `2xx` the request is forwarded to the upstream service, otherwise the response from heimdall is returned to the caller.

In case of Envoy Gateway the abovesaid configuration happens via a https://gateway.envoyproxy.io/contributions/design/security-policy/[`SecurityPolicy`] custom resource, that can be linked to a https://gateway-api.sigs.k8s.io/api-types/gateway[`Gateway`], https://gateway-api.sigs.k8s.io/api-types/httproute[`HTTPRoute`], or a https://gateway-api.sigs.k8s.io/api-types/grpcroute[`GRPCRoute`] resource.

NOTE: As of today, there is a limitation in the implementation of the Envoy Gateway - it does not allow cross-namespace reference of external auth services (see also https://github.com/envoyproxy/gateway/issues/3322[envoyproxy/gateway#3322]). That means, the https://gateway-api.sigs.k8s.io/api-types/httproute[`HTTPRoute`], the https://gateway-api.sigs.k8s.io/api-types/gateway[`Gateway`] resource and heimdall must be deployed in the same namespace.

== Global Configuration

To integrate heimdall with the gateway globally, that is, each and every request will be forwarded to heimdall for authentication and authorization purposes first, create a https://gateway.envoyproxy.io/contributions/design/security-policy/[`SecurityPolicy`] as shown below in the namespace, the https://gateway-api.sigs.k8s.io/api-types/gateway[`Gateway`] resource is deployed into.

[source, yaml]
----
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
name: ext-auth-heimdall # <1>
namespace: heimdall # <2>
spec:
targetRef: # <3>
group: gateway.networking.k8s.io
kind: Gateway
name: eg
namespace: heimdall
extAuth:
grpc:
backendRef: # <4>
name: heimdall
port: 4456
namespace: heimdall
----
<1> The name of the `SecurityPolicy`. You can change it to any other value if you like.
<2> The namespace for the policy. It must be the same namespace the `Gateway` resource and heimdall are deployed into. So change it to your namespace.
<3> Defines the `Gateway` resource, this policy should be applied to. Change the `name` property to the name of your `Gateway` resource and the `namespace` property to the proper namespace (same as in 2)
<4> Defines the reference to the heimdall `Service` using the gRPC protocol. Change the `name` and the `namespace` to the proper values of your setup.

== Route-level Configuration

The integration on the route level happens similar to the link:{{< relref "#_global_configuration" >}}[global integration]. The difference is that the https://gateway.envoyproxy.io/contributions/design/security-policy/[`SecurityPolicy`] is applied to an https://gateway-api.sigs.k8s.io/api-types/httproute[`HTTPRoute`] as shown below and not the https://gateway-api.sigs.k8s.io/api-types/gateway[`Gateway`] resource.

[source, yaml]
----
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
name: ext-auth-example # <1>
namespace: heimdall # <2>
spec:
targetRef: # <3>
group: gateway.networking.k8s.io
kind: HTTPRoute
name: heimdall
extAuth:
grpc:
backendRef: # <4>
name: heimdall
port: 4456
namespace: heimdall
----
<1> The name of the `SecurityPolicy`. You can change it to any other value if you like.
<2> The namespace for the policy. It must be the same namespace the `HTTPRoute` resource is deployed into, so the namespace, your application is deployed to. So change it to your namespace.
<3> Defines the `HTTPRoute` resource, this policy should be applied to. Change the `name` property to the name of your `HTTPRoute` resource and the `namespace` property to the proper namespace (same as in 2)
<4> Defines the reference to the heimdall `Service` using the gRPC protocol. Change the `name` and the `namespace` to the proper values of your setup.

== Security Considerations

The configuration options shown above are highly insecure, as the communication from the gateway to heimdall happens over plain HTTP. Therefore, it is highly recommended to enable TLS. This can be achieved by enabling TLS for heimdall and attaching a https://gateway-api.sigs.k8s.io/api-types/backendtlspolicy/[`BackendTLSPolicy`] resource shown below to heimdall's https://kubernetes.io/docs/concepts/services-networking/service/[`Service`].

[source, yaml]
----
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: BackendTLSPolicy
metadata:
name: heimdall-btls
namespace: heimdall # <1>
spec:
targetRef: # <2>
group: ''
kind: Service
namespace: heimdall
name: heimdall
sectionName: "4456"
tls: # <3>
caCertRefs:
- name: demo-ca # <4>
group: ''
kind: ConfigMap
hostname: heimdall # <5>
----
<1> Change it to the namespace in which heimdall is deployed
<2> The reference to heimdall's `Service`. Change the `name` and the `namespace` to the proper values.
<3> Here we configure the reference to the `ConfigMap` with the certificate of the CA, used to issue a TLS server authentication certificate for heimdall, as well as the hostname used by heimdall (and present in the SAN extension of heimdall's TLS certificate). The `ConfigMap` must be in the same namespace as the `BackendTLSPolicy`.
<4> The name of the `ConfigMap`. Change it to the proper value.
<5> The expected hostname used by heimdall. Change it to the proper value.

== Additional Resources

* A fully working example with Envoy Gateway is also available on https://github.com/dadrus/heimdall/tree/main/examples[GitHub].
* You can find the official external authentication guide for Envoy Gateway https://gateway.envoyproxy.io/v1.0.1/tasks/security/ext-auth/[here]. It contains a fully working setup with a demo application.
* https://gateway.envoyproxy.io/v1.0.1/tasks/security/secure-gateways/[Secure Gateways] is a highly recommended read as well.
2 changes: 1 addition & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ To be able to run the docker compose examples, you'll need Docker and docker-com

To be able to run the Kubernetes based examples, you'll need just, kubectl, kustomize, helm and a k8s cluster. Latter can also be created locally using kind. The examples are indeed using it.

Please note: The main branch may have breaking changes (see pending release PRs for details under https://github.com/dadrus/heimdall/pulls) which would make the usage of the referenced heimdall images impossible (even though the configuration files and rules reflect the latest changes). In such situations you'll have to build a heimdall image by yourself and update the setups to use it.
**Note:** The main branch may have breaking changes (see pending release PRs for details under https://github.com/dadrus/heimdall/pulls) which would make the usage of the referenced heimdall images impossible (even though the configuration files and rules reflect the latest changes). In such situations you'll have to use the dev image or build a heimdall image by yourself and update the setups to use it.
91 changes: 50 additions & 41 deletions examples/kubernetes/Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ nginx_version := '9.7.7'
contour_version := '17.0.0'
emissary_version := '8.7.2'
haproxy_version := '0.14.4'
envoy_gw_version := 'v1.0.1'
metallb_version := '0.13.10'
certmanager_version := '1.12.3'
certmanager_version := '1.14.5'
trustmanager_version := '0.9.2'

cluster_name := 'demo-cluster'
default_ingress_controller := "contour"
default_router := "contour"

setup-charts:
helm repo add bitnami https://charts.bitnami.com/bitnami
Expand All @@ -21,7 +23,7 @@ setup-charts:
helm repo add jetstack https://charts.jetstack.io
helm repo add dadrus https://dadrus.github.io/heimdall/charts
helm repo add datawire https://app.getambassador.io
helm repo add haproxy-ingress https://haproxy-ingress.github.io/charts
helm repo add haproxy https://haproxy-ingress.github.io/charts
helm repo update

## Installs Grafana
Expand Down Expand Up @@ -96,8 +98,8 @@ install-nginx-ingress-controller:
--wait

install-contour-ingress-controller:
helm upgrade --install contour-ingress-controller bitnami/contour \
-n contour-ingress-controller --create-namespace \
helm upgrade --install contour-controller bitnami/contour \
-n contour-controller --create-namespace \
--version {{contour_version}} \
-f contour/helm-values.yaml # used only to configure a global auth server

Expand All @@ -107,35 +109,33 @@ install-emissary-ingress-controller:
kubectl apply -f https://app.getambassador.io/yaml/emissary/${app_version}/emissary-crds.yaml
kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system

helm upgrade --install emissary-ingress datawire/emissary-ingress \
-n emissary-ingress-controller --create-namespace \
helm upgrade --install emissary datawire/emissary \
-n emissary-controller --create-namespace \
--version {{emissary_version}}

kubectl -n emissary-ingress-controller wait --for condition=available --timeout=90s deploy -lapp.kubernetes.io/instance=emissary-ingress
kubectl -n emissary-controller wait --for condition=available --timeout=90s deploy -lapp.kubernetes.io/instance=emissary

kubectl apply -f - <<EOF
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: emissary-tls-ingress-listener
namespace: emissary-ingress-controller
spec:
port: 8443
protocol: HTTPS
securityModel: XFP
hostBinding:
namespace:
from: ALL
EOF
kubectl apply -f emissary/listener.yaml


install-haproxy-ingress-controller:
helm install haproxy-ingress-controller haproxy-ingress/haproxy-ingress \
-n haproxy-ingress-controller --create-namespace \
helm upgrade --install haproxy-controller haproxy/haproxy \
-n haproxy-controller --create-namespace \
--version {{haproxy_version}} \
-f haproxy/helm-values.yaml \
--wait

install-envoy-gateway:
helm upgrade --install eg oci://docker.io/envoyproxy/gateway-helm --version {{envoy_gw_version}} \
-n envoy-gateway-system --create-namespace

kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

#kubectl rollout restart deployment cert-manager -n cert-manager

kubectl apply -f envoygw/gateway.yaml


install-lb:
#!/usr/bin/env bash
helm upgrade --install metallb metallb/metallb \
Expand All @@ -147,51 +147,58 @@ install-lb:

install-cert-manager:
#!/usr/bin/env bash

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v{{certmanager_version}}/cert-manager.crds.yaml

helm upgrade --install cert-manager jetstack/cert-manager \
-n cert-manager --create-namespace \
--version {{certmanager_version}} \
--set installCRDs=false \
--set featureGates='AdditionalCertificateOutputFormats=true' \
--set featureGates='AdditionalCertificateOutputFormats=true,ExperimentalGatewayAPISupport=true' \
--set webhook.featureGates='AdditionalCertificateOutputFormats=true' \
--wait

kubectl apply -n cert-manager -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned
spec:
selfSigned: {}
EOF
kubectl apply -n cert-manager -f cert-manager/cluster_issuer.yaml
kubectl apply -n cert-manager -f cert-manager/ca.yaml

install-echo-service ingress_controller=default_ingress_controller:
kustomize build quickstarts/demo-app/overlays/{{ingress_controller}}-ingress | kubectl apply -f -
install-echo-service router=default_router:
kustomize build quickstarts/demo-app/overlays/{{router}} | kubectl apply -f -

install-echo-service-with-proxy:
kustomize build quickstarts/proxy-demo/ | kubectl apply -f -

install-heimdall ingress_controller=default_ingress_controller:
install-heimdall router=default_router:
#!/usr/bin/env bash
kubectl create namespace heimdall

kubectl apply -f quickstarts/heimdall/namespace.yaml

helm upgrade --install trust-manager jetstack/trust-manager \
-n heimdall \
--version {{trustmanager_version}} \
--set app.trust.namespace=heimdall \
--wait

kubectl apply -f quickstarts/heimdall/certificate.yaml

extraArgs='extraArgs={}'
if [[ "{{ingress_controller}}" == "contour" || "{{ingress_controller}}" == "emissary" ]]; then
if [[ "{{router}}" == "contour" || "{{router}}" == "emissary" || "{{router}}" == "envoygw" ]]; then
extraArgs='extraArgs={--envoy-grpc}'
fi

helm upgrade --install heimdall dadrus/heimdall \
helm upgrade --install heimdall ../../charts/heimdall \
-n heimdall \
-f quickstarts/heimdall/config.yaml \
-f quickstarts/heimdall/helm-values.yaml \
--set "${extraArgs}" \
--wait

if [ "{{ingress_controller}}" == "contour" ]; then
if [ "{{router}}" == "contour" ]; then
kubectl apply -f quickstarts/heimdall/contour-extension-service.yaml
elif [ "{{ingress_controller}}" == "emissary" ]; then
elif [ "{{router}}" == "emissary" ]; then
kubectl apply -f quickstarts/heimdall/emissary-auth-service.yaml
elif [ "{{router}}" == "envoygw" ]; then
kubectl apply -f quickstarts/heimdall/backend-tls-policy.yaml
kubectl apply -f quickstarts/heimdall/envoygw-security-policy.yaml
fi

create-cluster:
Expand All @@ -207,6 +214,8 @@ install-haproxy-decision-demo: setup-cluster install-haproxy-ingress-controller

install-emissary-decision-demo: setup-cluster install-emissary-ingress-controller (install-heimdall "emissary") (install-echo-service "emissary")

install-envoygw-decision-demo: setup-cluster install-envoy-gateway (install-heimdall "envoygw") (install-echo-service "envoygw")

delete-cluster:
kind delete clusters {{cluster_name}}

2 changes: 1 addition & 1 deletion examples/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This directory contains working examples described in the getting started, as well as in the integration guides of the documentation. The demonstration of the decision operation mode is done via integration with the corresponding ingress controllers. As of now, these are [Contour](https://projectcontour.io), the [NGINX Ingress Controller](https://docs.nginx.com/nginx-ingress-controller/) and [HAProxy Ingress Controller](https://haproxy-ingress.github.io/).

**Note:** The main branch may have breaking changes (see pending release PRs for details under https://github.com/dadrus/heimdall/pulls) which would make the usage of the referenced heimdall images impossible (even though the configuration files and rules reflect the latest changes). In such situations you'll have to build a heimdall image by yourself and update the setups to use it.
**Note:** The main branch may have breaking changes (see pending release PRs for details under https://github.com/dadrus/heimdall/pulls) which would make the usage of the referenced heimdall images impossible (even though the configuration files and rules reflect the latest changes). In such situations you'll have to use the dev image or build a heimdall image by yourself and update the setups to use it.

# Prerequisites

Expand Down
24 changes: 24 additions & 0 deletions examples/kubernetes/cert-manager/ca.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: demo-ca
namespace: cert-manager
spec:
isCA: true
commonName: demo-ca
secretName: demo-ca-secret
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: selfsigned-issuer
kind: ClusterIssuer
group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: demo-ca-issuer
spec:
ca:
secretName: demo-ca-secret
6 changes: 6 additions & 0 deletions examples/kubernetes/cert-manager/cluster_issuer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-issuer
spec:
selfSigned: {}
12 changes: 12 additions & 0 deletions examples/kubernetes/emissary/listener.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: emissary-tls-ingress-listener
namespace: emissary-ingress-controller
spec:
port: 8443
protocol: HTTPS
securityModel: XFP
hostBinding:
namespace:
from: ALL
Loading
Loading