Skip to content

Commit

Permalink
Network policy recommendations
Browse files Browse the repository at this point in the history
Signed-off-by: Richard Wall <richard.wall@venafi.com>
  • Loading branch information
wallrj committed Nov 10, 2023
1 parent 564722a commit debf35b
Show file tree
Hide file tree
Showing 3 changed files with 134 additions and 0 deletions.
101 changes: 101 additions & 0 deletions content/docs/installation/best-practice.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,107 @@ are designed for backwards compatibility rather than for best practice or maximu
You may find that the default resources do not comply with the security policy on your Kubernetes cluster
and in that case you can modify the installation configuration using Helm chart values to override the defaults.

## Network Policy

An example of this recommendation is found in the Calico Documentation:
> We recommend creating an implicit default deny policy for your Kubernetes pods, regardless if you use Calico or Kubernetes network policy. This ensures that unwanted traffic is denied by default.
>
> 📖 [Calico Best practice: implicit default deny policy](https://docs.tigera.io/calico/latest/network-policy/get-started/kubernetes-default-deny#best-practice-implicit-default-deny-policy).
If you implement a default deny policy you will need to configure your CNI software
to allow the cert-manager components to connect to certain network resources.
Here is an overview of the network requirements:

1. **UDP: cert-manager (all) -> Kubernetes DNS**:
All cert-manager components perform UDP DNS queries for both cluster and external domain names.

1. **TCP: Kubernetes (API server) -> cert-manager (webhook)**:
The Kubernetes API server establishes HTTPS connections to the [cert-manager webhook component](../concepts/webhook.md).
Read the cert-manager [webhook troubleshooting guide](../troubleshooting/webhook.md)
to understand the webhook networking requirements.

1. **TCP: cert-manager (webhook, controller, cainjector, startupapicheck) -> Kubernetes API server**:
The cert-manager webhook, controller, cainjector and startupapicheck
establish HTTPS connections to the Kubernetes API server,
to interact with cert-manager custom resources and Kubernetes resources.
The cert-manager webhook is a special case;
it connects to the Kubernetes API server to use the `SubjectAccessReview` API,
to verify clients attempting to modify Approved or Denied conditions of CertificateRequest resources.

1. **TCP: cert-manager (controller) -> HashiCorp Vault (authentication and resource API endpoints)**:
The cert-manager controller may establish HTTPS connections to one or more Vault API endpoints,
if you are using the [Vault Issuer](../configuration/vault.md).
The target host and port of the Vault endpoints
are configured in the Issuer or ClusterIssuer resources.

1. **TCP: cert-manager (controller) -> Venafi TLS Protect (authentication and resource API endpoints)**:
The cert-manager controller may establish HTTPS connections to one or more Venafi API endpoints,
if you are using the [Venafi Issuer](../configuration/venafi.md).
The target host and port of the Venafi API endpoints
are configured in the Issuer or ClusterIssuer resources.

1. **TCP: cert-manager (controller) -> DNS API endpoints (for ACME DNS01)**:
The cert-manager controller may establish HTTPS connections to DNS API endpoints such as Amazon Route53,
and to any associated authentication endpoints,
if you are using the [ACME Issuer with DNS01 solvers](../configuration/acme/dns01/README.md#supported-dns01-providers).

1. **UDP / TCP: cert-manager (controller) -> External DNS**:
If you use the ACME Issuer, the cert-manager controller may send
DNS queries to recursive DNS servers,
as part of the ACME challenge self-check process.
It does this to ensure that the DNS01 or HTTP01 challenge is resolvable,
before asking the ACME server to perform its checks.

In the case of DNS01 it may also perform a series of DNS queries to authoritative DNS servers,
to compute the DNS zone in which to add the DNS01 challenge record.
In the case of DNS01, cert-manager also [supports DNS over HTTPS](../releases/release-notes/release-notes-1.13.md#dns-over-https-doh-support).

You can choose the host and port of the DNS servers, using the following [controller flags](../cli/controller.md):
`--acme-http01-solver-nameservers`,
`--dns01-recursive-nameservers`, and
`--dns01-recursive-nameservers-only`.

1. **TCP: ACME (Let's Encrypt) -> cert-manager (acmesolver)**:
If you use an ACME Issuer configured for HTTP01,
cert-manager will deploy an `acmesolver` Pod, a Service and an Ingress (or Gateway API) resource
in the namespace of the Issuer
or in the cert-manager namespace if it is a ClusterIssuer.
The ACME server will establish an HTTP connection to this Pod via your chosen ingress load balancer,
so your network policy must allow this.

> ℹ️ The acmesolver Pod **does not** require access to the Kubernetes API server.
### Calico Example

These examples use Calico custom resources.
You will need to adapt these for your CNI software.

#### Apply a default-deny policy to all *non-system* Pods

Use the [recommended Calico default deny policy](https://docs.tigera.io/calico/latest/network-policy/get-started/kubernetes-default-deny#enable-default-deny-calico-global-network-policy-non-namespaced).
This policy allows access to `kube-dns`,
which simplifies per Pod policies since you don't need to duplicate the DNS rules in every policy.
This policy also allows the Kubernetes API server to connect to the cert-manager webhook service,
if it is running in a Pod in the `kube-system` namespace.

> ℹ️ In Google GKE the Kubernetes API server runs outside the cluster.
🔗 <a href="deny-app-policy.yaml">`deny-app-policy.yaml`</a>
```yaml file=../../../public/docs/installation/best-practice/deny-app-policy.yaml
```

#### Allow all cert-manager components to connect to the Kubernetes API server

This policy allows access to the Kubernetes API server
by reference to the standard `kubernetes` Service resource which is found in the `default` namespace.

🔗 <a href="allow-egress-kubernetes-api.yaml">`allow-egress-kubernetes-api.yaml`</a>
```yaml file=../../../public/docs/installation/best-practice/allow-egress-kubernetes-api.yaml
```

> ℹ️ The standard [Kubernetes NetworkPolicy resources do not support Targeting of services by name](https://kubernetes.io/docs/concepts/services-networking/network-policies/#what-you-can-t-do-with-network-policies-at-least-not-yet).
> Instead you will need to [use IP address of the Kubernetes API server](https://stackoverflow.com/a/56494510), but this is fragile because the IP address may change.
## Isolate cert-manager on dedicated node pools

cert-manager is a cluster scoped operator and you should treat it as part of your platform's control plane.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# allow-egress-kubernetes-api.yaml
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
name: allow-egress-kubernetes-api
namespace: cert-manager
spec:
types:
- Egress
egress:
- action: Allow
destination:
services:
name: kubernetes
namespace: default
18 changes: 18 additions & 0 deletions public/docs/installation/best-practice/deny-app-policy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# deny-app-policy.yaml
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: deny-app-policy
spec:
namespaceSelector: has(projectcalico.org/name) && projectcalico.org/name not in {"kube-system", "calico-system", "calico-apiserver"}
types:
- Ingress
- Egress
egress:
# allow all namespaces to communicate to DNS pods
- action: Allow
protocol: UDP
destination:
selector: 'k8s-app == "kube-dns"'
ports:
- 53

0 comments on commit debf35b

Please sign in to comment.