Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a concrete example on how to restore a cert-manager installation #1353

Merged
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 36 additions & 6 deletions content/docs/devops-tips/backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,18 +101,16 @@ to a situation where updates to the `Ingress` (i.e a new DNS name) are not
applied to the `Certificate`.

To avoid this issue, in most cases `Certificate`s created via `ingress-shim`
can be excluded from the backup. Given that the restore happens
should be excluded from the backup. Given that the restore happens
in the correct order (`Secret` with the X.509 certificate restored before
the `Ingress`) `cert-manager` will be able to create a new `Certificate`
for the `Ingress` and determine that the existing `Secret` is for that `Certificate`.

### Velero

We have briefly tested backup and restore with `velero` `v1.5.3` and
`cert-manager` versions `v1.3.1` and `v1.3.0` as well as `velero` `v1.3.1`
and `cert-manager` `v1.1.0`.
We have tested backup and restore with `velero` `v1.12.2` and `cert-manager` version `v1.13.2`.

A few potential edge cases:
A few potential edge cases:

- Ensure that the backups include `cert-manager` CRDs.
For example, we have seen that if `--exclude-namespaces` flag is passed to
Expand All @@ -124,7 +122,7 @@ We have briefly tested backup and restore with `velero` `v1.5.3` and
exclude `Order`s, `Challenge`s and `CertificateRequest`s from the backup, see
[Excluding some cert-manager resources from backup](#excluding-some-cert-manager-resources-from-backup).

- Velero's [default restore order](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/server/server.go#L470)(`Secrets` before `Ingress`es, Custom Resources
- Velero's [default restore order](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/server/server.go#L470) (`Secrets` before `Ingress`es, Custom Resources
restored last), should ensure that there is no unnecessary certificate reissuance
due to the order of restore operation, see [Order of restore](#order-of-restore).
inteon marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -140,6 +138,38 @@ We have briefly tested backup and restore with `velero` `v1.5.3` and
`Certificate`s created for `Ingress`es from the backup even when not
re-creating the `Ingress` itself. See [Restoring Ingress Certificates](#restoring-ingress-certificates).


#### Example backup and restore using Velero

The following command will create a backup of all Kubernetes resources in the
default and cert-manager namespaces, excluding `Order`s, `Challenge`s and
`CertificateRequest`s (see above):
```bash
velero backup create \
full-backup \
--include-namespaces cert-manager,default \
--include-cluster-resources=true \
--exclude-resources challenges.acme.cert-manager.io,orders.acme.cert-manager.io,certificaterequests.cert-manager.io
```

We recommend that you restore the backup in two steps, first restoring the
`Secret`s and `Ingress`es and the `cert-manager` deployment, and then restoring
the rest of the resources. This is because `cert-manager`'s controller needs to
be able to create `Certificate` for the ingresses, so the owner reference is set.
inteon marked this conversation as resolved.
Show resolved Hide resolved

1. Restore everything except `Certificate` resources:
```bash
velero restore create \
--from-backup full-backup \
--exclude-resources certificates.cert-manager.io
```

2. Wait for cert-manager to create the `Certificate`s for the `Ingress`es (if cert-manager is having RBAC/ webhook issues, you might have to manually restart the deployments). After the auto-generated `Certificate`s are created, restore the manually created `Certificate`s:
```bash
velero restore create \
--from-backup full-backup
```

## Backing up CertificateRequests

We no longer recommend including `CertificateRequest` resources in a backup
Expand Down