Switch to alternative for current kube-resources-autosave functionality

[whereisaaron]

reshifter appears to roughly the same job as kube-resources-autosave, backing up all resources records in a cluster. It has the advantage that it include restore functionality too. It backs up or restores to/from disk or direct to/from S3.

https://github.com/mhausenblas/reshifter

reshifter is different from kube-resources-autosave in that it backs up and restores records using the etcd API rather than the k8s apiserver API.

reshifter is different from etcd snapshot in that it is Kubernetes aware and you can do things like backup or restore individual k8s namespaces.

Anyway, interesting option I thought.

[cknowles]

There’s also https://github.com/heptio/ark, it uses the kube API and has some functionality to resync a cluster from backup if there are missing items.

[whereisaaron]

Thanks @c-knowles I saw the 'ark' kubecon presentation had already taken a look at the project. Their scope is much broader, in that they aim to back up PV volume contents too. But it is also at a fairly early stage I think. But if the resource backup/restore bit is solid, then yeah it's an option too.

Talking to the ark team they mentioned they were looking at 'stash' which automates restic with custom resources to backup PVs including hostpath and EFS volumes. Also an interesting project for us I think. https://github.com/appscode/stash

[whereisaaron]

I've been testing stash and it is pretty interesting. It works by injecting sidecar containers into workloads that run scheduled snapshots of pod-attached volumes (of any kind) using restic.

The interesting part is you can install stash as an admission controller and it will inject the sidecar into (selected) workloads as they are deployed by mutating the manifests. This means you can transparently provide cluster-managed backup of EBS volumes even in StatefulSets or DaemonSets.

[cknowles]

It appears ark will also have restic support soon - vmware-tanzu/velero#19. Perhaps we support both in kube-aws. For sure it'd be good to replace the bash scripting kube-aws has with a more complete app.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.