Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backup/restore functionality - PVC subset #392

Merged
merged 29 commits into from
Jun 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
1b756c9
Implement Backups/Restores first prototype using PVC
miguelsorianod Apr 29, 2020
5347ff1
Add restore wait for apimanager readiness and resync of zync routes
miguelsorianod May 4, 2020
db3311e
Remove obsolete APIManagerRestore status fields
miguelsorianod May 27, 2020
813e3e3
Remove unused reconcileRestoreAPIManagerFromPVCJob method
miguelsorianod May 27, 2020
ef7ef24
Remove unneeded job deletion on reconcileAPIManagerBackupSharedInSecr…
miguelsorianod May 27, 2020
977b6b5
Make OC CLI Image configurable
miguelsorianod May 26, 2020
aab7173
Add openapi-gen annotations in APIManagerBackup and APIManagerRestore
miguelsorianod May 27, 2020
6dda59a
Fix APIManagerRestore shared secret dependency between steps
miguelsorianod May 27, 2020
17f014c
Add APIManagerBackup and APIManagerRestore CRD tests
miguelsorianod May 27, 2020
917fce5
Rename backupSource to backupDestination in APIManagerBackup
miguelsorianod Jun 2, 2020
2573589
Update backup doc to instruct user to cleanup CR after backup
miguelsorianod Jun 2, 2020
584f1e6
Update APIManagerBackup and APIManagerRestore types documentation
miguelsorianod Jun 3, 2020
444caae
Add APIManagerRestore start and completion time information
miguelsorianod Jun 3, 2020
5e846a5
Update OLM-related files with APIManagerBackup/Restore changes
miguelsorianod Jun 3, 2020
f958d5f
Update basereconciler resource methods to get type information from o…
miguelsorianod Jun 3, 2020
80731e5
Add APIManagerBackup information in destination backup PVC annotations
miguelsorianod Jun 8, 2020
74170fa
Implement K8s Job Cleanup to completely free used PVCs
miguelsorianod Jun 8, 2020
7db9798
Fix duplicated log message word in UpdateResourceStatus
miguelsorianod Jun 8, 2020
06aeae4
Fix options not being passed in DeleteResource
miguelsorianod Jun 8, 2020
b820424
Update backup and restore documentation
miguelsorianod Jun 8, 2020
ea93443
Create APIManagerBackup/Restore unique predictable K8s job names from…
miguelsorianod Jun 7, 2020
b0c3e51
Remove duplicated pods get permission in role.yaml
miguelsorianod Jun 19, 2020
dc39071
remove apimanagerFromName dead code in apimanagerbackup options provider
miguelsorianod Jun 19, 2020
6f8e119
update reconcileJob comments to clarify why annotations and labels ar…
miguelsorianod Jun 19, 2020
a215c05
Remove clarified comment about apimanager being required as backup op…
miguelsorianod Jun 19, 2020
066082d
Remove unused S3 backup/restore procedures not used at the moment
miguelsorianod Jun 19, 2020
2a39639
Remove unused at the moment APIManagerbackup conditions field
miguelsorianod Jun 19, 2020
eb9e48b
Add explanatory comment on APIManager restore K8s deserializer
miguelsorianod Jun 19, 2020
5f68780
Update 3scale-operator role to allow any resource in apps.3scale.net
miguelsorianod Jun 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions deploy/crds/apps.3scale.net_apimanagerbackups_crd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: apimanagerbackups.apps.3scale.net
spec:
group: apps.3scale.net
names:
kind: APIManagerBackup
listKind: APIManagerBackupList
plural: apimanagerbackups
singular: apimanagerbackup
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: APIManagerBackup represents an APIManager backup
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: APIManagerBackupSpec defines the desired state of APIManagerBackup
properties:
backupDestination:
description: Backup data destination configuration
properties:
persistentVolumeClaim:
description: PersistentVolumeClaim as backup data destination configuration
properties:
resources:
description: Resources configuration for the backup data PersistentVolumeClaim.
Ignored when VolumeName field is set
properties:
requests:
description: 'Storage Resource requests to be used on the
PersistentVolumeClaim. To learn more about resource requests
see: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: string
required:
- requests
type: object
storageClass:
description: Storage class to be used by the PersistentVolumeClaim.
Ignored when VolumeName field is set
type: string
volumeName:
description: Name of an existing PersistentVolume to be bound
to the backup data PersistentVolumeClaim
type: string
type: object
type: object
required:
- backupDestination
type: object
status:
description: APIManagerBackupStatus defines the observed state of APIManagerBackup
properties:
apiManagerSourceName:
description: Name of the APIManager from which the backup has been performed
type: string
backupPersistentVolumeClaimName:
description: Name of the backup data PersistentVolumeClaim. Only set
when PersistentVolumeClaim is used as the backup data destination
type: string
completed:
description: Set to true when backup has been completed
type: boolean
completionTime:
description: Backup completion time. It is represented in RFC3339 form
and is in UTC.
format: date-time
type: string
mainStepsCompleted:
description: Set to true when main steps have been completed. At this
point backup still cannot be considered fully completed due to some
remaining post-backup tasks are pending (cleanup, ...)
type: boolean
startTime:
description: Backup start time. It is represented in RFC3339 form and
is in UTC.
format: date-time
type: string
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
101 changes: 101 additions & 0 deletions deploy/crds/apps.3scale.net_apimanagerrestores_crd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: apimanagerrestores.apps.3scale.net
spec:
group: apps.3scale.net
names:
kind: APIManagerRestore
listKind: APIManagerRestoreList
plural: apimanagerrestores
singular: apimanagerrestore
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: APIManagerRestore represents an APIManager restore
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: APIManagerRestoreSpec defines the desired state of APIManagerRestore
properties:
restoreSource:
description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
Important: Run "operator-sdk generate k8s" to regenerate code after
modifying this file Add custom validation using kubebuilder tags:
https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html'
properties:
persistentVolumeClaim:
description: Restore data soure configuration
properties:
claimSource:
description: PersistentVolumeClaim source of an existing PersistentVolumeClaim.
See
properties:
claimName:
description: 'ClaimName is the name of a PersistentVolumeClaim
in the same namespace as the pod using this volume. More
info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims'
type: string
readOnly:
description: Will force the ReadOnly setting in VolumeMounts.
Default false.
type: boolean
required:
- claimName
type: object
required:
- claimSource
type: object
type: object
required:
- restoreSource
type: object
status:
description: APIManagerRestoreStatus defines the observed state of APIManagerRestore
properties:
apiManagerToRestoreRef:
description: Name of the APIManager to be restored
properties:
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?'
type: string
type: object
completed:
description: Set to true when backup has been completed
type: boolean
completionTime:
description: Restore completion time. It is represented in RFC3339 form
and is in UTC.
format: date-time
type: string
mainStepsCompleted:
description: Set to true when main steps have been completed. At this
point restore still cannot be considered fully completed due to some
remaining post-backup tasks are pending (cleanup, ...)
type: boolean
startTime:
description: Restore start time. It is represented in RFC3339 form and
is in UTC.
format: date-time
type: string
type: object
type: object
version: v1alpha1
versions:
- name: v1alpha1
served: true
storage: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: apps.3scale.net/v1alpha1
kind: APIManagerBackup
metadata:
name: example-apimanagerbackup
spec:
backupDestination:
persistentVolumeClaim:
volumeName: "mypreexistingvolume"
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: apps.3scale.net/v1alpha1
kind: APIManagerRestore
metadata:
name: example-apimanagerrestore
spec:
restoreSource:
persistentVolumeClaim:
claimSource:
claimName: "mybackeduppvc"
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,36 @@ metadata:
"wildcardDomain": "\u003cdesired-domain\u003e"
}
},
{
"apiVersion": "apps.3scale.net/v1alpha1",
"kind": "APIManagerBackup",
"metadata": {
"name": "example-apimanagerbackup"
},
"spec": {
"backupDestination": {
"persistentVolumeClaim": {
"volumeName": "mypreexistingvolume"
}
}
}
},
{
"apiVersion": "apps.3scale.net/v1alpha1",
"kind": "APIManagerRestore",
"metadata": {
"name": "example-apimanagerrestore"
},
"spec": {
"restoreSource": {
"persistentVolumeClaim": {
"claimSource": {
"claimName": "mybackeduppvc"
}
}
}
}
},
{
"apiVersion": "capabilities.3scale.net/v1alpha1",
"kind": "API",
Expand Down Expand Up @@ -277,8 +307,18 @@ spec:
image: centos/postgresql-10-centos7
- name: "Zync Postgresql"
image: centos/postgresql-10-centos7
- name: "OpenShift CLI"
image: quay.io/openshift/origin-cli:4.2
customresourcedefinitions:
owned:
- description: APIManagerBackup represents an APIManager backup
kind: APIManagerBackup
name: apimanagerbackups.apps.3scale.net
version: v1alpha1
- description: APIManagerRestore represents an APIManager restore
kind: APIManagerRestore
name: apimanagerrestores.apps.3scale.net
version: v1alpha1
- description: APIManager is the Schema for the apimanagers API
displayName: APIManager
kind: APIManager
Expand Down Expand Up @@ -435,6 +475,8 @@ spec:
value: centos/postgresql-10-centos7
- name: ZYNC_POSTGRESQL_IMAGE
value: centos/postgresql-10-centos7
- name: OC_CLI_IMAGE
value: quay.io/openshift/origin-cli:4.2
image: quay.io/3scale/3scale-operator:master
name: 3scale-operator
resources: {}
Expand Down Expand Up @@ -496,9 +538,9 @@ spec:
- apiGroups:
- ""
resources:
- pods
- pods/exec
verbs:
- get
- create
- apiGroups:
- apps
resources:
Expand Down
2 changes: 2 additions & 0 deletions deploy/operator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,5 @@ spec:
value: "centos/postgresql-10-centos7"
- name: ZYNC_POSTGRESQL_IMAGE
value: "centos/postgresql-10-centos7"
- name: OC_CLI_IMAGE
value: "quay.io/openshift/origin-cli:4.2"
8 changes: 2 additions & 6 deletions deploy/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,6 @@ rules:
- configmaps
- secrets
- serviceaccounts
# For some reason there's an error creating serviceaccounts
# if you do not include permissions to bindings/finalizers.
# A related PR to this problem is:
# https://github.com/openshift/origin/pull/16253
- bindings/finalizers
verbs:
- create
Expand Down Expand Up @@ -63,9 +59,9 @@ rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
verbs:
- get
- create
- apiGroups:
- apps
resources:
Expand Down
Loading