extend activestandby taskimage to safely handle ingresses which have Fastly annotations

[twardnw]

No description provided.

[shreddedbacon]

Just for visibility.

The main thing that would need to be done is have the task search for this label on any ingress objects before it performs the creation of the HostMigration CRD.

dioscuri.amazee.io/migrate=true

Then check if the annotation fastly.amazee.io/watch exists (true and false, the label MUST exist, this is to cover anything that may be disabled for a reason).
If that annotation exists, then check the current values of the following annotations, and patch them with the following. (maybe store the original values in a seperate annotation like fastly.amazee.io/X-backup)

fastly.amazee.io/delete-external-resources=false
fastly.amazee.io/paused=true
fastly.amazee.io/watch=false

This needs to be done in both the source and destination namespaces.

Then the task would perform the host-migration task as normal.

Once the task is complete, then the annotations need to be reverted (use the value from X-backup, then delete the X-backup annotations)

[shreddedbacon]

Theoretically, we could perform the entire migration inside of this task and remove the need for the dioscuri operator entirely...
Edit: nope, the user that the task runs, has no permission to perform changes in the other namespace that it needs to do migrations with:

Task failed to get the object list from kubernetes, error was: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:ci-active-standby-control-k8s-master-b:lagoon-deployer" cannot list resource "ingresses" in API group "networking.k8s.io" in the namespace "ci-active-standby-control-k8s-master-a"

[shreddedbacon]

Working on initial support in dioscuri for this here: amazeeio/dioscuri#19

For now, the best thing would be to not do any annotation or labelling within cluster so that the ingress are not aware of the fastly-controller. So manually uploading certificates, and manually adding the domains to the required services.

[shreddedbacon]

The linked issue would solve this by adding the new label activestandby.lagoon.sh/migrating=true when performing the migration. This means that other systems that rely on changes to ingress can be aware of when the process takes place (the fastly-controller for amazeeio) and not perform the actions it normally would.