If cluster creation fails at a certain point, it cannot be deleted

[Callisto13]

What happened:
It's a weird one, I have only seen it a couple of times and I am not 100% sure of the exact sequence of events which makes it happen.

In this case I had forgotten to start my (local) Flintlock server before creating a (local) CAPMVM cluster, so I saw the error

Error while dialing dial tcp 192.168.0.31:9091: connect: connection refused

Which fine, I started the flintlock server, and ran kubectl delete clusters.cluster.x-k8s.io mvm-test to get a clean run of things.

The delete does not seem to be picked up by the capmvm-controller, and 15 mins later it is still hanging.

What did you expect to happen:
The cluster should be deleted no matter what stage it is in creation.

I am able to delete the cluster on other random failures.

How to reproduce it:

• Start CAPMVM in tilt
• Do not start a flintlock server
• Create a cluster setting the host to something which is not running
• See it fail
• Try to delete the cluster

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• capmvm version: head at 243c19a
• OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="21.04 (Hirsute Hippo)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 21.04"
VERSION_ID="21.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-pol
icy"
VERSION_CODENAME=hirsute
UBUNTU_CODENAME=hirsute