Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster stuck in inconsistent state when nlb is deleted out of lifecycle. #347

Closed
sindhusri16 opened this issue Jan 29, 2024 · 2 comments · Fixed by #350
Closed

Cluster stuck in inconsistent state when nlb is deleted out of lifecycle. #347

sindhusri16 opened this issue Jan 29, 2024 · 2 comments · Fixed by #350
Labels
bug Something isn't working

Comments

@sindhusri16
Copy link
Member

What happened:
User manually deleted the nlb hosting the control plane of the cluster from console. Deletion of this cluster is stuck in an inconsistent state as nlb reconciler is always hitting a 404 not found when searching for this nlb.

What you expected to happen:
Cluster should not be stuck in 'deleting' phase, it could reconcile the nlb as deleted and go forward with deletion.

How to reproduce it (as minimally and precisely as possible):
Create a cluster and delete the nlb-apiserver from console, then delete the cluster.

Anything else we need to know?:
Logs:
kubectl get events -A | grep oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs
oke 14m Normal FailureDomainsReady ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs FailureDomain is in ready state
oke 48m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found....
oke 31m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found....
oke 14m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found.

Environment:

  • CAPOCI version: v0.11.5
  • Cluster-API version (use clusterctl version):
  • Kubernetes version (use kubectl version):
  • Docker version (use docker info):
  • OS (e.g. from /etc/os-release):
@sindhusri16 sindhusri16 added the bug Something isn't working label Jan 29, 2024
@sindhusri16
Copy link
Member Author

Could the team please confirm the ETA for this fix.

@shyamradhakrishnan
Copy link
Member

Approx ETA: 28/Feb/2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants