Bug: Zarf fails to init successfully if there's a pod that's currently stuck in terminating state and the node on that terminating pod doesn't exist anymore

[zack-is-cool]

Zarf fails to init successfully if there's a pod that's currently stuck in terminating state and the node on that terminating pod doesn't exist anymore.

I think this chunk of code is related - it doesn't seem to check if a pod is in a bad state, i.e. terminating and possibly the node that it was supposed to live on is now gone 😬

zarf/src/pkg/cluster/injector.go

Lines 453 to 466 in 08c92e1

	findImages:
	for _, pod := range pods.Items {
	nodeName := pod.Spec.NodeName
	// If this pod doesn't have a node (i.e. is Pending), skip it
	if nodeName == "" {
	continue
	}
	nodeDetails, err := c.GetNode(nodeName)
	if err != nil {
	return nil, fmt.Errorf("unable to get the node %s", pod.Spec.NodeName)
	}

I know this is a very specific scenario, but it happened -

• We had an EKS cluster and were upgrading it from 1.27 -> 1.29
• Rolled all node and eks cluster versions to 1.29
• Pod stuck in terminating
• This terminating pod.Spec.NodeName has a node on it that doesn't exist anymore because nodes were rolled to get new eks k8s version
• Zarf tries to run GetNode on this node from k8s api, that errors out
• Zarf prints error: Unable to generate a list of candidate images to perform the registry injection

[JaseKoonce]

Zarf init error:

The Node listed in the log is the same one the the stuck pod is "scheduled" to but it no longer exists