Self pod discovery does not respect pod terminating state

[nefelim4ag]

What happened:
With disabled publishService and used pod self-discovery of Node IP.
With more than one ingress-nginx pod in Deployment or DaemonSet.
While updating or if you manually delete the pod. While terminating pod will not be removed from ingress LoadBalancer status.

With current lifecycle hook /wait-shutdown controller will stop answer "ready" status and forwarded signal will stop Nginx.
So pod effectively will answer connection refused and NotReady.
If you try duct tape that behavior, with LifeCycle preStop hook like sleep 120, that will not work. Pod will be in Terminating & Ready state, so it still be in Ingress LoadBalancer status.

What you expected to happen:
Pod statuses are correctly handled.

Issue is here.
Controller only checks pod conditions, but Terminating state is special.
Correctly it must be handled like:

if v.GetDeletionTimestamp() == nil {
  // Pod not in terminating state
}

external-dns patch

How to reproduce this issue:
Deploy ingress-nginx helm:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm template --create-namespace -n test ingress-nginx ingress-nginx/ingress-nginx \
  --set controller.publishService.enabled=false \
  --set controller.kind=Deployment \
  --set "controller.lifecycle.preStop.exec.command={sh,-c,sleep 120; kill 1}"

Remove any pod or trigger kubectl -n ingress-nginx rollout restart deployment ingress-nginx-controller

Example ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test
spec:
  ingressClassName: nginx
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ingress-nginx-controller-metrics
                port:
                  name: metrics

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

could you write a step by step guide for someone to copy/paste on a minikube or kind cluster. You have left too much info out of the description so readers have to make assumptions and that may not be accurate as per your use case.

/remove-kind bug

[longwuyuan]

/triage needs-information

[nefelim4ag]

@longwuyuan updated