Emit pod status metrics for pending pods from api server

[mitali-salvi]

Description:
Pending pod metrics are not being emitted from the pod store due to cadvisor running on individual node and since pending pods are not assigned to any node, they dont show up

Testing:

kubectl get pods -n prometheus    | grep Pending                                     
NAME                                                READY   STATUS    RESTARTS       AGE
prometheus-alertmanager-0                           0/1     Pending   0              98d
prometheus-server-8486b7c658-bzvxs                  0/2     Pending   0              98d

{
  "CloudWatchMetrics":
    [
      {
        "Namespace": "ContainerInsights",
        "Dimensions":
          [
            ["ClusterName", "Namespace", "PodName"],
            ["ClusterName"],
            ["ClusterName", "FullPodName", "Namespace", "PodName"],
          ],
        "Metrics":
          [
            { "Name": "pod_status_scheduled", "Unit": "Count" },
            { "Name": "pod_status_unknown", "Unit": "Count" },
            { "Name": "pod_status_pending", "Unit": "Count" },
            { "Name": "pod_status_running", "Unit": "Count" },
            { "Name": "pod_status_succeeded", "Unit": "Count" },
            { "Name": "pod_status_failed", "Unit": "Count" },
            { "Name": "pod_status_ready", "Unit": "Count" },
          ],
      },
    ],
  "ClusterName": "cloudwatchagent-operator",
  "FullPodName": "prometheus-alertmanager-0",
  "InstanceId": "i-01e21d00258711849",
  "InstanceType": "t3.small",
  "Namespace": "prometheus",
  "NodeName": "ip-192-168-35-27.ec2.internal",
  "PodName": "prometheus-alertmanager-0",
  "Sources": ["apiserver"],
  "Timestamp": "1699491151016",
  "Type": "Pod",
  "Version": "0",
  "kubernetes":
    {
      "host": "ip-192-168-35-27.ec2.internal",
      "labels":
        {
          "app.kubernetes.io/instance": "prometheus",
          "app.kubernetes.io/name": "alertmanager",
          "controller-revision-hash": "prometheus-alertmanager-5f59cbb8f9",
          "statefulset.kubernetes.io/pod-name": "prometheus-alertmanager-0",
        },
      "namespace_name": "prometheus",
      "pod_id": "ad964342-9ff9-4d4a-937a-8c8ad69cd038",
      "pod_name": "prometheus-alertmanager-0",
      "pod_owners":
        [
          {
            "owner_kind": "StatefulSet",
            "owner_name": "prometheus-alertmanager",
          },
        ],
    },
  "pod_status": "Pending",
  "pod_status_failed": 0,
  "pod_status_pending": 1,
  "pod_status_ready": 0,
  "pod_status_running": 0,
  "pod_status_scheduled": 0,
  "pod_status_succeeded": 0,
  "pod_status_unknown": 0,
}

[sky333999]

nit: Can we use the option pattern so the signature doesnt have to change everywhere.

[mitali-salvi]

this will be taken care of as part of the code cleanup task