Opencost integration #995

Azanul · 2023-09-25T18:03:29Z

Problem

#801

Solution

Used Opencost for cost calculation of k8s components

Changes Made

Added a new field opencostBaseUrl in k8s config
Used opencostBaseUrl for opencost integration

Notes

Assumed that Opencost is already correctly configure with the cluster.
Do pay attention to the way opencostBaseUrl is parsed from config, I'm not sure if that's the preferred way of doing this.
Fixes Integrate OpenCost for Kubernetes clusters #801

Checklist

Code follows the contributing guidelines
Changes have been thoroughly tested
Documentation has been updated, if necessary
Any dependencies have been added to the project, if necessary

Reviewers

@mlabouardy
@jakepage91

Signed-off-by: Azanul <azanulhaque@gmail.com>

mlabouardy

Great job @Azanul, this a major feature for Komiser 👏🏻
I've left few comments :)

providers/k8s/core/pods.go

Signed-off-by: Azanul <azanulhaque@gmail.com>

mlabouardy · 2023-09-26T09:04:19Z

Thanks for the quick changes @Azanul :)

AvineshTripathi · 2023-09-26T09:06:12Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

mlabouardy · 2023-09-26T09:10:48Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?

komiser/providers/k8s/core/pods.go

Line 29 in c8237dc

res, err := client.K8sClient.Client.CoreV1().Pods("").List(ctx, config)

Azanul · 2023-09-26T09:14:54Z

$ kubectl get ns
NAME              STATUS   AGE
default           Active   28h
kube-node-lease   Active   28h
kube-public       Active   28h
kube-system       Active   28h
opencost          Active   28h
prometheus        Active   28h

Opencost "pod" response for a local minikube cluster

{
  "code": 200,
  "status": "success",
  "data": [
    {
      "coredns-5d78c9869d-2ktxj": {
        "name": "coredns-5d78c9869d-2ktxj",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "coredns",
          "controller": "coredns",
          "controllerKind": "deployment",
          "namespace": "kube-system",
          "pod": "coredns-5d78c9869d-2ktxj",
          "services": [
            "kube-dns"
          ],
          "labels": {
            "k8s_app": "kube-dns",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_hash": "5d78c9869d"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 73400320,
        "ramByteRequestAverage": 73400320,
        "ramByteUsageAverage": 0,
        "ramByteHours": 1773841066.66667,
        "ramCost": 0.007,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08339,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "etcd-minikube": {
        "name": "etcd-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "etcd",
          "namespace": "kube-system",
          "pod": "etcd-minikube",
          "labels": {
            "component": "etcd",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 104857600,
        "ramByteRequestAverage": 104857600,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2534058666.66667,
        "ramCost": 0.01,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-apiserver-minikube": {
        "name": "kube-apiserver-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-apiserver",
          "namespace": "kube-system",
          "pod": "kube-apiserver-minikube",
          "labels": {
            "component": "kube-apiserver",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.25,
        "cpuCoreRequestAverage": 0.25,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 6.04167,
        "cpuCost": 0.19098,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.19098,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-controller-manager-minikube": {
        "name": "kube-controller-manager-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-controller-manager",
          "namespace": "kube-system",
          "pod": "kube-controller-manager-minikube",
          "labels": {
            "component": "kube-controller-manager",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.2,
        "cpuCoreRequestAverage": 0.2,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 4.83333,
        "cpuCost": 0.15279,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.15279,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-proxy-f6clv": {
        "name": "kube-proxy-f6clv",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-proxy",
          "controller": "kube-proxy",
          "controllerKind": "daemonset",
          "namespace": "kube-system",
          "pod": "kube-proxy-f6clv",
          "labels": {
            "controller_revision_hash": "56999f657b",
            "k8s_app": "kube-proxy",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-scheduler-minikube": {
        "name": "kube-scheduler-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-scheduler",
          "namespace": "kube-system",
          "pod": "kube-scheduler-minikube",
          "labels": {
            "component": "kube-scheduler",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.07639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls": {
        "name": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-state-metrics",
          "controller": "my-prometheus-kube-state-metrics",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
          "services": [
            "my-prometheus-kube-state-metrics"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "kube-state-metrics",
            "app_kubernetes_io_part_of": "kube-state-metrics",
            "app_kubernetes_io_version": "2.10.0",
            "helm_sh_chart": "kube-state-metrics-5.13.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_hash": "6c4b65c7b9"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-prometheus-node-exporter-gsr4h": {
        "name": "my-prometheus-prometheus-node-exporter-gsr4h",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "node-exporter",
          "controller": "my-prometheus-prometheus-node-exporter",
          "controllerKind": "daemonset",
          "namespace": "prometheus",
          "pod": "my-prometheus-prometheus-node-exporter-gsr4h",
          "services": [
            "my-prometheus-prometheus-node-exporter"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "prometheus-node-exporter",
            "app_kubernetes_io_part_of": "prometheus-node-exporter",
            "app_kubernetes_io_version": "1.6.1",
            "controller_revision_hash": "769f45ff7c",
            "helm_sh_chart": "prometheus-node-exporter-4.23.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-server-6fffb9f-gfjxf": {
        "name": "my-prometheus-server-6fffb9f-gfjxf",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "my-prometheus-server",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-server-6fffb9f-gfjxf",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 8589934592,
        "pvByteHours": 207590085973.3333,
        "pvCost": 0.01059,
        "pvs": {
          "cluster=cluster-one:name=pvc-8a9fc731-530d-4344-a574-f4c8cb5c0f91": {
            "byteHours": 207590085973.3333,
            "cost": 0.0105936072
          }
        },
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.01059,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "opencost-785545d6d7-w5bj4": {
        "name": "opencost-785545d6d7-w5bj4",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "opencost",
          "controllerKind": "deployment",
          "namespace": "opencost",
          "pod": "opencost-785545d6d7-w5bj4",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "opencost"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.02,
        "cpuCoreRequestAverage": 0.02,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0.48333,
        "cpuCost": 0.01528,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 110000000,
        "ramByteRequestAverage": 110000000,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2658333333.33333,
        "ramCost": 0.01049,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.02577,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "storage-provisioner": {
        "name": "storage-provisioner",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "storage-provisioner",
          "namespace": "kube-system",
          "pod": "storage-provisioner",
          "labels": {
            "addonmanager_kubernetes_io_mode": "Reconcile",
            "integration_test": "storage-provisioner",
            "kubernetes_io_metadata_name": "kube-system"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      }
    }
  ]
}

ref: https://www.opencost.io/docs/integrations/api

Azanul · 2023-09-26T09:15:34Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?

komiser/providers/k8s/core/pods.go

Line 29 in c8237dc

res, err := client.K8sClient.Client.CoreV1().Pods("").List(ctx, config)

Correct

Azanul added 4 commits September 25, 2023 18:59

go mod tidy

6dcac82

Signed-off-by: Azanul <azanulhaque@gmail.com>

opencost integration

2b8d55b

Signed-off-by: Azanul <azanulhaque@gmail.com>

opencost url through config

2f628ff

Signed-off-by: Azanul <azanulhaque@gmail.com>

k8s pod cost calculation

aa1ad26

Signed-off-by: Azanul <azanulhaque@gmail.com>

Azanul requested review from mlabouardy, ShubhamPalriwala and AvineshTripathi as code owners September 25, 2023 18:03

Merge branch 'develop' into opencost-integration

bd719c9

mlabouardy requested changes Sep 26, 2023

View reviewed changes

providers/k8s/core/pods.go Outdated Show resolved Hide resolved

providers/k8s/core/pods.go Outdated Show resolved Hide resolved

Azanul and others added 2 commits September 26, 2023 14:32

opencost made optional

248fc97

Signed-off-by: Azanul <azanulhaque@gmail.com>

Merge branch 'develop' into opencost-integration

e6b7884

Azanul requested a review from mlabouardy September 26, 2023 09:03

mlabouardy approved these changes Sep 26, 2023

View reviewed changes

Merge branch 'develop' into opencost-integration

c8237dc

mlabouardy added this to the v3.1.1 milestone Sep 26, 2023

mlabouardy added the kubernetes label Sep 26, 2023

mlabouardy merged commit b51ff81 into tailwarden:develop Sep 26, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Opencost integration #995

Opencost integration #995

Azanul commented Sep 25, 2023

mlabouardy left a comment

mlabouardy commented Sep 26, 2023

AvineshTripathi commented Sep 26, 2023

mlabouardy commented Sep 26, 2023

Azanul commented Sep 26, 2023

Azanul commented Sep 26, 2023

Opencost integration #995

Opencost integration #995

Conversation

Azanul commented Sep 25, 2023

Problem

Solution

Changes Made

Notes

Checklist

Reviewers

mlabouardy left a comment

Choose a reason for hiding this comment

mlabouardy commented Sep 26, 2023

AvineshTripathi commented Sep 26, 2023

mlabouardy commented Sep 26, 2023

Azanul commented Sep 26, 2023

Azanul commented Sep 26, 2023