Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Opencost integration #995

Merged
merged 8 commits into from
Sep 26, 2023
Merged

Opencost integration #995

merged 8 commits into from
Sep 26, 2023

Conversation

Azanul
Copy link
Collaborator

@Azanul Azanul commented Sep 25, 2023

Problem

#801

Solution

Used Opencost for cost calculation of k8s components

Changes Made

  • Added a new field opencostBaseUrl in k8s config
  • Used opencostBaseUrl for opencost integration

Notes

  • Assumed that Opencost is already correctly configure with the cluster.
  • Do pay attention to the way opencostBaseUrl is parsed from config, I'm not sure if that's the preferred way of doing this.
  • Fixes Integrate OpenCost for Kubernetes clusters #801

Checklist

  • Code follows the contributing guidelines
  • Changes have been thoroughly tested
  • Documentation has been updated, if necessary
  • Any dependencies have been added to the project, if necessary

Reviewers

@mlabouardy
@jakepage91

@Azanul
go mod tidy
6dcac82 
Signed-off-by: Azanul <azanulhaque@gmail.com>
@Azanul
opencost integration
2b8d55b 
Signed-off-by: Azanul <azanulhaque@gmail.com>
@Azanul
opencost url through config
2f628ff 
Signed-off-by: Azanul <azanulhaque@gmail.com>
@Azanul
k8s pod cost calculation
aa1ad26 
Signed-off-by: Azanul <azanulhaque@gmail.com>
mlabouardy
mlabouardy requested changes Sep 26, 2023
View reviewed changes
Copy link
Collaborator

@mlabouardy mlabouardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @Azanul, this a major feature for Komiser 👏🏻
I've left few comments :)

Azanul and others added 2 commits September 26, 2023 14:32
@Azanul Azanul requested a review from mlabouardy September 26, 2023 09:03
@mlabouardy
Copy link
Collaborator

Thanks for the quick changes @Azanul :)

@AvineshTripathi
Copy link
Collaborator

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

@mlabouardy
Copy link
Collaborator

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?

komiser/providers/k8s/core/pods.go

Line 29 in c8237dc

res, err := client.K8sClient.Client.CoreV1().Pods("").List(ctx, config)

@Azanul
Copy link
Collaborator Author

Azanul commented Sep 26, 2023 

$ kubectl get ns
NAME              STATUS   AGE
default           Active   28h
kube-node-lease   Active   28h
kube-public       Active   28h
kube-system       Active   28h
opencost          Active   28h
prometheus        Active   28h

Opencost "pod" response for a local minikube cluster

{
  "code": 200,
  "status": "success",
  "data": [
    {
      "coredns-5d78c9869d-2ktxj": {
        "name": "coredns-5d78c9869d-2ktxj",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "coredns",
          "controller": "coredns",
          "controllerKind": "deployment",
          "namespace": "kube-system",
          "pod": "coredns-5d78c9869d-2ktxj",
          "services": [
            "kube-dns"
          ],
          "labels": {
            "k8s_app": "kube-dns",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_hash": "5d78c9869d"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 73400320,
        "ramByteRequestAverage": 73400320,
        "ramByteUsageAverage": 0,
        "ramByteHours": 1773841066.66667,
        "ramCost": 0.007,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08339,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "etcd-minikube": {
        "name": "etcd-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "etcd",
          "namespace": "kube-system",
          "pod": "etcd-minikube",
          "labels": {
            "component": "etcd",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 104857600,
        "ramByteRequestAverage": 104857600,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2534058666.66667,
        "ramCost": 0.01,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-apiserver-minikube": {
        "name": "kube-apiserver-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-apiserver",
          "namespace": "kube-system",
          "pod": "kube-apiserver-minikube",
          "labels": {
            "component": "kube-apiserver",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.25,
        "cpuCoreRequestAverage": 0.25,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 6.04167,
        "cpuCost": 0.19098,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.19098,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-controller-manager-minikube": {
        "name": "kube-controller-manager-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-controller-manager",
          "namespace": "kube-system",
          "pod": "kube-controller-manager-minikube",
          "labels": {
            "component": "kube-controller-manager",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.2,
        "cpuCoreRequestAverage": 0.2,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 4.83333,
        "cpuCost": 0.15279,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.15279,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-proxy-f6clv": {
        "name": "kube-proxy-f6clv",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-proxy",
          "controller": "kube-proxy",
          "controllerKind": "daemonset",
          "namespace": "kube-system",
          "pod": "kube-proxy-f6clv",
          "labels": {
            "controller_revision_hash": "56999f657b",
            "k8s_app": "kube-proxy",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-scheduler-minikube": {
        "name": "kube-scheduler-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-scheduler",
          "namespace": "kube-system",
          "pod": "kube-scheduler-minikube",
          "labels": {
            "component": "kube-scheduler",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.07639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls": {
        "name": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-state-metrics",
          "controller": "my-prometheus-kube-state-metrics",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
          "services": [
            "my-prometheus-kube-state-metrics"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "kube-state-metrics",
            "app_kubernetes_io_part_of": "kube-state-metrics",
            "app_kubernetes_io_version": "2.10.0",
            "helm_sh_chart": "kube-state-metrics-5.13.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_hash": "6c4b65c7b9"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-prometheus-node-exporter-gsr4h": {
        "name": "my-prometheus-prometheus-node-exporter-gsr4h",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "node-exporter",
          "controller": "my-prometheus-prometheus-node-exporter",
          "controllerKind": "daemonset",
          "namespace": "prometheus",
          "pod": "my-prometheus-prometheus-node-exporter-gsr4h",
          "services": [
            "my-prometheus-prometheus-node-exporter"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "prometheus-node-exporter",
            "app_kubernetes_io_part_of": "prometheus-node-exporter",
            "app_kubernetes_io_version": "1.6.1",
            "controller_revision_hash": "769f45ff7c",
            "helm_sh_chart": "prometheus-node-exporter-4.23.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-server-6fffb9f-gfjxf": {
        "name": "my-prometheus-server-6fffb9f-gfjxf",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "my-prometheus-server",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-server-6fffb9f-gfjxf",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 8589934592,
        "pvByteHours": 207590085973.3333,
        "pvCost": 0.01059,
        "pvs": {
          "cluster=cluster-one:name=pvc-8a9fc731-530d-4344-a574-f4c8cb5c0f91": {
            "byteHours": 207590085973.3333,
            "cost": 0.0105936072
          }
        },
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.01059,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "opencost-785545d6d7-w5bj4": {
        "name": "opencost-785545d6d7-w5bj4",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "opencost",
          "controllerKind": "deployment",
          "namespace": "opencost",
          "pod": "opencost-785545d6d7-w5bj4",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "opencost"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.02,
        "cpuCoreRequestAverage": 0.02,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0.48333,
        "cpuCost": 0.01528,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 110000000,
        "ramByteRequestAverage": 110000000,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2658333333.33333,
        "ramCost": 0.01049,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.02577,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "storage-provisioner": {
        "name": "storage-provisioner",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "storage-provisioner",
          "namespace": "kube-system",
          "pod": "storage-provisioner",
          "labels": {
            "addonmanager_kubernetes_io_mode": "Reconcile",
            "integration_test": "storage-provisioner",
            "kubernetes_io_metadata_name": "kube-system"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      }
    }
  ]
}

ref: https://www.opencost.io/docs/integrations/api

@Azanul
Copy link
Collaborator Author

Azanul commented Sep 26, 2023

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?

komiser/providers/k8s/core/pods.go

Line 29 in c8237dc

res, err := client.K8sClient.Client.CoreV1().Pods("").List(ctx, config)

Correct

@mlabouardy mlabouardy added this to the v3.1.1 milestone Sep 26, 2023
@mlabouardy mlabouardy merged commit b51ff81 into tailwarden:develop Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mlabouardy mlabouardy mlabouardy approved these changes

@ShubhamPalriwala ShubhamPalriwala Awaiting requested review from ShubhamPalriwala

@AvineshTripathi AvineshTripathi Awaiting requested review from AvineshTripathi AvineshTripathi is a code owner

Assignees
No one assigned
Labels
kubernetes
Projects
None yet
Milestone
v3.1.1
Development

Successfully merging this pull request may close these issues.

Integrate OpenCost for Kubernetes clusters
3 participants
@Azanul @mlabouardy @AvineshTripathi