docs: Add Carbon Efficient design document #4686
Conversation
✅ Deploy Preview for karpenter-docs-prod canceled.
|
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: Tim Bannister
Co-authored-by: Tim Bannister <tim@scalefactory.com>
|
@ellistarn and @sftim I have made some changes based on your feedback. You may re-review the design document if you want to :) |
|
I have changed the overall naming of this design proposal from Carbon Aware to Carbon Efficient. The reason behind this is that it is a more correct description of what we aim to achieve here. Carbon awareness is about using spatial and temporal shifting of workloads to reduce the carbon impact. We are not doing that. Instead we focus on reducing the carbon impact without shifting workloads by improving carbon efficiency. Therefore, the term carbon efficient is more suitable in this context. For more information, see Green Software Foundation, Section: Carbon Efficiency in Green Software Practitioner. However, I haven't change file name, because doing so would ruin the transparency/readability of the comments on this PR. |
JacobValdemar
changed the title
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
Status update (December 29, 2023)I am handing in my master's thesis within the next couple of days. Within 3 weeks I will publicly defend it at Aarhus University. After that I will share the master's thesis report with you. Therein you can read about the implementation of option 3 and an evaluation of the impact of using Carbon-Efficient Karpenter. Based on that we can discuss the next steps we want to take. The work we are doing can significantly reduce the environmental footprint of cloud-based Kubernetes clusters. I hope you are all still motivated, because I am! :D |
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
|
Dear Karpenter community, I am happy to say that I can finally share my master's thesis with you. In this thesis, a carbon efficient cluster autoscaler for Kubernetes is developed by adapting Karpenter to minimize cloud instance carbon emission estimates from BoaviztAPI. Specifically, this is an implementation of Option 3: Override instance price with carbon price described in this design document. The developed Carbon Efficient Karpenter shows a statistically significant median reduction in cloud instance carbon emissions when compared to the original Karpenter. I recommend that you read Chapter 7 (Design and Implementation) and Chapter 8 (Experiments and Results). Check it out! 🚀 https://github.com/JacobValdemar/carbon-efficient-karpenter-thesis |
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
|
Hey all. What would you think about merging the feature described in my master's thesis into current Karpenter Provider AWS? It will enable users to opt in for carbon efficient cluster autoscaling with no impact on existing users, very limited impact on the Karpenter design and almost no maintenance cost for Karpenter developers. Considering the large reduction potential described in my master's thesis (approx. 20% in Ireland and USA), what do you think about this idea? |
| ### Option 4: Enable custom instance price overrides | ||
| Enable administrators to configure custom instance price overrides, e.g. in a ConfigMap. A configuration using emission factors (varying with region and instance type) masked as prices can be pre-generated. Administrators then copy-paste a Carbon Efficient `priceOverride` into their environment. | ||
|
|
||
| ```yaml | ||
| priceOverrides: | ||
| - instanceType: "m5.large" | ||
| region: "eu-west-1" |
There was a problem hiding this comment.
Price overrides is interesting in multiple respects. For example there are reserved instances customers will reserve on azure and get a better contract for. So its actually cheaper for karpenter ot use those instances they got at a discount than the typical "retail pricing api" pricing data for the instance price.
So this field of pricing overrides can be used and extended to other usecases. I don't think overriding all existing instance types would be feasible or maintainable though.
There was a problem hiding this comment.
Yes, I have heard in multiple occasions that price overrides could be useful for various reasons.
In my Master's Thesis, overriding the price of all existing instance types is actually what I did. It is feasible and maintainable with enough automation. I think it makes sense in this context, but might be infeasible in other use cases.
| <summary>Change to `karpenter-core/pkg/controllers/deprovisioning/controller.go`</summary> | ||
|
|
||
| ```diff | ||
| -func NewController(clk clock.Clock, kubeClient client.Client, provisioner *provisioning.Provisioner, | ||
| - cp cloudprovider.CloudProvider, recorder events.Recorder, cluster *state.Cluster) *Controller { | ||
| +func NewController(ctx context.Context, clk clock.Clock, kubeClient client.Client, provisioner *provisioning.Provisioner, | ||
| + cp cloudprovider.CloudProvider, recorder events.Recorder, cluster *state.Cluster) *Controller { | ||
|
|
||
| + if settings.FromContext(ctx).CarbonEfficientEnabled { | ||
| + return &Controller{ | ||
| + clock: clk, | ||
| + kubeClient: kubeClient, | ||
| + cluster: cluster, | ||
| + provisioner: provisioner, | ||
| + recorder: recorder, | ||
| + cloudProvider: cp, | ||
| + lastRun: map[string]time.Time{}, | ||
| + deprovisioners: []Deprovisioner{ | ||
| + NewExpiration(clk, kubeClient, cluster, provisioner, recorder), | ||
| + NewDrift(kubeClient, cluster, provisioner, recorder), | ||
| + NewEmptiness(clk), | ||
| + NewEmptyMachineConsolidation(clk, cluster, kubeClient, provisioner, cp, recorder), | ||
| + NewCarbonEfficientMultiMachineConsolidation(clk, cluster, kubeClient, provisioner, cp, recorder), | ||
| + NewCarbonEfficientSingleMachineConsolidation(clk, cluster, kubeClient, provisioner, cp, recorder), | ||
| + }, | ||
| + } | ||
| + } |
There was a problem hiding this comment.
Code gets outdated in a design doc, this bit already is outdated with the karpenter-core reference.
| ### Option 2: Use Carbon Efficient filtering/sorting methods | ||
|
|
||
| #### Consolidation | ||
| Create carbon efficient implementations of low-level functions like `filterByPrice`, `filterOutSameType`, `getCandidatePrices`, etc. that are used when Carbon Efficient is enabled. Usage of aforementioned functions might assume that it is price that they are getting, but in reality it is data about carbon emissions. |
There was a problem hiding this comment.
Can you go into further depth where the data consolidation will use or consider? How do cloud providers provide this carbon aware data to karpenter core?
There was a problem hiding this comment.
The idea in that option was that these low-level methods would be "swappable" with carbon-aware methods. I now think that is a bad idea.
You ask how cloud providers would provide data on carbon emissions to Karpenter core. I don't know that. AWS doesn't provide no transparent and granular data about the carbon impact of servers so I guess the answer would be "they wouldn't". What I implemented in my master's thesis use BoaviztAPI to obtain estimates of the carbon impact of instance types.
My suggestion is that we move along with the solution proposed in my thesis because it is well-documented, already implemented (just needs adaption to new Karpenter version), has been thoroughly evaluated, and proves to result in firm carbon emission reductions.
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
|
Really no motivation for adding a carbon-efficient feature to Karpenter? |
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
Fixes #N/A
Description
I would really appreciate your support, feedback, and collaboration on this design document. Your involvement would be immensely valuable.
As described in RFC kubernetes-sigs/karpenter#675 and discussed on Slack, I am documenting a proposed design. This is a living document, and I expect your comments to necessitate changes to the current design before it is merged. I am still a new contributor to Karpenter, so while I have the best intentions, I lack complete technical insight; therefore, do not hesitate to correct me.
Together, we can make a significant impact on the sustainability of Kubernetes clusters!
How was this change tested?
Does this change impact docs?
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.