Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cluster resource modeling api #2386

Merged
merged 1 commit into from
Aug 18, 2022
Merged

Add cluster resource modeling api #2386

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions api/openapi-spec/swagger.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14054,6 +14054,22 @@
}
}
},
"com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.AllocatableModeling": {
"description": "AllocatableModeling represents the number of nodes in which allocatable resources in a specific resource model grade. E.g. AllocatableModeling{Grade: 2, Count: 10} means 10 nodes belong to resource model in grade 2.",
"type": "object",
"properties": {
"count": {
"description": "Count is the number of nodes that own the resources delineated by this modeling.",
"type": "integer",
"format": "int32"
},
"grade": {
"description": "Grade is the index of ResourceModel.",
"type": "integer",
"format": "int32"
}
}
},
"com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.Cluster": {
"description": "Cluster represents the desire state and status of a member cluster.",
"type": "object",
Expand Down Expand Up @@ -14171,6 +14187,14 @@
"description": "Region represents the region of the member cluster locate in.",
"type": "string"
},
"resourceModels": {
"description": "ResourceModels is the list of resource modeling in this cluster. Each modeling quota can be customized by the user. Modeling name must be one of the following: cpu, memory, storage, ephemeral-storage. If the user does not define the modeling name and modeling quota, it will be the default model. The default model grade from 0 to 8. When grade = 0 or grade = 1, the default model's cpu quota and memory quota is a fix value. When grade greater than or equal to 2, each default model's cpu quota is [2^(grade-1), 2^grade), 2 \u003c= grade \u003c= 7 Each default model's memory quota is [2^(grade + 2), 2^(grade + 3)), 2 \u003c= grade \u003c= 7 E.g. grade 0 likes this: - grade: 0\n ranges:\n - name: \"cpu\"\n min: 0 C\n max: 1 C\n - name: \"memory\"\n min: 0 GB\n max: 4 GB\n\n- grade: 1\n ranges:\n - name: \"cpu\"\n min: 1 C\n max: 2 C\n - name: \"memory\"\n min: 4 GB\n max: 16 GB\n\n- grade: 2\n ranges:\n - name: \"cpu\"\n min: 2 C\n max: 4 C\n - name: \"memory\"\n min: 16 GB\n max: 32 GB\n\n- grade: 7\n range:\n - name: \"cpu\"\n min: 64 C\n max: 128 C\n - name: \"memory\"\n min: 512 GB\n max: 1024 GB\n\ngrade 8, the last one likes below. No matter what Max value you pass, the meaning of Max value in this grade is infinite. You can pass any number greater than Min value. - grade: 8\n range:\n - name: \"cpu\"\n min: 128 C\n max: MAXINT\n - name: \"memory\"\n min: 1024 GB\n max: MAXINT",
"type": "array",
"items": {
"default": {},
"$ref": "#/definitions/com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.ResourceModel"
}
},
"secretRef": {
"description": "SecretRef represents the secret contains mandatory credentials to access the member cluster. The secret should hold credentials as follows: - secret.data.token - secret.data.caBundle",
"$ref": "#/definitions/com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.LocalSecretReference"
Expand Down Expand Up @@ -14264,6 +14288,45 @@
}
}
},
"com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.ResourceModel": {
"description": "ResourceModel describes the modeling that you want to statistics.",
"type": "object",
"properties": {
"grade": {
"description": "Grade is the index for the resource modeling.",
"type": "integer",
"format": "int32"
},
"ranges": {
"description": "Ranges describes the resource quota ranges.",
"type": "array",
"items": {
"default": {},
"$ref": "#/definitions/com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.ResourceModelRange"
}
}
}
},
"com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.ResourceModelRange": {
"description": "ResourceModelRange describes the detail of each modeling quota that ranges from min to max. Please pay attention, by default, the value of min can be inclusive, and the value of max cannot be inclusive. E.g. in an interval, min = 2, max =10 is set, which means the interval [2,10). This rule ensure that all intervals have the same meaning. If the last interval is +∞, it is definitely unreachable. Therefore, we define the right interval as the open interval. For a valid interval, the value on the right is greater than the value on the left, in other words, max must be greater than min. It is strongly recommended that the [Min, Max) of all ResourceModelRanges can make a continuous interval.",
"type": "object",
"properties": {
"max": {
"description": "Max is the maximum amount of this resource represented by resource name. Special Instructions, for the last ResourceModelRange, which no matter what Max value you pass, the meaning is infinite. Because for the last item, any ResourceModelRange's quota larger than Min will be classified to the last one. Of course, the value of the Max field is always greater than the value of the Min field. It should be true in any case.",
"default": {},
"$ref": "#/definitions/io.k8s.apimachinery.pkg.api.resource.Quantity"
},
"min": {
"description": "Min is the minimum amount of this resource represented by resource name. Note: The Min value of first grade(usually 0) always acts as zero. E.g. [1,2) equal to [0,2).",
"default": {},
"$ref": "#/definitions/io.k8s.apimachinery.pkg.api.resource.Quantity"
},
"name": {
"description": "Name is the name for the resource that you want to categorize.",
"type": "string"
}
}
},
"com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.ResourceSummary": {
"description": "ResourceSummary represents the summary of resources in the member cluster.",
"type": "object",
Expand All @@ -14276,6 +14339,14 @@
"$ref": "#/definitions/io.k8s.apimachinery.pkg.api.resource.Quantity"
}
},
"allocatableModelings": {
"description": "AllocatableModelings represents the statistical resource modeling.",
"type": "array",
"items": {
"default": {},
"$ref": "#/definitions/com.github.karmada-io.karmada.pkg.apis.cluster.v1alpha1.AllocatableModeling"
}
},
"allocated": {
"description": "Allocated represents the resources of a cluster that have been scheduled. Total amount of required resources of all Pods that have been scheduled to nodes.",
"type": "object",
Expand Down
135 changes: 135 additions & 0 deletions pkg/apis/cluster/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,30 @@ package cluster

import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// ResourceName is the name identifying various resources in a ResourceList.
type ResourceName string

// Resource names must be not more than 63 characters, consisting of upper- or lower-case alphanumeric characters,
// with the -, _, and . characters allowed anywhere, except the first or last character.
// The default convention, matching that for annotations, is to use lower-case names, with dashes, rather than
// camel case, separating compound words.
// Fully-qualified resource typenames are constructed from a DNS-style subdomain, followed by a slash `/` and a name.
const (
// ResourceCPU in cores. (e,g. 500m = .5 cores)
ResourceCPU ResourceName = "cpu"
// ResourceMemory in bytes. (e,g. 500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
ResourceMemory ResourceName = "memory"
// ResourceStorage is volume size, in bytes (e,g. 5Gi = 5GiB = 5 * 1024 * 1024 * 1024)
ResourceStorage ResourceName = "storage"
// ResourceEphemeralStorage is local ephemeral storage, in bytes. (e,g. 500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
// The resource name for ResourceEphemeralStorage is alpha and it can change across releases.
ResourceEphemeralStorage ResourceName = "ephemeral-storage"
)

//revive:disable:exported

// +genclient
Expand Down Expand Up @@ -104,6 +125,104 @@ type ClusterSpec struct {
// any resource that does not tolerate the Taint.
// +optional
Taints []corev1.Taint

// ResourceModels is the list of resource modeling in this cluster. Each modeling quota can be customized by the user.
// Modeling name must be one of the following: cpu, memory, storage, ephemeral-storage.
// If the user does not define the modeling name and modeling quota, it will be the default model.
// The default model grade from 0 to 8.
// When grade = 0 or grade = 1, the default model's cpu quota and memory quota is a fix value.
// When grade greater than or equal to 2, each default model's cpu quota is [2^(grade-1), 2^grade), 2 <= grade <= 7
// Each default model's memory quota is [2^(grade + 2), 2^(grade + 3)), 2 <= grade <= 7
// E.g. grade 0 likes this:
// - grade: 0
// ranges:
// - name: "cpu"
// min: 0 C
// max: 1 C
// - name: "memory"
// min: 0 GB
// max: 4 GB
//
// - grade: 1
// ranges:
// - name: "cpu"
// min: 1 C
// max: 2 C
// - name: "memory"
// min: 4 GB
// max: 16 GB
//
// - grade: 2
// ranges:
// - name: "cpu"
// min: 2 C
// max: 4 C
// - name: "memory"
// min: 16 GB
// max: 32 GB
//
// - grade: 7
// range:
// - name: "cpu"
// min: 64 C
// max: 128 C
// - name: "memory"
// min: 512 GB
// max: 1024 GB
//
// grade 8, the last one likes below. No matter what Max value you pass,
// the meaning of Max value in this grade is infinite. You can pass any number greater than Min value.
// - grade: 8
// range:
// - name: "cpu"
// min: 128 C
// max: MAXINT
// - name: "memory"
// min: 1024 GB
// max: MAXINT
//
// +optional
ResourceModels []ResourceModel
}

// ResourceModel describes the modeling that you want to statistics.
type ResourceModel struct {
// Grade is the index for the resource modeling.
// +optional
Grade int
Comment on lines +190 to +192
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: what's the expected behavior when user sets ResourceModels with grade 1:CPU[2,4), grade 3:CPU[8,16), grade 4:CPU[16,32)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, IIRC we currently don't define the output of kubectl describe cluster, but it might be great if we can define a human readable format of ResourceModels like:

ResourceModels:
	Grade 1:	CPU [2,4)	Memory [4G,8G)
	Grade 2:	CPU [4,8)	Memory [8G,16G)
	Grade 3:	CPU [8,16)	Memory [16G,32G)

A method like ResourceModel.Stringify() shall be helpful in this feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when user sets ResourceModels with grade 1:CPU[2,4), grade 3:CPU[8,16), grade 4:CPU[16,32), if some resource's CPU is 6, this resource will be assigned to a higher grade. In other words, it will belong to grade 3:CPU[8,16). Does it make sense?

By the way, in this case, grade2 doesn't exist? Or the user forgot to enter it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ResourceModel.Stringify() feature is very nice. If time is enough, I can do it together in this version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when user sets ResourceModels with grade 1:CPU[2,4), grade 3:CPU[8,16), grade 4:CPU[16,32), if some resource's CPU is 6, this resource will be assigned to a higher grade.

+1. We can take this as the default behavior. But I don't think we should note this case.

By the way, in this case, grade2 doesn't exist? Or the user forgot to enter it?

This is an abnormal case, probably due to a mistake.
This usage is ambiguous, my suggestion is to validate the input at webhook(Validation Plugin) to avoid discontinuous ranges.


// Ranges describes the resource quota ranges.
// +optional
Ranges []ResourceModelRange
}

// ResourceModelRange describes the detail of each modeling quota that ranges from min to max.
// Please pay attention, by default, the value of min can be inclusive, and the value of max cannot be inclusive.
// E.g. in an interval, min = 2, max =10 is set, which means the interval [2,10).
// This rule ensure that all intervals have the same meaning. If the last interval is +∞,
// it is definitely unreachable. Therefore, we define the right interval as the open interval.
// For a valid interval, the value on the right is greater than the value on the left,
// in other words, max must be greater than min.
// It is strongly recommended that the [Min, Max) of all ResourceModelRanges can make a continuous interval.
type ResourceModelRange struct {
// Name is the name for the resource that you want to categorize.
// +optional
Name ResourceName

// Min is the minimum amount of this resource represented by resource name.
// Note: The Min value of first grade(usually 0) always acts as zero.
// E.g. [1,2) equal to [0,2).
// +optional
Min resource.Quantity

// Max is the maximum amount of this resource represented by resource name.
// Special Instructions, for the last ResourceModelRange, which no matter what Max value you pass,
// the meaning is infinite. Because for the last item,
// any ResourceModelRange's quota larger than Min will be classified to the last one.
// Of course, the value of the Max field is always greater than the value of the Min field.
// It should be true in any case.
// +optional
Max resource.Quantity
Comment on lines +218 to +225
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Special Instructions, for the last ResourceModelRange, which no matter what Max value you pass,
the meaning is infinite. Because for the last item
Shall we make the last grade ignore user-input max value, or, add a computed last/largest grade according to last/largest user-input max value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My current implementation is that whatever value the last max is, those greater than min will be placed in the last group. This ensures that there will be no missing ResourceModelRange. Do you think my solution is reasonable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the implementation behavior, I guess what @kevin-wangzefeng mean is we should clearly note the behavior on API comments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, the behavior description should be added to ResourceModels(.spec.ResourceModels), not on ResourceModelRange which just represents the range. How we use/treat ranges is decided at ResourceModels.

}

const (
Expand Down Expand Up @@ -216,6 +335,22 @@ type ResourceSummary struct {
// Total amount of required resources of all Pods that have been scheduled to nodes.
// +optional
Allocated corev1.ResourceList

// AllocatableModelings represents the statistical resource modeling.
// +optional
AllocatableModelings []AllocatableModeling
}

// AllocatableModeling represents the number of nodes in which allocatable resources in a specific resource model grade.
// E.g. AllocatableModeling{Grade: 2, Count: 10} means 10 nodes belong to resource model in grade 2.
type AllocatableModeling struct {
// Grade is the index of ResourceModel.
// +optional
Grade int

// Count is the number of nodes that own the resources delineated by this modeling.
// +optional
Count int
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
Expand Down
Loading