Skip to content

Commit

Permalink
Add placementStrategy API
Browse files Browse the repository at this point in the history 
Signed-off-by: melserngawy <melserng@redhat.com>
  • Loading branch information
@serngawy
serngawy committed Jun 16, 2023
1 parent 088e8fe commit 7f121af
Show file tree
Hide file tree
Showing 5 changed files with 350 additions and 1 deletion.
180 changes: 180 additions & 0 deletions cluster/v1beta1/0000_02_clusters.open-cluster-management.io_placements.crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,154 @@ spec:
items:
type: string
type: array
decisionStrategy:
description: DecisionStrategy divide the created placement decision
to groups and define number of clusters per decision group.
properties:
groupStrategy:
description: GroupStrategy define strategies to divide selected
clusters to decision groups.
properties:
clustersPerDecisionGroup:
anyOf:
- type: integer
- type: string
default: 100%
description: ClustersPerDecisionGroup is a specific number
or percentage of the total selected clusters. The specific
number will divide the placementDecisions to decisionGroups
each group has max number of clusters equal to that specific
number. The percentage will divide the placementDecisions
to decisionGroups each group has max number of clusters
based on the total num of selected clusters and percentage.
ex; for a total 100 clusters selected, ClustersPerDecisionGroup
equal to 20% will divide the placement decision to 5 groups
each group should have 20 clusters. Default is having all
clusters in a single group. If the DecisionGroups field
defined, it will be considered first to create the decisionGroups
then the ClustersPerDecisionGroup will be used to determine
the rest of decisionGroups.
x-kubernetes-int-or-string: true
decisionGroups:
description: DecisionGroups represents a list of predefined
groups to put decision results.
items:
description: DecisionGroup define a subset of clusters that
will be added to placementDecisions with groupName label.
properties:
groupClusterSelector:
description: LabelSelector to select clusters subset
by label.
properties:
claimSelector:
description: ClaimSelector represents a selector
of ManagedClusters by clusterClaims in status
properties:
matchExpressions:
description: matchExpressions is a list of cluster
claim selector requirements. The requirements
are ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
type: object
labelSelector:
description: LabelSelector represents a selector
of ManagedClusters by label
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement
is a selector that contains values, a key,
and an operator that relates the key and
values.
properties:
key:
description: key is the label key that
the selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty.
If the operator is Exists or DoesNotExist,
the values array must be empty. This
array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is
"In", and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
type: object
groupName:
description: Group name to be added as label value to
the created placement Decisions labels with label
key cluster.open-cluster-management.io/decision-group-name
pattern: ^[a-zA-Z0-9][-A-Za-z0-9_.]{0,61}[a-zA-Z0-9]$
type: string
required:
- groupClusterSelector
- groupName
type: object
type: array
required:
- clustersPerDecisionGroup
type: object
type: object
numberOfClusters:
description: NumberOfClusters represents the desired number of ManagedClusters
to be selected which meet the placement requirements. 1) If not
Expand Down Expand Up @@ -466,6 +614,38 @@ spec:
- type
type: object
type: array
decisionGroups:
description: List of decision groups determined by the placement and
DecisionStrategy.
items:
description: Present decision groups status based on the DecisionStrategy
definition.
properties:
clusterCount:
default: 0
description: Total number of clusters in the decision group.
Clusters count is equal or less than the clusterPerDecisionGroups
defined in the decision strategy.
format: int32
type: integer
decisionGroupIndex:
description: Present the decision group index. If there is no
decision strategy defined all placement decisions will be
in group index 0
format: int32
type: integer
decisionGroupName:
description: Decision group name that is defined in the DecisionStrategy's
DecisionGroup.
type: string
decisions:
description: List of placement decisions names associated with
the decision group
items:
type: string
type: array
type: object
type: array
numberOfSelectedClusters:
description: NumberOfSelectedClusters represents the number of selected
ManagedClusters
Expand Down
53 changes: 53 additions & 0 deletions cluster/v1beta1/helpers.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -175,3 +175,56 @@ func (pdct *PlacementDecisionClustersTracker) Existing() sets.Set[string] {

return pdct.existingScheduledClusters
}

// RolloutStrategy Type
type RolloutStrategyType string

const (
//All means apply the workload to all clusters in the decision groups at once.
All RolloutStrategyType = "All"
//Progressive means apply the workload to the selected clusters progressively per cluster.
Progressive RolloutStrategyType = "Progressive"
//ProgressivePerGroup means apply the workload to the selected clusters progressively per group.
ProgressivePerGroup RolloutStrategyType = "ProgressivePerGroup"
)

// MandatoryDecisionGroup set the decision group name and group index.
type MandatoryDecisionGroup struct {
// GroupName of the decision group should match the placementDecisions label value with label key cluster.open-cluster-management.io/decision-group-name
// +optional
GroupName string `json:"groupName,omitempty"`

// GroupIndex of the decision group should match the placementDecisions label value with label key cluster.open-cluster-management.io/decision-group-index
// +optional
GroupIndex int32 `json:"groupIndex,omitempty"`
}

// Rollout strategy to be used by workload applier controller.
// +k8s:deepcopy-gen=false
type RolloutStrategy struct {
// Rollout strategy Types are All, Progressive and ProgressivePerGroup
// 1) All means apply the workload to all clusters in the decision groups at once.
// 2) Progressive means apply the workload to the selected clusters progressively per cluster. The workload will not be applied to the next cluster unless one of the current applied clusters reach the successful state or timeout.
// 3) ProgressivePerGroup means apply the workload to decisionGroup clusters progressively per group. The workload will not be applied to the next decisionGroup unless all clusters in the current group reach the successful state or timeout.
// +kubebuilder:default:=All
// +optional
Type RolloutStrategyType `json:"type,omitempty"`

// List of the decision groups names or indexes to apply the workload first and fail if workload did not reach successful ((not proceed to apply workload to other decision groups/clusters).
// +optional
MandatoryDecisionGroups []MandatoryDecisionGroup `json:"mandatoryDecisionGroups,omitempty"`

// Timeout define how long workload applier controller will wait till workload reach successful state in the cluster. Only considered for Rollout Type Progressive and ProgressivePerGroup.
// Timeout default value is None meaning the workload applier will not proceed apply workload to other clusters if did not reach the successful state.
// Timeout must be defined in [0-9h]|[0-9m]|[0-9s] format examples; 2h , 90m , 360s
// +kubebuilder:validation:Pattern="^(([0-9])+[h|m|s])|None$"
// +kubebuilder:default:=None
// +optional
Timeout string `json:"timeout,omitempty"`

// MaxConcurrency is the max number of clusters to deploy workload concurrently. The default value for MaxConcurrency is determined from the clustersPerDecisionGroup defined in the placement->DecisionStrategy.
// When RolloutStrategy Type defined as ProgressivePerGroup the MaxConcurrency must not be bigger than the clustersPerDecisionGroup defined in the placement->DecisionStrategy otherwise it is the workload applier controller responsibility to re-group the placement clusters and keep track each group status.
// +kubebuilder:validation:Pattern="^((100|[0-9]{1,2})%|[0-9]+)$"
// +optional
MaxConcurrency string `json:"maxConcurrency,omitempty"`
}
68 changes: 68 additions & 0 deletions cluster/v1beta1/types_placement.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package v1beta1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
v1 "open-cluster-management.io/api/cluster/v1"
)

Expand Down Expand Up @@ -94,6 +95,49 @@ type PlacementSpec struct {
// certain taints to be selected by placements with matching tolerations.
// +optional
Tolerations []Toleration `json:"tolerations,omitempty"`

// DecisionStrategy divide the created placement decision to groups and define number of clusters per decision group.
// +optional
DecisionStrategy DecisionStrategy `json:"decisionStrategy,omitempty"`
}

// DecisionGroup define a subset of clusters that will be added to placementDecisions with groupName label.
type DecisionGroup struct {
// Group name to be added as label value to the created placement Decisions labels with label key cluster.open-cluster-management.io/decision-group-name
// +kubebuilder:validation:Required
// +kubebuilder:validation:Pattern="^[a-zA-Z0-9][-A-Za-z0-9_.]{0,61}[a-zA-Z0-9]$"
// +required
GroupName string `json:"groupName,omitempty"`

// LabelSelector to select clusters subset by label.
// +kubebuilder:validation:Required
// +required
ClusterSelector ClusterSelector `json:"groupClusterSelector,omitempty"`
}

// Group the created placementDecision into decision groups based on the number of clusters per decision group.
type GroupStrategy struct {
// DecisionGroups represents a list of predefined groups to put decision results.
// +optional
DecisionGroups []DecisionGroup `json:"decisionGroups,omitempty"`

// ClustersPerDecisionGroup is a specific number or percentage of the total selected clusters.
// The specific number will divide the placementDecisions to decisionGroups each group has max number of clusters equal to that specific number.
// The percentage will divide the placementDecisions to decisionGroups each group has max number of clusters based on the total num of selected clusters and percentage.
// ex; for a total 100 clusters selected, ClustersPerDecisionGroup equal to 20% will divide the placement decision to 5 groups each group should have 20 clusters.
// Default is having all clusters in a single group.
// If the DecisionGroups field defined, it will be considered first to create the decisionGroups then the ClustersPerDecisionGroup will be used to determine the rest of decisionGroups.
// +kubebuilder:validation:Required
// +kubebuilder:default:="100%"
// +optional
ClustersPerDecisionGroup intstr.IntOrString `json:"clustersPerDecisionGroup,omitempty"`
}

// DecisionStrategy divide the created placement decision to groups and define number of clusters per decision group.
type DecisionStrategy struct {
// GroupStrategy define strategies to divide selected clusters to decision groups.
// +optional
GroupStrategy GroupStrategy `json:"groupStrategy,omitempty"`
}

// ClusterPredicate represents a predicate to select ManagedClusters.
Expand Down Expand Up @@ -329,11 +373,35 @@ const (
TolerationOpEqual TolerationOperator = "Equal"
)

// Present decision groups status based on the DecisionStrategy definition.
type DecisionGroupStatus struct {
// Present the decision group index. If there is no decision strategy defined all placement decisions will be in group index 0
// +optional
DecisionGroupIndex int32 `json:"decisionGroupIndex"`

// Decision group name that is defined in the DecisionStrategy's DecisionGroup.
// +optional
DecisionGroupName string `json:"decisionGroupName"`

// List of placement decisions names associated with the decision group
// +optional
Decisions []string `json:"decisions"`

// Total number of clusters in the decision group. Clusters count is equal or less than the clusterPerDecisionGroups defined in the decision strategy.
// +kubebuilder:default:=0
// +optional
ClustersCount int32 `json:"clusterCount"`
}

type PlacementStatus struct {
// NumberOfSelectedClusters represents the number of selected ManagedClusters
// +optional
NumberOfSelectedClusters int32 `json:"numberOfSelectedClusters"`

// List of decision groups determined by the placement and DecisionStrategy.
// +optional
DecisionGroups []DecisionGroupStatus `json:"decisionGroups"`

// Conditions contains the different condition status for this Placement.
// +optional
Conditions []metav1.Condition `json:"conditions"`
Expand Down
7 changes: 6 additions & 1 deletion cluster/v1beta1/types_placementdecision.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,14 @@ type PlacementDecision struct {
Status PlacementDecisionStatus `json:"status,omitempty"`
}

// The placementDecsion label name holding the placement name
// The placementDecsion labels
const (
// Placement owner name.
PlacementLabel string = "cluster.open-cluster-management.io/placement"
// decision group index.
DecisionGroupIndexLabel string = "cluster.open-cluster-management.io/decision-group-index"
// decision group name.
DecisionGroupNameLabel string = "cluster.open-cluster-management.io/decision-group-name"
)

// PlacementDecisionStatus represents the current status of the PlacementDecision.
Expand Down
Loading

0 comments on commit 7f121af

Please sign in to comment.