Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sandbox environment: add limit range creation #770

Merged
merged 2 commits into from
Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions infrastructure/docker-registry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@


## What is it
From the [Docker Registry](https://docs.docker.com/registry/) official documentation: A Registry is a stateless, highly scalable server-side application that stores and lets you distribute Docker images.
From the [Docker Registry](https://docs.docker.com/registry/) official documentation: A Registry is a stateless, highly scalable server-side application that stores and lets you distribute Docker images.
[Harbor](https://goharbor.io/) is an open source registry having a lot of features, such as an advanced UI, a vulnerability scanner, robot accounts and so on. For more information visit the official web page.

## Why do we need it?
Expand Down Expand Up @@ -40,9 +40,9 @@ To install Harbor, it is possible to leverage the [official Helm Chart](https://
6. PVC that can be shared across nodes (i.e., with `ReadWriteMany` access mode) or external object storage

### Redis Configuration
In our architecture we have a [Redis-Sentinel](https://redis.io/topics/sentinel) service, instead of [Redis Cluster](https://redis.io/topics/cluster-tutorial), because with this architecture Sentinel manages automatically the failover of the master.
In our architecture we have a [Redis-Sentinel](https://redis.io/docs/manual/sentinel/) service, instead of [Redis Cluster](https://redis.io/docs/manual/scaling/), because with this architecture Sentinel manages automatically the failover of the master.
To enable the `Redis-Sentinel ` architecture it is necessary to configure the following parameter in the redis file values (`redis-service-values.yaml`):
```yaml
```yaml
sentinel.enabled=true
```

Expand Down Expand Up @@ -83,4 +83,3 @@ helm upgrade harbor harbor/harbor --namespace harbor \
```
Warning: credentials and secret parameters have been redacted from the values file stored in this repository.
Look [here](https://github.com/goharbor/harbor-helm) for a complete configuration guide.

Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ rules:
verbs: ["get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"]

- apiGroups: [""]
resources: ["namespaces", "resourcequotas", "secrets"]
resources: ["namespaces", "resourcequotas", "limitranges", "secrets"]
verbs: ["get", "list", "watch", "create", "update", "delete", "deletecollection"]

- apiGroups: ["rbac.authorization.k8s.io"]
Expand Down
37 changes: 37 additions & 0 deletions operators/pkg/forge/limitrange.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright 2020-2022 Politecnico di Torino
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package forge

import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
)

// SandboxLimitRangeSpec forges the Limit Range spec for sandbox namespaces.
func SandboxLimitRangeSpec() corev1.LimitRangeSpec {
return corev1.LimitRangeSpec{
Limits: []corev1.LimitRangeItem{{
Type: corev1.LimitTypeContainer,
DefaultRequest: corev1.ResourceList{
corev1.ResourceCPU: *resource.NewScaledQuantity(10, resource.Milli),
corev1.ResourceMemory: *resource.NewScaledQuantity(50, resource.Mega),
},
Default: corev1.ResourceList{
corev1.ResourceCPU: *resource.NewScaledQuantity(100, resource.Milli),
corev1.ResourceMemory: *resource.NewScaledQuantity(250, resource.Mega),
},
}},
}
}
51 changes: 51 additions & 0 deletions operators/pkg/forge/limitrange_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright 2020-2022 Politecnico di Torino
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package forge_test

import (
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"

"github.com/netgroup-polito/CrownLabs/operators/pkg/forge"
)

var _ = Describe("Limit range spec forging", func() {
Describe("The forge.SandboxLimitRangeSpec function", func() {
var spec corev1.LimitRangeSpec

JustBeforeEach(func() { spec = forge.SandboxLimitRangeSpec() })

It("Should have a single limit entry of type container", func() {
Expect(spec.Limits).To(HaveLen(1))
Expect(spec.Limits[0].Type).To(BeIdenticalTo(corev1.LimitTypeContainer))
})

It("Should set the correct default requests", func() {
Expect(spec.Limits[0].DefaultRequest[corev1.ResourceCPU]).To(WithTransform(
func(q resource.Quantity) string { return q.String() }, Equal("10m")))
Expect(spec.Limits[0].DefaultRequest[corev1.ResourceMemory]).To(WithTransform(
func(q resource.Quantity) string { return q.String() }, Equal("50M")))
})

It("Should set the correct default limits", func() {
Expect(spec.Limits[0].Default[corev1.ResourceCPU]).To(WithTransform(
func(q resource.Quantity) string { return q.String() }, Equal("100m")))
Expect(spec.Limits[0].Default[corev1.ResourceMemory]).To(WithTransform(
func(q resource.Quantity) string { return q.String() }, Equal("250M")))
})
})
})
16 changes: 16 additions & 0 deletions operators/pkg/tenant-controller/sandbox.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,22 @@ func (r *TenantReconciler) enforceSandboxResourcesPresence(ctx context.Context,
log.V(utils.FromResult(res)).Info("sandbox resource quota correctly enforced", "resource quota", klog.KObj(&resourceQuota), "result", res)
tenant.Status.SandboxNamespace.Created = true

// Enforce limit range presence
limitRange := corev1.LimitRange{
ObjectMeta: metav1.ObjectMeta{Name: "sandbox-limit-range", Namespace: sandboxnsName},
}
res, err = ctrl.CreateOrUpdate(ctx, r.Client, &limitRange, func() error {
limitRange.SetLabels(forge.SandboxObjectLabels(resourceQuota.GetLabels(), tenant.Name))
limitRange.Spec = forge.SandboxLimitRangeSpec()
return ctrl.SetControllerReference(tenant, &limitRange, r.Scheme)
})
if err != nil {
log.Error(err, "failed to enforce resource", "limit range", klog.KObj(&limitRange))
return err
}
log.V(utils.FromResult(res)).Info("sandbox limit range correctly enforced", "limit range", klog.KObj(&limitRange), "result", res)
tenant.Status.SandboxNamespace.Created = true

return err
}

Expand Down
32 changes: 31 additions & 1 deletion operators/pkg/tenant-controller/sandbox_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ var _ = Describe("Sandbox", func() {
resQuotaNSname types.NamespacedName
sbResQuota corev1.ResourceQuota

limitRangeNSname types.NamespacedName
sbLimitRange corev1.LimitRange

ownerRef metav1.OwnerReference
err error
)
Expand All @@ -70,11 +73,15 @@ var _ = Describe("Sandbox", func() {
sbResQuota = corev1.ResourceQuota{
ObjectMeta: forge.NamespacedNameToObjectMeta(resQuotaNSname),
}
sbLimitRange = corev1.LimitRange{
ObjectMeta: forge.NamespacedNameToObjectMeta(limitRangeNSname),
}

sbNamespace.SetCreationTimestamp(metav1.NewTime(time.Now()))
sbRoleBinding.SetCreationTimestamp(metav1.NewTime(time.Now()))
sbResQuota.SetCreationTimestamp(metav1.NewTime(time.Now()))
clientBuilder.WithObjects(&sbNamespace, &sbRoleBinding, &sbResQuota)
sbLimitRange.SetCreationTimestamp(metav1.NewTime(time.Now()))
clientBuilder.WithObjects(&sbNamespace, &sbRoleBinding, &sbResQuota, &sbLimitRange)
}

DescribeBodySandboxNamespacePresence := func() {
Expand Down Expand Up @@ -128,6 +135,22 @@ var _ = Describe("Sandbox", func() {
}
})
}
DescribeBodySandboxLimitRangePresence := func() {
It("Limit range should be present and have the expected labels", func() {
Expect(reconciler.Get(ctx, limitRangeNSname, &sbLimitRange)).To(Succeed())
for k, v := range forge.SandboxObjectLabels(nil, tenantName) {
Expect(sbLimitRange.GetLabels()).To(HaveKeyWithValue(k, v))
}
Expect(sbLimitRange.GetOwnerReferences()).To(ContainElement(ownerRef))
})

It("Role binding should be present and have the expected spec", func() {
Expect(reconciler.Get(ctx, limitRangeNSname, &sbLimitRange)).To(Succeed())
// The following asserts the correctness of a single field, leaving more thorough checks to the appropriate unit tests.
Expect(sbLimitRange.Spec.Limits).To(HaveLen(1))
Expect(sbLimitRange.Spec.Limits[0].Type).To(BeIdenticalTo(corev1.LimitTypeContainer))
})
}
DescribeBodySandboxNamespaceAbsence := func() {
It("Should set the sandbox status empty", func() {
Expect(tenant.Status.SandboxNamespace).To(BeIdenticalTo(clv1alpha2.NameCreated{Name: "", Created: false}))
Expand Down Expand Up @@ -163,9 +186,14 @@ var _ = Describe("Sandbox", func() {
Name: "sandbox-resource-quota",
Namespace: sandboxNSname.Name,
}
limitRangeNSname = types.NamespacedName{
Name: "sandbox-limit-range",
Namespace: sandboxNSname.Name,
}
sbNamespace = corev1.Namespace{}
sbRoleBinding = rbacv1.RoleBinding{}
sbResQuota = corev1.ResourceQuota{}
sbLimitRange = corev1.LimitRange{}
})

JustBeforeEach(func() {
Expand All @@ -186,6 +214,7 @@ var _ = Describe("Sandbox", func() {
Describe("Assessing the namespace presence", func() { DescribeBodySandboxNamespacePresence() })
Describe("Assessing the resource quota presence", func() { DescribeBodySandboxResourceQuotaPresence() })
Describe("Assessing the role binding presence", func() { DescribeBodySandboxRoleBindingPresence() })
Describe("Assessing the limit range presence", func() { DescribeBodySandboxLimitRangePresence() })
})

When("Sandbox resources are already present", func() {
Expand All @@ -197,6 +226,7 @@ var _ = Describe("Sandbox", func() {
Describe("Assessing the namespace presence", func() { DescribeBodySandboxNamespacePresence() })
Describe("Assessing the resource quota presence", func() { DescribeBodySandboxResourceQuotaPresence() })
Describe("Assessing the role binding presence", func() { DescribeBodySandboxRoleBindingPresence() })
Describe("Assessing the limit range presence", func() { DescribeBodySandboxLimitRangePresence() })
})
})

Expand Down