Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for scheduled object creation #19

Merged
merged 12 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .apidoc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
processor:
ignoreTypes:
- "(.*)List$"
- "TypeMeta"
render:
kubernetesVersion: 1.25
9 changes: 9 additions & 0 deletions .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ linters:
- gci # Disable gci import ordering checker since its buggy
- depguard # Disallows to use non-listed packages
- tagalign # Disallows to use non-aligned tags
linters-settings:
funlen:
lines: 90
issues:
exclude-rules:
- path: .*/metrics.go
Expand All @@ -19,3 +22,9 @@ run:
tests: false
skip-dirs:
- cmd/benchmark
skip-files:
- cmd/.*.go
- mocks/.*.go
- common/config.go
- .*/groupversion_info.go
- .*/zz_generated.deepcopy.go
41 changes: 37 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,30 @@
<img alt="logo" width="600" src="https://abload.de/img/mayfly-logo-darkt9eye.png">
</picture>

> Mayfly is a Kubernetes operator that enables you to create temporary resources on the cluster that will expire after a certain period of time.
> Mayfly is a Kubernetes operator that enables you to have time-based resources. They creates or deletes on the specified time.

## 📖 General Information

### 📄 Summary

The Mayfly Operator allows you to have your resources on your cluster for a temporary time.
It deletes those resources from the cluster, according to the Mayfly expiration annotation that you set to specify how long the resource should remain active. This can be used to create temporary resources, temporary accesses, or simply to keep your cluster organized and tidy.
The Mayfly Operator allows you to have your resources on your cluster for a temporary time by the given expiration or mayfly create the resources at the time you specified.
It deletes those resources from the cluster, according to the Mayfly expiration annotation that you set to specify how long the resource should remain active. This can be used to create temporary resources, temporary accesses, or simply to keep your cluster organized and tidy.
Also, It creates the resources you specify at the given time by using `ScheduleResource` custom resource definition. You can also merge these two features together, just to have some resource created in the future and only for a specific amount of time.

### 🛠 Configuration

Mayfly is an easy-to-use and configurable project that uses resource watches and schedulers to delete your resources at the appropriate time. It is simple to set up and customize.
To specify which resources should be monitored and cleaned up, you can set the `RESOURCES` environment variable to a comma-separated list of `{ApiVersion};{Kind}` as text. This allows you to customize which resources are targeted for cleanup.
To specify which resources should be monitored and cleaned up, you can set the `RESOURCES` environment variable to a comma-separated list of `{ApiVersion};{Kind}` as text. This allows you to customize which resources are targeted for cleanup with expiration annotations.

Example:
```
export RESOURCES="v1;Secret,test.com/v1alpha;MyCRD"
```

## 🚀 Usage

### Resouce Expiration

Once you have determined which resources you want Mayfly to monitor, you can set the `mayfly.cloud.namecheap.com/expire` annotation on those resources with a duration value. This will cause Mayfly to delete the resources once the specified duration has passed, based on the time of their creation.
Keep in mind that the expiration will be calculated based on the creation time of the resource.

Expand All @@ -43,6 +47,35 @@ spec:
- infinity
```

### Scheduled Resource Creation

The `ScheduledResource` CRD allows you to schedule the creation of an object in the future. This can be combined with the expire annotation, enabling Mayfly to create and remove certain objects for a temporary period in the future.

Example:
```
apiVersion: cloud.namecheap.com/v1alpha1
kind: ScheduledResource
metadata:
annotations:
mayfly.cloud.namecheap.com/expire: 60m
name: example
namespace: default
spec:
in: 30m
content: |
apiVersion: v1
kind: Secret
metadata:
name: example
namespace: default
annotations:
mayfly.cloud.namecheap.com/expire: 30m
data:
.secret-file: dmFsdWUtMg0KDQo=
status:
condition: Scheduled
```
This feature is particularly useful for setting up temporary resources that are only needed for a short period, reducing clutter and improving the efficiency of resource management.

## 🛳️ Deployment

Expand Down
4 changes: 2 additions & 2 deletions cmd/benchmark/benchmark.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"fmt"
time "time"

"github.com/NCCloud/mayfly/pkg"
"github.com/NCCloud/mayfly/pkg/common"

v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand All @@ -30,7 +30,7 @@ func (b *Benchmark) Listener() Result {
kind := make(map[string]int)
now := time.Now()
for _, resourceKind := range b.config.Resources {
resourceList := pkg.NewResourceInstanceList(resourceKind)
resourceList := common.NewResourceInstanceList(resourceKind)

resourcesListErr := b.mgrClient.List(context.Background(), resourceList)
if resourcesListErr != nil {
Expand Down
6 changes: 3 additions & 3 deletions cmd/benchmark/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"os"
"time"

"github.com/NCCloud/mayfly/pkg"
"github.com/NCCloud/mayfly/pkg/common"

"github.com/go-echarts/go-echarts/v2/charts"
"github.com/go-echarts/go-echarts/v2/opts"
Expand All @@ -19,7 +19,7 @@ import (
)

var (
config *pkg.Config
config *common.Config
mgrClient client.Client
pageTitle = "Mayfly Benchmark"
benchmarkHtml = "mayfly_benchmark.html"
Expand All @@ -29,7 +29,7 @@ func init() {
scheme := runtime.NewScheme()
utilruntime.Must(clientgoscheme.AddToScheme(scheme))

config = pkg.NewConfig()
config = common.NewConfig()

var mgrClientErr error
mgrClient, mgrClientErr = client.New(ctrl.GetConfigOrDie(), client.Options{Scheme: scheme})
Expand Down
6 changes: 3 additions & 3 deletions cmd/benchmark/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ package main
import (
"time"

"github.com/NCCloud/mayfly/pkg"
"github.com/NCCloud/mayfly/pkg/common"

"sigs.k8s.io/controller-runtime/pkg/client"
)

type Benchmark struct {
granularity time.Duration
config *pkg.Config
config *common.Config
mgrClient client.Client
startedAt time.Time
count int
Expand All @@ -29,7 +29,7 @@ type Point struct {
time time.Time
}

func NewBenchmark(mgrClient client.Client, config *pkg.Config, count int) *Benchmark {
func NewBenchmark(mgrClient client.Client, config *common.Config, count int) *Benchmark {
return &Benchmark{
granularity: 5 * time.Second,
config: config,
Expand Down
26 changes: 20 additions & 6 deletions cmd/manager/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,14 @@ package main
import (
"fmt"

"github.com/NCCloud/mayfly/pkg/apis/v1alpha1"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"

"github.com/NCCloud/mayfly/pkg/common"
"github.com/NCCloud/mayfly/pkg/controllers"

"sigs.k8s.io/controller-runtime/pkg/metrics/server"

"github.com/NCCloud/mayfly/pkg"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/cache"
Expand All @@ -21,7 +26,10 @@ const (
func main() {
logger := zap.New()
scheme := runtime.NewScheme()
config := pkg.NewConfig()
config := common.NewConfig()

utilruntime.Must(v1alpha1.AddToScheme(scheme))
ctrl.SetLogger(logger)

logger.Info("Configuration", "config", config)

Expand All @@ -35,22 +43,28 @@ func main() {
LeaderElection: config.EnableLeaderElection,
LeaderElectionID: "mayfly-leader.cloud.namecheap.com",
Cache: cache.Options{
SyncPeriod: config.SyncPeriod,
SyncPeriod: &config.SyncPeriod,
},
})
if managerErr != nil {
panic(managerErr)
}

client := manager.GetClient()
scheduler := pkg.NewScheduler(config, client)
scheduler := common.NewScheduler(config, client)

for _, resource := range config.Resources {
if err := pkg.NewController(config, client, resource, scheduler).SetupWithManager(manager); err != nil {
panic(err)
if expirationControllerErr := controllers.NewExpirationController(config, client, resource, scheduler).
SetupWithManager(manager); expirationControllerErr != nil {
panic(expirationControllerErr)
}
}

if scheduledResourceControllerErr := controllers.NewScheduledResourceController(config, client, scheduler).
SetupWithManager(manager); scheduledResourceControllerErr != nil {
panic(scheduledResourceControllerErr)
}

if addHealthCheckErr := manager.AddHealthzCheck("healthz", healthz.Ping); addHealthCheckErr != nil {
panic(addHealthCheckErr)
}
Expand Down
64 changes: 64 additions & 0 deletions deploy/crds/cloud.namecheap.com_scheduledresources.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.13.0
name: scheduledresources.cloud.namecheap.com
spec:
group: cloud.namecheap.com
names:
kind: ScheduledResource
listKind: ScheduledResourceList
plural: scheduledresources
singular: scheduledresource
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .spec.in
name: In
type: string
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- jsonPath: .status.condition
name: Condition
type: string
name: v1alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
content:
type: string
in:
type: string
required:
- content
- in
type: object
status:
properties:
condition:
type: string
required:
- condition
type: object
type: object
served: true
storage: true
subresources:
status: {}
37 changes: 33 additions & 4 deletions devops.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,29 @@
#!/usr/bin/env bash

export CONTROLLER_GEN_VERSION="v0.13.0"
export GOLANGCI_LINT_VERSION="v1.54.2"
export MOCKERY_GEN_VERSION="v2.36.0"
export GOFUMPT_VERSION="v0.5.0"
export TESTENV_VERSION="1.25.x!"

prerequisites() {
if ! command -v golangci-lint &>/dev/null; then
if [[ "$(controller-gen --version 2>&1)" != *"$CONTROLLER_GEN_VERSION"* ]]; then
go install sigs.k8s.io/controller-tools/cmd/controller-gen@"${CONTROLLER_GEN_VERSION}"
fi
if [[ "$(golangci-lint --version 2>&1)" != *"$GOLANGCI_LINT_VERSION"* ]]; then
go install github.com/golangci/golangci-lint/cmd/golangci-lint@"${GOLANGCI_LINT_VERSION}"
fi
if ! command -v gofumpt &>/dev/null; then
go install mvdan.cc/gofumpt@latest
if [[ "$(mockery --version 2>&1)" != *"$MOCKERY_GEN_VERSION"* ]]; then
go install github.com/vektra/mockery/v2@"${MOCKERY_GEN_VERSION}"
fi
if [[ "$(gofumpt --version 2>&1)" != *"$GOFUMPT_VERSION"* ]]; then
go install mvdan.cc/gofumpt@"${GOFUMPT_VERSION}"
fi
if ! command -v crd-ref-docs &>/dev/null; then
go install github.com/elastic/crd-ref-docs@latest
fi
if ! command -v setup-envtest &>/dev/null; then
go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
fi
}

Expand All @@ -16,8 +32,21 @@ lint() {
golangci-lint run --timeout=10m
}

generate() {
rm -rf deploy/crds
controller-gen object paths="./..."
controller-gen crd paths="./..." output:dir=deploy/crds
sed '/Compiled/d' pkg/apis/v1alpha1/zz_generated.deepcopy.go > pkg/apis/v1alpha1/zz_generated.deepcopy.gotmp
mv pkg/apis/v1alpha1/zz_generated.deepcopy.gotmp pkg/apis/v1alpha1/zz_generated.deepcopy.go
crd-ref-docs --source-path=./pkg/apis --config .apidoc.yaml --renderer markdown --output-path=./docs/api.md
}

install() {
kubectl apply -f deploy/crds
}

test() {
go test -v ./...
go test -v -coverpkg=./... ./...
}

prerequisites
Expand Down
Loading
Loading