Skip to content

Commit

Permalink
Add envtest testing docs to extend cronjob example
Browse files Browse the repository at this point in the history
  • Loading branch information
gabbifish committed May 18, 2020
1 parent 1eccd8f commit 5ad2ca6
Show file tree
Hide file tree
Showing 5 changed files with 449 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
/*
We can create the tests in a separate file, cronjob_controller_test.go.
Keeping consistent with the Gingko usage in our autogenerated suite_test.go code, we’ll use Gingko to write our tests.
*/

/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// +kubebuilder:docs-gen:collapse=Apache License

/*
As usual, we start with necessary imports.
*/
package controllers

import (
"context"
"reflect"
"time"

cronjobv1 "tutorial.kubebuilder.io/project/api/v1"

. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
batchv1 "k8s.io/api/batch/v1"
batchv1beta1 "k8s.io/api/batch/v1beta1"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/types"
)

/*
We then set up consts we'll use in our tests, including test names for our objects as well as helper definitions
like SUCCESS and FAILURE to make our tests more readable. The usage of timeout and interval durations will become evident soon.
*/
const (
CronjobName string = "test-cronjob"
CronjobNamespace string = "test-cronjob-namespace"
JobName string = "test-job"

SUCCESS bool = true
FAILURE bool = false

timeout = time.Second * 10
interval = time.Millisecond * 250
)

/*
We also expose the GVK for our CronJob version, which we will use to manage ownership of Jobs that belong to our test CronJob.
One way you can set this GVK is by defining it as a global var like so:
*/
var (
Kind string = reflect.TypeOf(cronjobv1.CronJob{}).Name()
GVK schema.GroupVersionKind = cronjobv1.GroupVersion.WithKind(Kind)
)

/*
We still have a little more setup before we can write our tests.
The first step to writing a simple integration test is to actually create an instance of CronJob you can run tests against.
Note that to create a CronJob, you’ll need to create a stub CronJob struct that contains your CronJob’s specifications.
We define a `testHelper_GetCronJobStub()` function, as well as stub creation functions for downstream objects like `testHelper_GetJobTemplateSpecStub()`,
that can create a CronJob definition for us.
Some k8s Kinds require values for certain spec fields, so you’ll need to make sure your stubs populate those required fields.
*/
func testHelper_GetCronJobStub(name string, namespace string) *cronjobv1.CronJob {
return &cronjobv1.CronJob{
TypeMeta: metav1.TypeMeta{
APIVersion: "batch.tutorial.kubebuilder.io/v1",
Kind: "CronJob",
},
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
},
Spec: cronjobv1.CronJobSpec{
// For simplicity, we only fill out the required fields and fall back to the CronJob CRD's defaults wherever possible.
// Runs every minute!
Schedule: "1 * * * *",
JobTemplate: *testHelper_GetJobTemplateSpecStub(),
},
}
}

/*
We also create a stubbed Job that will help us create Jobs that belong to our test CronJob.
We again make use of a stub function to create our Job, and make use of downstream stub functions like `testHelper_GetJobTemplateSpecStub()`.
*/

func testHelper_JobStub(name string, namespace string, jobStatus batchv1.JobStatus) *batchv1.Job {
return &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
},
Spec: batchv1.JobSpec{
Template: *testHelper_PodTemplateSpecStub(),
},
Status: jobStatus,
}
}

/*
Given that CronJob and Job are a parents of JobTemplate objects, you’ll end up having to create
some stubs for downstream objects that belong to your CronJob.
*/

func testHelper_GetJobTemplateSpecStub() *batchv1beta1.JobTemplateSpec {
return &batchv1beta1.JobTemplateSpec{
Spec: batchv1.JobSpec{
// For simplicity, we only fill out the required fields.
Template: *testHelper_PodTemplateSpecStub(),
},
}
}

func testHelper_PodTemplateSpecStub() *v1.PodTemplateSpec {
return &v1.PodTemplateSpec{
Spec: v1.PodSpec{
// For simplicity, we only fill out the required fields.
Containers: []v1.Container{
{
Name: "test-container",
Image: "test-image",
},
},
RestartPolicy: v1.RestartPolicyOnFailure,
},
}
}

/*
Now we set up the Gingko testing framework to run CronJob controller tests.
*/
var _ = Describe("Run CronJob Controller", func() {
/*
In this test, we use our test k8sClient to create an instance of CronJob!
*/
Context("Can create cronjob", func() {
It("Should create all sub resources successfully", func() {
By("Create cronjob")
ctx := context.Background()
// Creates CronJob test-cronjob in our test cluster.
cronjobCRD := testHelper_GetCronJobStub(CronjobName, CronjobNamespace)
Expect(k8sClient.Create(ctx, cronjobCRD)).Should(Succeed())
})
})

/*
Now that we've created a CronJob in our test cluster, the next step is to write a test that actually tests our CronJob controller’s behavior.
Let’s test the CronJob controller’s logic responsible for updating CronJob.Status.Active with actively running jobs.
We’ll verify that when a CronJob has a single active downstream Job, its CronJob.Status.Active field contains a reference to this Job.
*/

Context("Can perform status updates", func() {
It("CronJob Status.Active count should increase as Jobs are added", func() {
ctx := context.Background()

/*
First, we should get the test CronJob we created earlier, and verify that it currently does not have any active jobs.
Note that, because the k8s apiserver may not have finished creating a CronJob after our `Create()` call from earlier, we will use Gomega’s Eventually() testing function instead of Expect() to give the apiserver an opportunity to finish creating our CronJob.
`Eventually()` will repeatedly run the function provided as an argument every interval seconds until
(a) the function’s output matches what’s expected in the subsequent `Should()` call, or
(b) the number of attempts * timeout period exceed the provided interval value.
In the examples below, let `FAILURE bool = false` and `SUCCESS bool = true`.
timeout and interval are Go Duration values of your choosing.
*/
By("Cronjob status should initially have no active jobs")
cronjobLookupKey := types.NamespacedName{Name: CronjobName, Namespace: CronjobNamespace}
createdCronjob := &cronjobv1.CronJob{}

// We'll need to retry getting this newly creatd CronJob, given that creation may not immediately happen.
Eventually(func() bool {
err := k8sClient.Get(ctx, cronjobLookupKey, createdCronjob)
if err != nil {
return FAILURE
}
return SUCCESS
}, timeout, interval).Should((Equal(SUCCESS)))
Expect(len(createdCronjob.Status.Active)).Should(Equal(0))

/*
Next, we actually create a stubbed Job that will belong to our CronJob.
We again make use of a stub function to create our Job, which we defined earlier with our other stub functions.
We take the stubbed Job and set its owner reference to point to our test CronJob.
This ensures that the test Job belongs to, and is tracked by, our test CronJob.
Once that’s done, we create our new Job instance.
*/
By("Create active job underneath cronjob")
testJob := testHelper_JobStub(JobName, CronjobNamespace, batchv1.JobStatus{
Active: 2,
})

// Set owner reference. Note that your CronJob’s GroupVersionKind is required to set up this owner reference.
controllerRef := metav1.NewControllerRef(createdCronjob, GVK)
testJob.SetOwnerReferences([]metav1.OwnerReference{*controllerRef})
Expect(k8sClient.Create(ctx, testJob)).Should(Succeed())

/*
Adding this Job to our test CronJob should trigger our controller’s reconciler logic.
After that, we can write a test that evaluates whether our controller eventually updates our CronJob’s Status field as expected!
*/
By("Active Jobs should become one")
Eventually(func() bool {
err := k8sClient.Get(ctx, cronjobLookupKey, createdCronjob)
if err != nil {
return FAILURE
}

if len(createdCronjob.Status.Active) != 1 {
return FAILURE
}

for _, job := range createdCronjob.Status.Active {
if job.Name == JobName {
return SUCCESS
}
}
return FAILURE
}, timeout, interval).Should((Equal(SUCCESS)))
})
})

})

/*
After writing all this code, you can run `go test ./...` in your `controllers/` directory again to run your new test!
*/
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
/*
When you created the CronJob API with `kubebuilder create api`, Kubebuilder already did some test work for you.
Kubebuilder generates a controllers/suite_test.go file that does the bare bones of setting up a test environment.
Your automatically generated file will contain multiple helpful components.
First, it will contain necessary imports.
These primarily pull in the Gingko and Gomega testing frameworks and the envtest k8s testing package.
These imports also make available the k8s client you’ll use in your testing, and import the API you build to expose your CronJob Kind.
*/

/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// +kubebuilder:docs-gen:collapse=Apache License

package controllers

import (
"path/filepath"
"testing"

. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"

batchv1 "tutorial.kubebuilder.io/project/api/v1"

"k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/envtest"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
// +kubebuilder:scaffold:imports
)

// These tests use Ginkgo (BDD-style Go testing framework). Refer to
// http://onsi.github.io/ginkgo/ to learn more about Ginkgo.

/*
Now, the start code sets up your envtest cluster.
It loads the CRDs you generated via Kubebuilder, and then starts running the envtest cluster however you configured it
(this example runs a simple envtest cluster of just etcd and the apiserver by default).
After the cluster starts running, we’ll need a client to connect to it.
The autogenerated suite_test.go file will do this for you and also add your custom Kind to the scheme the client will use.
This way, the client will be able to perform CRUD operations on instances of your custom Kind.
*/

var cfg *rest.Config
var k8sClient client.Client // You'll be using this client in your tests.
var testEnv *envtest.Environment

var _ = BeforeSuite(func(done Done) {
logf.SetLogger(zap.LoggerTo(GinkgoWriter, true))

// Step 1: Configure envtest.
// Include CRDDirectoryPaths arg if you're testing a new CRD.
By("bootstrapping test environment")
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{filepath.Join("..", "config", "crd", "bases")},
}

// Step 2: Start envtest!
var err error
cfg, err = testEnv.Start()
Expect(err).ToNot(HaveOccurred())
Expect(cfg).ToNot(BeNil())

// Step 3: If using new CRD, add its corresponding Kind to the client-go
// k8s scheme. After this, it will be available in your k8s clientset.
err = batchv1.AddToScheme(scheme.Scheme)
Expect(err).NotTo(HaveOccurred())

// Step 4: Get that k8s client and use it for tests!
k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme})
Expect(err).ToNot(HaveOccurred())
Expect(k8sClient).ToNot(BeNil())

/*
One thing that this autogenerated file is missing, however, is a way to actually start your controller.
The code above will set up a client that for interacting with your custom Kind,
but will not be able to test your controller behavior.
If you want to test your custom controller logic, you’ll need to add some familiar-looking manager logic
to your BeforeSuite() function, so you can register your custom controller to run on this test cluster.
You may notice that the code below runs your controller with nearly identical logic to your CronJob project’s main.go!
The only difference is that the manager is started in a separate goroutine so it does not block the cleanup of envtest
when you’re done running your tests.
Once you've added the code below, you can actually delete all the code under step 4 above, because you can get k8sClient from the manager
defined in step 5.
*/

// Step 5: Run your test controller
// +kubebuilder:scaffold:scheme
mgr, err := ctrl.NewManager(cfg, ctrl.Options{
Scheme: scheme.Scheme,
})
Expect(err).ToNot(HaveOccurred())
err = (&CronJobReconciler{
Client: mgr.GetClient(),
Log: ctrl.Log.WithName("controllers").WithName("CronJob"),
}).SetupWithManager(mgr)

Expect(err).ToNot(HaveOccurred())
go func() {
err = mgr.Start(ctrl.SetupSignalHandler())
Expect(err).ToNot(HaveOccurred())
}()

k8sClient = mgr.GetClient()
Expect(k8sClient).ToNot(BeNil())

/*
Finally, we close the Gingko test suite's done channel when setup is completed.
*/

close(done)
}, 60)

/*
Kubebuilder also generates also boilerplate functions for cleaning up envtest and actually running your test files in your controllers/ directory.
You won't need to touch these.
*/

var _ = AfterSuite(func() {
By("tearing down the test environment")
err := testEnv.Stop()
Expect(err).ToNot(HaveOccurred())
})

func TestAPIs(t *testing.T) {
RegisterFailHandler(Fail)

RunSpecsWithDefaultAndCustomReporters(t,
"Controller Suite",
[]Reporter{envtest.NewlineReporter{}})
}

/*
Now that you have your controller running on a test cluster and a client ready to perform operations on your CronJob, we can start writing integration tests!
*/
Loading

0 comments on commit 5ad2ca6

Please sign in to comment.