Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

馃尡 Add e2e tests for sample external plugin #3419

Merged
merged 2 commits into from
May 31, 2023
Merged

馃尡 Add e2e tests for sample external plugin #3419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
597d2d6
test: add e2e tests for sample external plugin
Eileen-Yu May 18, 2023
926dd60
ci: add a script to install sample external plugin
Eileen-Yu May 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions test/e2e/ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
source "$(dirname "$0")/../common.sh"
source "$(dirname "$0")/setup.sh"

build_sample_external_plugin

export KIND_CLUSTER="kind"
create_cluster ${KIND_K8S_VERSION}
trap delete_cluster EXIT
Expand Down
32 changes: 32 additions & 0 deletions test/e2e/externalplugin/e2e_suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/*
Copyright 2023 The Kubernetes Authors.
camilamacedo86 marked this conversation as resolved.
Show resolved Hide resolved

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package externalplugin

import (
"fmt"
"testing"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

// Run e2e tests using the Ginkgo runner.
func TestE2E(t *testing.T) {
RegisterFailHandler(Fail)
fmt.Fprintf(GinkgoWriter, "Starting sample external plugin kubebuilder suite\n")
RunSpecs(t, "Kubebuilder sample external plugin e2e suite")
}
111 changes: 111 additions & 0 deletions test/e2e/externalplugin/generate_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
Copyright 2023 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package externalplugin

import (
"path/filepath"

pluginutil "sigs.k8s.io/kubebuilder/v3/pkg/plugin/util"

//nolint:golint
//nolint:revive
. "github.com/onsi/ginkgo/v2"

//nolint:golint
//nolint:revive
. "github.com/onsi/gomega"

//nolint:golint
//nolint:revive
//nolint:golint
//nolint:revive
"sigs.k8s.io/kubebuilder/v3/test/e2e/utils"
)

var _ = Describe("kubebuilder", func() {
Context("plugin sampleexternalplugin/v1", func() {
var (
kbc *utils.TestContext
)

BeforeEach(func() {
var err error
kbc, err = utils.NewTestContext(pluginutil.KubebuilderBinName, "GO111MODULE=on")
Expect(err).NotTo(HaveOccurred(), "Prepare NewTestContext should return no error.")
Expect(kbc.Prepare()).To(Succeed())
})

AfterEach(func() {
kbc.Destroy()
})

It("should generate a runnable project with sample external plugin", func() {
GenerateProject(kbc)
})

})
})

// GenerateProject implements a sampleexternalplugin/v1 external plugin project defined by a TestContext.
func GenerateProject(kbc *utils.TestContext) {
var err error

By("initializing a project")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See that before init you need to ensure that the plugin was built.
It is not better build the plugin here instead of create the sh?
Could we not move the code to here and to do that in go so that is easier to keep maintained as well?

See that you can run it locally either if you move all to here by just trigging the tests from your IDE

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't disagree that it might be easier to maintain, but I think there is a lot of precedence to build stuff in a shell script for the e2e tests. IMO we shouldn't try to reinvent the wheel by adding this to the go code and follow the existing pattern for building and configuring e2e tests with shell scripts.

Copy link
Member

@camilamacedo86 camilamacedo86 May 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @everettraven,

but I think there is a lot of precedence to build stuff in a shell script for the e2e tests.

We currently do not utilize any shell scripting to accomplish the tasks required for our end-to-end (e2e) tests. It's important to note that these tests can be executed locally as long as you have a functional Kubernetes cluster. However, after this PR is merged, this functionality will likely be disrupted and might no longer be possible.

To elaborate, introducing shell scripting for e2e tests as we are doing here will break the development environment. This is contrary to our current setup. The shells scripts in place are Optional to setup the k8s locally either and btw I never used them or to call the test on prow.

Nonetheless, if @Eileen-Yu is able to successfully integrate shell scripting without compromising the functionality (despite my concerns that it will negatively affect the ability to run e2e tests in a development environment), I'm open to having this PR merged and making necessary changes in a follow-up. Please note, however, that it appears these tests are currently failing due to the lack of the necessary steps within the e2e test environment. Therefore, when the init command is called, it results in a failure. See the following for more details:"

     s: "kubebuilder init --plugins sampleexternalplugin/v1 --domain sample.domain.com failed with error: (exit status 1) Error: no plugin could be resolved with key \"sampleexternalplugin/v1\"\nUsage:\n  kubebuilder [flags]\n\nExamples:\nThe first step is to initialize your project:\n    kubebuilder init [--plugins=<PLUGIN KEYS> [--project-version=<PROJECT VERSION>]]\n\n<PLUGIN KEYS> is a comma-separated list of plugin keys from the following table\nand <PROJECT VERSION> a supported project version for these plugins.\n\n                             Plugin keys | Supported project versions\n-----------------------------------------+----------------------------\n               base.go.kubebuilder.io/v3 |                          3\n               base.go.kubebuilder.io/v4 |                          3\n        declarative.go.kubebuilder.io/v1 |                       2, 3\n deploy-image.go.kubebuilder.io/v1-alpha |                          3\n                    go.kubebuilder.io/v2 |                       2, 3\n                    go.kubebuilder.io/v3 |                          3\n                    go.kubebuilder.io/v4 |                          3\n         grafana.kubebuilder.io/v1-alpha |                          3\n      kustomize.common.kubebuilder.io/v1 |                          3\n      kustomize.common.kubebuilder.io/v2 |                          3\n\nFor more specific help for the init command of a certain plugins and project version\nconfiguration please run:\n    kubebuilder init --help --plugins=<PLUGIN KEYS> [--project-version=<PROJECT VERSION>]\n\nDefault plugin keys: \"go.kubebuilder.io/v4\"\nDefault project version: \"3\"\n\n\nFlags:\n  -h, --help                     help for kubebuilder\n      --plugins strings          plugin keys to be used for this subcommand execution\n      --project-version string   project version (default \"3\")\n\n2023/05/24 13:03:58 no plugin could be resolved with key \"sampleexternalplugin/v1\"\n",
      }
  occurred

Copy link
Contributor

@Kavinjsir Kavinjsir May 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @camilamacedo86 , I'm trying to follow your idea:

Since test/e2e/utils provides context to bring rich prerequisites, including k8s cluster, kubectl, kubebuilder.
That is how e2e tests get performed for internal plugins.

So, it is not necessary/consistent to deal with prerequisites on a different pattern, such as running a new script.

Is that something you mentioned of and such external-plugin should NOT break?

We currently do not utilize any shell scripting to accomplish the tasks required for our end-to-end (e2e) tests.

Here is my opinion:
We have a well-structured directory under test/e2e:

  1. those single files such as local.sh, setup.sh are used to provide the general environment for e2e testing
  2. plugin related stuff are put into separate files such as v4, deployimage

To maintain that way, when we create a new e2e test suite for external-plugin, it is not necessary/recommended to touch those public files in 1.

Then, if building and configuring external-plugin is needed, we can follow what we did to other plugins:

  1. At the very beginning in the ginkgo context, use go code to build and configure external-plugin
  2. At the end of the ginkgo context, delete the temporary asset for external-plugin

err = kbc.Init(
"--plugins", "sampleexternalplugin/v1",
"--domain", "sample.domain.com",
)
ExpectWithOffset(1, err).NotTo(HaveOccurred())

var initFileContentsTmpl = "A simple text file created with the `init` subcommand\nDOMAIN: sample.domain.com"
initFileContainsExpr, err := pluginutil.HasFileContentWith(
filepath.Join(kbc.Dir, "initFile.txt"), initFileContentsTmpl)
ExpectWithOffset(1, err).NotTo(HaveOccurred(), "Check initFile.txt should return no error.")
ExpectWithOffset(1, initFileContainsExpr).To(BeTrue(), "The init file does not contain the expected expression.")

By("creating API definition")
err = kbc.CreateAPI(
"--plugins", "sampleexternalplugin/v1",
"--number=2",
"--group", kbc.Group,
"--version", kbc.Version,
"--kind", kbc.Kind,
)
ExpectWithOffset(1, err).NotTo(HaveOccurred())

var apiFileContentsTmpl = "A simple text file created with the `create api` subcommand\nNUMBER: 2"
apiFileContainsExpr, err := pluginutil.HasFileContentWith(
filepath.Join(kbc.Dir, "apiFile.txt"), apiFileContentsTmpl)
ExpectWithOffset(1, err).NotTo(HaveOccurred(), "Check apiFile.txt should return no error.")
Eileen-Yu marked this conversation as resolved.
Show resolved Hide resolved
ExpectWithOffset(1, apiFileContainsExpr).To(BeTrue(), "The api file does not contain the expected expression.")

By("scaffolding webhook")
err = kbc.CreateWebhook(
"--plugins", "sampleexternalplugin/v1",
"--hooked",
"--group", kbc.Group,
"--version", kbc.Version,
"--kind", kbc.Kind,
)
ExpectWithOffset(1, err).NotTo(HaveOccurred())

var webhookFileContentsTmpl = "A simple text file created with the `create webhook` subcommand\nHOOKED!"
webhookFileContainsExpr, err := pluginutil.HasFileContentWith(
filepath.Join(kbc.Dir, "webhookFile.txt"), webhookFileContentsTmpl)
ExpectWithOffset(1, err).NotTo(HaveOccurred(), "Check webhookFile.txt should return no error.")
ExpectWithOffset(1, webhookFileContainsExpr).To(BeTrue(), "The webhook file does not contain the expected expression.")
}
2 changes: 2 additions & 0 deletions test/e2e/local.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
source "$(dirname "$0")/../common.sh"
source "$(dirname "$0")/setup.sh"

build_sample_external_plugin

export KIND_CLUSTER="local-kubebuilder-e2e"
create_cluster ${KIND_K8S_VERSION}
if [ -z "${SKIP_KIND_CLEANUP:-}" ]; then
Expand Down
31 changes: 31 additions & 0 deletions test/e2e/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,35 @@ function test_cluster {
go test $(dirname "$0")/grafana $flags -timeout 30m
go test $(dirname "$0")/deployimage $flags -timeout 30m
go test $(dirname "$0")/v4 $flags -timeout 30m
go test $(dirname "$0")/externalplugin $flags -timeout 30m
}

function build_sample_external_plugin {
# TODO: Dynamically set exteranl plugin destination based on OS platform
# EXTERNAL_PLUGIN_DESTINATION_PREFIX="${HOME}/Library/Application Support/kubebuilder/plugins"
# For Linux:
XDG_CONFIG_HOME="${HOME}/.config"
EXTERNAL_PLUGIN_DESTINATION_PREFIX="$XDG_CONFIG_HOME/kubebuilder/plugins"

PLUGIN_NAME="sampleexternalplugin"
PLUGIN_VERSION="v1"
EXTERNAL_PLUGIN_DESTINATION="${EXTERNAL_PLUGIN_DESTINATION_PREFIX}/${PLUGIN_NAME}/${PLUGIN_VERSION}"
EXTERNAL_PLUGIN_PATH="${EXTERNAL_PLUGIN_DESTINATION}/${PLUGIN_NAME}"

if [ -d "$EXTERNAL_PLUGIN_DESTINATION" ]; then
echo "$EXTERNAL_PLUGIN_DESTINATION does exist."
if [ -e "$EXTERNAL_PLUGIN_PATH" ]; then
echo "clean up old binary..."
rm "$EXTERNAL_PLUGIN_PATH"
fi
else
mkdir -p "$EXTERNAL_PLUGIN_DESTINATION"
fi

REPO_ROOT_DIR="$(git rev-parse --show-toplevel)"
SOURCE_DIR="${REPO_ROOT_DIR}/docs/book/src/simple-external-plugin-tutorial/testdata/sampleexternalplugin/v1"

cd $SOURCE_DIR && go build -o $PLUGIN_NAME && mv $PLUGIN_NAME "$EXTERNAL_PLUGIN_PATH"

cd $REPO_ROOT_DIR
}
Loading