diff --git a/docs/development.md b/docs/development.md index a247ea993a..f5fa3aacbb 100644 --- a/docs/development.md +++ b/docs/development.md @@ -20,140 +20,7 @@ cd kpt-config-sync ## Run tests -### Unit tests -Unit tests are small focused test that runs quickly. Run them with: -``` -make test -``` - -### E2E tests - -Config Sync also has e2e tests. These can be run on [kind] or GKE and can take -a long time to finish. - -The e2e tests will use the most recently built manifests on your local filesystem, -which are written to the `.output/staging` directory. These are created -when running `make` targets such as `build-manifests` and `config-sync-manifest`. -See [building from source](#build-from-source) for more information. - -For the complete list of arguments accepted by the e2e tests, see [flags.go](../e2e/flags.go) -or invoke the e2e tests with the `--usage` flag. -```shell -go test ./e2e/... --usage -``` - -Below is a non-exhaustive list of some useful arguments for running the e2e tests. -These can be provided on the command line with `go test` or with program arguments in your IDE. - -- `--usage` - If true, print usage and exit. -- `--e2e` - If true, run end-to-end tests. (required to run the e2e tests). -- `--debug` - If true, do not destroy cluster and clean up temporary directory after test. -- `--share-test-env` - Specify that the test is using a shared test environment instead of fresh installation per test case. -- `--test-cluster` - The cluster config used for testing. Allowed values are: `kind` and `gke`. -- `--num-clusters` - Number of clusters to run tests on in parallel. Overrides the `--test.parallel` flag. -- `--create-clusters` - Whether to create clusters in the test framework. Allowed values are [`true`, `lazy`, `false`]. If set to `lazy`, the tests will adopt an existing cluster. -- `--destroy-clusters` - Whether to destroy clusters after test execution. Allowed values are [`true`, `auto`, `false`]. If set to `auto`, the tests will only destroy the cluster if it was created by the tests. - -Here are some useful flags from [go test](https://pkg.go.dev/cmd/go#hdr-Testing_flags): -- `--test.v` - More verbose output. -- `--test.run` -- Run only tests matching the provided regular expression. - -### E2E tests (kind) - -This section provides instructions on how to run the e2e tests on [kind]. - -#### Prerequisites - -Install [kind] -```shell -make install-kind -``` - -- This will put kind in `$(go env GOPATH)/bin`. This directory will need to be added to `$PATH` if it isn't already. -- After upgrading kind, you usually need to delete all your existing kind clusters so that kind functions correctly. -- Deleting all running kind clusters usually fixes kind issues. - -#### Running E2E tests on kind - -Run all of the tests (this will take a while): -``` -make test-e2e-go-multirepo -``` - -To execute e2e multi-repo tests locally with kind, build and push the Config Sync -images to the local kind registry and then execute tests using go test. -```shell -make config-sync-manifest-local -go test ./e2e/... --e2e --debug --test.v --test.run (test name regexp) -``` - -To use already existing images without building everything from scratch, rebuild -only the manifests and then rerun the tests. -```shell -make build-manifests IMAGE_TAG= -go test ./e2e/... --e2e -``` - -### E2E tests (GKE) - -This section provides instructions on how to run the e2e tests on GKE. - -#### Prerequisites - -Follow the [instructions to provision a dev environment]. - -To execute e2e multi-repo tests with a GKE cluster, build and push the Config Sync -images to GCR and then use go test. The images will be pushed to the GCP project -from you current gcloud context. - -```shell -# Ensure gcloud context is set to correct project -gcloud config set project -# Build images/manifests and push images -make config-sync-manifest -``` - -#### Running E2E tests on GKE (test-generated GKE clusters) - -The e2e test scaffolding supports creating N clusters before test execution and -reusing them between tests. This enables running the tests in parallel, where the -number of clusters equals the number of test threads. The clusters can be configured -to be deleted or orphaned after test execution. - -Provide the `--create-clusters` option with: -- `true` - always create clusters with tests (error if already exist) -- `lazy` - create or adopt cluster if already exists -- `false` - never create clusters with tests (error if not found) - -Provide the `--destroy-clusters` option with: -- `true` - always destroy clusters after tests -- `auto` - only destroy clusters if they were created by the tests -- `false` - never destroy clusters after tests - -```shell -# GKE cluster options can be provided as command line flags -go test ./e2e/... --e2e --test.v --share-test-env --gcp-project= --test-cluster=gke \ - --cluster-prefix=${USER}-test --create-clusters=lazy --destroy-clusters=false --num-clusters=5 --gcp-zone=us-central1-a \ - --test.run (test name regexp) -``` - -#### Running E2E tests on GKE (pre-provisioned GKE clusters) - -The e2e test scaffolding also supports accepting the name of a pre-provisioned -cluster. It will generate the kubeconfig based on the inputs provided. This mode -only supports running a single test thread against a single cluster. The cluster -will not be deleted at the end of the test execution. - -```shell -# GKE cluster options can also be provided as env vars -export GCP_PROJECT= -export GCP_CLUSTER= -# One of GCP_REGION and GCP_ZONE must be set (but not both) -export GCP_REGION= -export GCP_ZONE= -# Run the tests with image prefix/tag from previous step and desired test regex -go test ./e2e/... --e2e --debug --test.v --share-test-env=true --test-cluster=gke --test.run (test name regexp) -``` +See [testing](./testing.md). ## Build @@ -271,5 +138,3 @@ make run-oss [make]: https://www.gnu.org/software/make/ [docker]: https://www.docker.com/get-started [create your own fork]: https://docs.github.com/en/get-started/quickstart/fork-a-repo -[kind]: https://kind.sigs.k8s.io/ -[instructions to provision a dev environment]: ../e2e/testinfra/terraform/README.md diff --git a/docs/installation.md b/docs/installation.md index 4223e24bf0..0a8dc4770d 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -24,10 +24,10 @@ Sync images. 1. Follow the [development instructions] to build Config Sync from source. 2. Upon success the docker images are published to your GCR repository and the -KRM manifests are placed in `./.output/oss/manifests`. The manifests can be +KRM manifests are placed in `./.output/staging/oss`. The manifests can be applied directly to your cluster. ```shell -kubectl apply -f ./.output/oss/manifests +kubectl apply -f ./.output/staging/oss/config-sync-manifest.yaml ``` 3. If the Pod is in a `ImagePullBackoff` or `ErrImagePull` state, that indicates the Compute Engine default service account doesn't have permission to pull diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000000..2fce456440 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,140 @@ +# Testing + +### Unit tests +Unit tests are small focused tests that runs quickly. Run them with: + +``` +make test +``` + +### E2E tests + +Config Sync also has e2e tests. These can be run on [kind] or GKE and can take +a long time to finish. + +The e2e tests will use the most recently built manifests on your local filesystem, +which are written to the `.output/staging` directory. These are created +when running `make` targets such as `build-manifests` and `config-sync-manifest`. +See [building from source](#build-from-source) for more information. + +For the complete list of arguments accepted by the e2e tests, see [flags.go](../e2e/flags.go) +or invoke the e2e tests with the `--usage` flag. +```shell +go test ./e2e/... --usage +``` + +Below is a non-exhaustive list of some useful arguments for running the e2e tests. +These can be provided on the command line with `go test` or with program arguments in your IDE. + +- `--usage` - If true, print usage and exit. +- `--e2e` - If true, run end-to-end tests. (required to run the e2e tests). +- `--debug` - If true, do not destroy cluster and clean up temporary directory after test. +- `--share-test-env` - Specify that the test is using a shared test environment instead of fresh installation per test case. +- `--test-cluster` - The cluster config used for testing. Allowed values are: `kind` and `gke`. +- `--num-clusters` - Number of clusters to run tests on in parallel. Overrides the `--test.parallel` flag. +- `--create-clusters` - Whether to create clusters in the test framework. Allowed values are [`true`, `lazy`, `false`]. If set to `lazy`, the tests will adopt an existing cluster. +- `--destroy-clusters` - Whether to destroy clusters after test execution. Allowed values are [`true`, `auto`, `false`]. If set to `auto`, the tests will only destroy the cluster if it was created by the tests. + +Here are some useful flags from [go test](https://pkg.go.dev/cmd/go#hdr-Testing_flags): +- `--test.v` - More verbose output. +- `--test.run` -- Run only tests matching the provided regular expression. + +### E2E tests (kind) + +This section provides instructions on how to run the e2e tests on [kind]. + +#### Prerequisites + +Install [kind] +```shell +make install-kind +``` + +- This will put kind in `$(go env GOPATH)/bin`. This directory will need to be added to `$PATH` if it isn't already. +- After upgrading kind, you usually need to delete all your existing kind clusters so that kind functions correctly. +- Deleting all running kind clusters usually fixes kind issues. + +#### Running E2E tests on kind + +Run all of the tests (this will take a while): +``` +make test-e2e-go-multirepo +``` + +To execute e2e multi-repo tests locally with kind, build and push the Config Sync +images to the local kind registry and then execute tests using go test. +```shell +make config-sync-manifest-local +go test ./e2e/... --e2e --debug --test.v --test.run (test name regexp) +``` + +To use already existing images without building everything from scratch, rebuild +only the manifests and then rerun the tests. +```shell +make build-manifests IMAGE_TAG= +go test ./e2e/... --e2e +``` + +### E2E tests (GKE) + +This section provides instructions on how to run the e2e tests on GKE. + +#### Prerequisites + +Follow the [instructions to provision a dev environment]. + +To execute e2e multi-repo tests with a GKE cluster, build and push the Config Sync +images to GCR and then use go test. The images will be pushed to the GCP project +from you current gcloud context. + +```shell +# Ensure gcloud context is set to correct project +gcloud config set project +# Build images/manifests and push images +make config-sync-manifest +``` + +#### Running E2E tests on GKE (test-generated GKE clusters) + +The e2e test scaffolding supports creating N clusters before test execution and +reusing them between tests. This enables running the tests in parallel, where the +number of clusters equals the number of test threads. The clusters can be configured +to be deleted or orphaned after test execution. + +Provide the `--create-clusters` option with: +- `true` - always create clusters with tests (error if already exist) +- `lazy` - create or adopt cluster if already exists +- `false` - never create clusters with tests (error if not found) + +Provide the `--destroy-clusters` option with: +- `true` - always destroy clusters after tests +- `auto` - only destroy clusters if they were created by the tests +- `false` - never destroy clusters after tests + +```shell +# GKE cluster options can be provided as command line flags +go test ./e2e/... --e2e --test.v --share-test-env --gcp-project= --test-cluster=gke \ + --cluster-prefix=${USER}-test --create-clusters=lazy --destroy-clusters=false --num-clusters=5 --gcp-zone=us-central1-a \ + --test.run (test name regexp) +``` + +#### Running E2E tests on GKE (pre-provisioned GKE clusters) + +The e2e test scaffolding also supports accepting the name of a pre-provisioned +cluster. It will generate the kubeconfig based on the inputs provided. This mode +only supports running a single test thread against a single cluster. The cluster +will not be deleted at the end of the test execution. + +```shell +# GKE cluster options can also be provided as env vars +export GCP_PROJECT= +export GCP_CLUSTER= +# One of GCP_REGION and GCP_ZONE must be set (but not both) +export GCP_REGION= +export GCP_ZONE= +# Run the tests with image prefix/tag from previous step and desired test regex +go test ./e2e/... --e2e --debug --test.v --share-test-env=true --test-cluster=gke --test.run (test name regexp) +``` + +[kind]: https://kind.sigs.k8s.io/ +[instructions to provision a dev environment]: ../e2e/testinfra/terraform/README.md