Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kind-dev command to start a kind dev cluster #338

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ kind-test: ## Run functional tests for SERVICE
kind-helm-test: ## Run the Helm tests for SERVICE
@AWS_SERVICE=$(AWS_SERVICE) ./scripts/run-helm-tests.sh

kind-dev: ## Run a dev cluster for SERVICE
@AWS_SERVICE=$(AWS_SERVICE) ./scripts/run-dev.sh

test-recommended-policy:
@AWS_SERVICE=$(AWS_SERVICE) source ./scripts/iam-policy-test-runner.sh && assert_iam_policies

Expand Down
43 changes: 26 additions & 17 deletions scripts/controller-setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,27 +38,11 @@ build_and_install_controller() {
_install_deployment "$__controller_namespace" "$__img_name"
}

_build_controller_image() {
local __img_name=$1

local local_build="$(get_is_local_build)"
LOCAL_MODULES="$local_build" AWS_SERVICE_DOCKER_IMG="$__img_name" ${CODE_GENERATOR_SCRIPTS_DIR}/build-controller-image.sh ${AWS_SERVICE} 1>/dev/null
}

_load_controller_image() {
local __cluster_name=$1
local __img_name=$2

kind load docker-image --name "$__cluster_name" --nodes="$__cluster_name"-worker,"$__cluster_name"-control-plane "$__img_name"
}

_install_deployment() {
install_crd_and_rbac(){
local __controller_namespace=$1
local __img_name=$2

local service_controller_source_dir="$ROOT_DIR/../$AWS_SERVICE-controller"
local service_config_dir="$service_controller_source_dir/config"
local test_config_dir="$ROOT_DIR/build/clusters/$cluster_name/config/test"

# Register the ACK service controller's CRDs in the target k8s cluster
debug_msg "Loading CRD manifests for $AWS_SERVICE into the cluster ... "
Expand All @@ -76,6 +60,31 @@ _install_deployment() {

debug_msg "Loading RBAC manifests for $AWS_SERVICE into the cluster ... "
kustomize build "$service_config_dir"/rbac | kubectl apply -f - 1>/dev/null
}

_build_controller_image() {
local __img_name=$1

local local_build="$(get_is_local_build)"
LOCAL_MODULES="$local_build" AWS_SERVICE_DOCKER_IMG="$__img_name" ${CODE_GENERATOR_SCRIPTS_DIR}/build-controller-image.sh ${AWS_SERVICE} 1>/dev/null
}

_load_controller_image() {
local __cluster_name=$1
local __img_name=$2

kind load docker-image --name "$__cluster_name" --nodes="$__cluster_name"-worker,"$__cluster_name"-control-plane "$__img_name"
}

_install_deployment() {
local __controller_namespace=$1
local __img_name=$2

install_crd_and_rbac "$__controller_namespace"

local service_controller_source_dir="$ROOT_DIR/../$AWS_SERVICE-controller"
local service_config_dir="$service_controller_source_dir/config"
local test_config_dir="$ROOT_DIR/build/clusters/$cluster_name/config/test"

# Create the ACK service controller Deployment in the target k8s cluster
mkdir -p "$test_config_dir"
Expand Down
63 changes: 63 additions & 0 deletions scripts/run-dev.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think run-dev is a very descriptive name for this file. Could you please rename it to something like setup-kind-cluster or create-dev-cluster?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya , renamed to create-dev-cluster

Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env bash

# ./scripts/run-dev.sh quickly setup cluster for ACK development. It
# ensures that a K8s cluster is accessible, then configures the ACK CRD, RBAC
# but not install the controller

set -Eeo pipefail

SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
ROOT_DIR="$SCRIPTS_DIR/.."

AWS_SERVICE=$(echo "${AWS_SERVICE:-""}" | tr '[:upper:]' '[:lower:]')

CONTROLLER_NAMESPACE=${CONTROLLER_NAMESPACE:-"ack-system"}

CLUSTER_NAME="ack-dev-cluster-$AWS_SERVICE"

source "$SCRIPTS_DIR/lib/aws.sh"
source "$SCRIPTS_DIR/lib/common.sh"
source "$SCRIPTS_DIR/lib/config.sh"
source "$SCRIPTS_DIR/lib/logging.sh"

source "$SCRIPTS_DIR/controller-setup.sh"
source "$SCRIPTS_DIR/kind.sh"

ensure_cluster() {
info_msg "Creating KIND cluster ..."
setup_kind_cluster "$CLUSTER_NAME" "$CONTROLLER_NAMESPACE"

info_msg "Installing CRDs , common and RBAC manifest..."
install_crd_and_rbac "$CONTROLLER_NAMESPACE"
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have an ensure_cluster method in another file that achieves a similar premise - https://github.com/aws-controllers-k8s/test-infra/blob/main/scripts/run-e2e-tests.sh#L31-L44

This method also uses the config YAML to determine the name, in case the user wants to use a statically named cluster. Rather than redefining this method here, maybe you could modify that one to have a similar effect and then import and use it in this script.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactor here refactor ensure_cluster


run() {
ensure_aws_credentials
ensure_cluster
local kubeconfig_path="$ROOT_DIR/build/clusters/$CLUSTER_NAME/kubeconfig"
info_msg "Before running the controller, you need kubeconfig and aws credentials."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Probably don't need this line of explanation. I think we can assume anyone using this method already knows this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

info_msg "After executing the above source command to set the kubeconfig to environment, "
info_msg "run the following command to start the controller from the controller repo:"
echo ""
echo "go run ./cmd/controller/main.go --aws-region eu-central-1 --log-level debug --enable-development-logging"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Instead of hardcoding eu-central-1, get the region from the config YAML (like this)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo ""
info_msg "if you run the controller from your code editor/IDE, you can set the following environment variables:"
echo "KUBECONFIG=$kubeconfig_path"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already called out from inside the setup_kind_cluster method. See https://github.com/aws-controllers-k8s/test-infra/blob/main/scripts/kind.sh#L27

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the source command set the kubeconfig in current terminal only, if starting the controller from goland or vscode, we have to set KUBECONFIG into them. but you're right, better to mention in document rather than here.

exit $?
}

ensure_inputs() {
[[ -z "$AWS_SERVICE" ]] && { error_msg "Expected \`AWS_SERVICE\` to be defined"; exit 1; } || :
}

ensure_binaries() {
check_is_installed "kubectl"
}

ensure_inputs
ensure_binaries

# The purpose of the `return` subshell command in this script is to determine
# whether the script was sourced, or whether it is being executed directly.
# https://stackoverflow.com/a/28776166
(return 0 2>/dev/null) || run