Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: prepare README for 0.2.0 GA #76

Merged
merged 3 commits into from
Aug 8, 2024
Merged

docs: prepare README for 0.2.0 GA #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f4b845b
docs: prepare README for 0.2.0 GA
johannesfrey Jul 16, 2024
ebbfa1e
docs: adjust quickstart guide
johannesfrey Jul 16, 2024
4ac519f
chore: rename HELM_VALUES to VCLUSTER_YAML
johannesfrey Aug 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ jobs:
export CLUSTER_NAME=vcluster-k3s
export CLUSTER_NAMESPACE=vcluster-k3s
export CHART_VERSION=0.20.0-beta.12
export HELM_VALUES=$(cat ./test/e2e/k3s-values.yaml | sed -z 's/\n/\\n/g')
export VCLUSTER_YAML=$(cat ./test/e2e/k3s-values.yaml | sed -z 's/\n/\\n/g')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -

Expand All @@ -131,7 +131,7 @@ jobs:
export CLUSTER_NAMESPACE=vcluster-k0s
export CHART_VERSION=0.20.0-beta.12
export CHART_NAME=vcluster
export HELM_VALUES=$(cat ./test/e2e/k0s-values.yaml | sed -z 's/\n/\\n/g')
export VCLUSTER_YAML=$(cat ./test/e2e/k0s-values.yaml | sed -z 's/\n/\\n/g')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -

Expand All @@ -148,7 +148,7 @@ jobs:
export CLUSTER_NAMESPACE=vcluster-k8s
export CHART_VERSION=0.20.0-beta.12
export CHART_NAME=vcluster
export HELM_VALUES=$(cat ./test/e2e/k8s-values.yaml | sed -z 's/\n/\\n/g')
export VCLUSTER_YAML=$(cat ./test/e2e/k8s-values.yaml | sed -z 's/\n/\\n/g')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -

Expand All @@ -165,7 +165,7 @@ jobs:
export CLUSTER_NAMESPACE=vcluster-eks
export CHART_VERSION=0.20.0-beta.12
export CHART_NAME=vcluster
export HELM_VALUES=$(cat ./test/e2e/eks-values.yaml | sed -z 's/\n/\\n/g')
export VCLUSTER_YAML=$(cat ./test/e2e/eks-values.yaml | sed -z 's/\n/\\n/g')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -

Expand Down
143 changes: 37 additions & 106 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,37 +8,29 @@ This is a [Cluster API](https://cluster-api.sigs.k8s.io/introduction.html) provi
# Installation instructions

Prerequisites:
- [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl) (v1.1.5+)
- [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl) (v1.7.4+)
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- A Kubernetes cluster where you will have cluster-admin permissions
- Optional, depending on how you expose the vcluster instance - [vcluster CLI](https://www.vcluster.com/docs/getting-started/setup)

Install the vcluster provider (latest stable release)
Install the vcluster provider

```shell
clusterctl init --infrastructure vcluster
```

or for the latest alpha release (which uses the v0.20 [vcluster.yaml](https://www.vcluster.com/docs/vcluster/configure/vcluster-yaml/) format per default):

```shell
clusterctl init --infrastructure vcluster:v0.2.0-alpha.2
```

Next you will generate a manifest file for a vcluster instance and create it in the management cluster.
Cluster instance is configured using clusterctl parameters and environment variables - CHART_NAME, CHART_REPO, CHART_VERSION, VCLUSTER_HOST and VCLUSTER_PORT.
In the example commands below, the HELM_VALUES variable will be populated with the contents of the `values.yaml` file.
In the example commands below, the VCLUSTER_YAML variable will be populated with the contents of the `values.yaml` file.
```shell
export CLUSTER_NAME=vcluster
export CLUSTER_NAMESPACE=vcluster
export KUBERNETES_VERSION=1.28.9
johannesfrey marked this conversation as resolved.
Show resolved Hide resolved
export HELM_VALUES=""
export VCLUSTER_YAML=""
# Uncomment if you want to use vcluster values
# export HELM_VALUES=$(cat devvalues.yaml | sed -z 's/\n/\\n/g')
# export VCLUSTER_YAML=$(cat devvalues.yaml | awk '{printf "%s\\n", $0}')
kubectl create namespace ${CLUSTER_NAMESPACE}
clusterctl generate cluster ${CLUSTER_NAME} \
--infrastructure vcluster \
--kubernetes-version ${KUBERNETES_VERSION} \
--target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -
```

Expand All @@ -51,7 +43,7 @@ At this point the cluster is ready to be used. Please refer to the next chapter
**Note**: at the moment, the provider is able to host vclusters only in the cluster where the vcluster provider is running([management cluster](https://cluster-api.sigs.k8s.io/user/concepts.html#management-cluster)). Support for the remote host clusters is on our roadmap - [loft-sh/cluster-api-provider-vcluster#6](https://github.com/loft-sh/cluster-api-provider-vcluster/issues/6).

# How to connect to your vcluster
There are multiple methods for exposing your vcluster instance, and they are described in the [vcluster docs](https://www.vcluster.com/docs/operator/external-access). If you follow the docs exactly, you will need to use the vcluster CLI to retrieve kubeconfig. When using this CAPI provider you have an alternative - `clusterctl get kubeconfig ${CLUSTER_NAME} --namespace ${CLUSTER_NAMESPACE} > ./kubeconfig.yaml`, more details about this are in the [CAPI docs](https://cluster-api.sigs.k8s.io/clusterctl/commands/get-kubeconfig.html). Virtual cluster kube config will be written to: ./kubeconfig.yaml. You can access the cluster via `kubectl --kubeconfig ./kubeconfig.yaml get namespaces`.
There are multiple methods for exposing your vcluster instance, and they are described in the [vcluster docs](https://www.vcluster.com/docs/operator/external-access). If you follow the docs exactly, you will need to use the vcluster CLI to retrieve kubeconfig. When using this CAPI provider you have an alternative - `clusterctl get kubeconfig ${CLUSTER_NAME} --namespace ${CLUSTER_NAMESPACE} > ./kubeconfig.yaml`, more details about this are in the [CAPI docs](https://cluster-api.sigs.k8s.io/clusterctl/commands/get-kubeconfig.html). You can access the cluster via `kubectl --kubeconfig ./kubeconfig.yaml get namespaces`.

However, if you are not exposing the vcluster instance with an external hostname, but you want to connect to it from outside the cluster, you will need to use the [vcluster CLI](https://www.vcluster.com/docs/getting-started/setup):
```shell
Expand Down Expand Up @@ -86,13 +78,6 @@ kind: VCluster
metadata:
name: ${CLUSTER_NAME}
spec:
# Kubernetes version that should be used in this vcluster instance, e.g. "1.23".
# The patch number from the version will be ignored, and the latest supported one
# by the used chart will be installed.
# Versions out of the supported range will be ignored, and earliest/latest supported
# version will be used instead.
kubernetesVersion: "1.24"

# We are using vcluster Helm charts for the installation and upgrade.
# The helmRelease sub-fields allow you to set Helm values and chart repo, name and version.
# Sources of the charts can be found here - https://github.com/loft-sh/vcluster/tree/main/charts
Expand All @@ -104,10 +89,14 @@ spec:
# Please refer to vcluster documentation for the extensive explanation of the features,
# and the appropriate Helm values that need to be set for your use case - https://www.vcluster.com/docs
values: |-
# example:
# syncer:
# extraArgs:
# - --tls-san=myvcluster.mydns.abc
# controlPlane:
# distro:
# k8s:
# enabled: true
# backingStore:
# etcd:
# deploy:
# enabled: true

chart:
# By default, the "https://charts.loft.sh" repo is used
Expand All @@ -122,7 +111,7 @@ spec:
#
# Please refer to the vcluster Releases page for the list of the available versions:
# https://github.com/loft-sh/vcluster/releases
version: ${CHART_VERSION:=0.19.5}
version: ${CHART_VERSION:=0.20.0}

# controlPlaneEndpoint represents the endpoint used to communicate with the control plane.
# You may leave this field empty, and then CAPVC will try to fill in this information based
Expand Down Expand Up @@ -172,13 +161,13 @@ go run -mod vendor main.go

Next, in a separate terminal you will generate a manifest file for a vcluster instance.
Cluster instance is configured from a template file using environment variables - CLUSTER_NAME, CHART_NAME, CHART_REPO, CHART_VERSION, VCLUSTER_HOST and VCLUSTER_PORT. Only the CHART_VERSION and CLUSTER_NAME variables are mandatory.
In the example commands below, the HELM_VALUES variable will be populated with the contents of the `devvalues.yaml` file, don't forget to re-run the `export HELM_VALUES...` command when the `devvalues.yaml` changes.
In the example commands below, the VCLUSTER_YAML variable will be populated with the contents of the `devvalues.yaml` file, don't forget to re-run the `export VCLUSTER_YAML...` command when the `devvalues.yaml` changes.
```shell
export CLUSTER_NAME=test
export CLUSTER_NAMESPACE=test
export CHART_VERSION=0.19.5
export CHART_VERSION=0.20.0
export CHART_NAME=vcluster
export HELM_VALUES=$(cat devvalues.yaml | sed -z 's/\n/\\n/g')
export VCLUSTER_YAML=$(cat devvalues.yaml | awk '{printf "%s\\n", $0}')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -
```
Expand All @@ -189,95 +178,37 @@ kubectl wait --for=condition=ready vcluster -n $CLUSTER_NAMESPACE $CLUSTER_NAME
```
At this point the cluster is ready to be used. Please refer to "How to connect to your vcluster" chapter above to get the credentials.

# Specifying Custom vCluster Helm Versions
# Customizing Helm related values

## Specifiying Chart related values

You can specify a custom version of the vCluster Helm chart by setting the CHART_VERSION environment variable. This allows you to pin the vCluster installation to a specific version of the Helm chart.

Example:

```shell
export CHART_VERSION=0.19.5
```

## Specifying Custom Images for Different Kubernetes Distributions in vCluster

Depending on your needs, you might want to use specific versions of Kubernetes distributions and their components in your vCluster. Here's how you can specify custom images for EKS, k0s, k3s, and standard Kubernetes (k8s).

## EKS Custom Images

For an EKS-based vCluster, specify the custom images for the Kubernetes API server, controller manager, etcd, and CoreDNS in your Helm values file:

```shell
# eks-values.yaml
api:
image: public.ecr.aws/eks-distro/kubernetes/kube-apiserver:v1.28.2-eks-1-28-6
controller:
image: public.ecr.aws/eks-distro/kubernetes/kube-controller-manager:v1.28.2-eks-1-28-6
etcd:
image: public.ecr.aws/eks-distro/etcd-io/etcd:v3.5.9-eks-1-28-6
coredns:
image: public.ecr.aws/eks-distro/coredns/coredns:v1.10.1-eks-1-28-6
```

```shell
export CHART_NAME=vcluster-eks
```

### k0s Custom Images
For a k0s-based vCluster, specify the custom vCluster image in your Helm values file:

```shell
# k0s-values.yaml
vcluster:
image: k0sproject/k0s:v1.29.1-k0s.0
export CHART_VERSION=0.20.0
```

```shell
export CHART_NAME=vcluster-k0s
```

### k3s Custom Images
For a k3s-based vCluster, specify the custom vCluster image in your Helm values file:

```shell
# k3s-values.yaml
vcluster:
image: rancher/k3s:v1.29.0-k3s1
```
## Specifying custom values for virtual clusters

```shell
export CHART_NAME=vcluster
```
Depending on your needs, you might want to let CAPVC create a virtual cluster accordingly by e.g. settings those values in a corresponding file that is fed to the `VCLUSTER_YAML` environment variable.

### Standard Kubernetes (k8s) Custom Images
For a standard Kubernetes-based vCluster, specify the custom images for the Kubernetes API server, scheduler, controller manager, and etcd in your Helm values file:
Example:

```shell
# k8s-values.yaml
api:
image: registry.k8s.io/kube-apiserver:v1.29.0
scheduler:
image: registry.k8s.io/kube-scheduler:v1.29.0
controller:
image: registry.k8s.io/kube-controller-manager:v1.29.0
etcd:
image: registry.k8s.io/etcd:3.5.10-0
```
cat > /tmp/values.yaml <<EOF
controlPlane:
distro:
k8s:
enabled: true
backingStore:
etcd:
deploy:
enabled: true
EOF

```shell
export CHART_NAME=vcluster-k8s
export VCLUSTER_YAML=$(cat /tmp/values.yaml | awk '{printf "%s\\n", $0}')
```

## Applying Custom Image Configurations

After setting up the respective values files, use the HELM_VALUES environment variable to pass these values when creating the vCluster instance:

```shell
export CLUSTER_NAME=test
export CLUSTER_NAMESPACE=test
export CHART_VERSION=0.19.5
export HELM_VALUES=$(cat ./[distribution]-values.yaml | sed -z 's/\n/\\n/g')
kubectl create namespace ${CLUSTER_NAMESPACE}
cat templates/cluster-template.yaml | ./bin/envsubst | kubectl apply -n ${CLUSTER_NAMESPACE} -f -
```
Replace [distribution] with eks, k0s, k3s, or k8s as per your requirement.
For all possible values please see the [official docs](https://www.vcluster.com/docs/vcluster/configure/vcluster-yaml/).
12 changes: 12 additions & 0 deletions config/default/manager_image_patch.yaml-e
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
# Change the value of image field below to your controller image URL
- image: docker.io/loftsh/cluster-api-provider-vcluster:main
name: manager
11 changes: 11 additions & 0 deletions config/default/manager_pull_policy_patch.yaml-e
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller-manager
namespace: system
spec:
template:
spec:
containers:
- name: manager
imagePullPolicy: Always
12 changes: 12 additions & 0 deletions config/rbac/provider_role_binding.yaml-e
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: manager-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ${CLUSTER_ROLE:=cluster-admin}
subjects:
- kind: ServiceAccount
name: controller-manager
namespace: system
8 changes: 3 additions & 5 deletions docs/quick-start.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ In this guide we will deploy Nginx in a kind cluster and verify connectivity wit

## Prerequisites
Ensure that you have the following installed:
- [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl) v1.2 or greater
- [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl) v1.7.4 or greater
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [kind](https://kind.sigs.k8s.io/)
- [vcluster CLI](https://www.vcluster.com/docs/getting-started/setup)
Expand All @@ -22,7 +22,7 @@ Using `kind` we will create a management cluster for Cluster API to use.
kind create cluster

Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.23.4) 🖼
✓ Ensuring node image (kindest/node:v1.30.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
Expand Down Expand Up @@ -62,13 +62,11 @@ Next we will create our virtual cluster within our `kind` cluster.
```shell
export CLUSTER_NAME=kind
export CLUSTER_NAMESPACE=vcluster
export KUBERNETES_VERSION=1.23.4
export HELM_VALUES="service:\n type: NodePort"
export VCLUSTER_YAML=""

kubectl create namespace ${CLUSTER_NAMESPACE}
clusterctl generate cluster ${CLUSTER_NAME} \
--infrastructure vcluster \
--kubernetes-version ${KUBERNETES_VERSION} \
--target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -
```

Expand Down
2 changes: 1 addition & 1 deletion templates/cluster-template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ metadata:
name: ${CLUSTER_NAME}
spec:
helmRelease:
values: "${HELM_VALUES:=}"
values: "${VCLUSTER_YAML:=}"
chart:
name: ${CHART_NAME:=vcluster}
repo: ${CHART_REPO:=https://charts.loft.sh}
Expand Down