diff --git a/content/_redirects b/content/_redirects index 0874b0776c..8eefb4c46a 100644 --- a/content/_redirects +++ b/content/_redirects @@ -91,4 +91,7 @@ docs/started/requirements/ /docs/started/getting-started/ /docs/other-guides/job-scheduling/ /docs/use-cases/job-scheduling/ /docs/other-guides/upgrade/ /docs/upgrading/upgrade/ -/docs/use-cases/kubeflow-on-multinode-cluster/ /docs/other-guides/kubeflow-on-multinode-cluster/ \ No newline at end of file +/docs/use-cases/kubeflow-on-multinode-cluster/ /docs/other-guides/kubeflow-on-multinode-cluster/ + +# Remove Kubeflow installation on existing EKS cluster +/docs/aws/deploy/existing-cluster/ /docs/aws/deploy/install-kubeflow/ \ No newline at end of file diff --git a/content/docs/aws/deploy/existing-cluster.md b/content/docs/aws/deploy/existing-cluster.md deleted file mode 100644 index 3342ea7424..0000000000 --- a/content/docs/aws/deploy/existing-cluster.md +++ /dev/null @@ -1,63 +0,0 @@ -+++ -title = "Initial cluster setup for existing cluster" -weight = 6 -+++ - -## Before you start - -This is one step of [installing Kubeflow](/docs/aws/deploy/install-kubeflow), please make sure you have completed the prerequisite steps there before proceeding. - -### Deploy Kubeflow on existing Amazon EKS Cluster - -If you would like to deploy Kubeflow on existing Amazon EKS cluster, the only difference in setup is when you initialize the platform setup. Since you manage your own cluster resources, you need to provide `AWS_CLUSTER_NAME`, `cluster region` and `worker roles`. - -1. Download `kfctl` binary and config file - - ```shell - # Add kfctl to PATH, to make the kfctl binary easier to use. - tar -xvf kfctl__.tar.gz - export PATH=$PATH:"" - - # Download config files - export CONFIG="/tmp/kfctl_aws.yaml" - wget https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_aws.yaml -O ${CONFIG} - ``` - - -1. Retrieve the Amazon EKS cluster name, AWS Region, and IAM role name for your worker nodes. Set these values in the manifest file. - - ```shell - export AWS_CLUSTER_NAME= - export KFAPP=${AWS_CLUSTER_NAME} - ``` - - > Note: To get your Amazon EKS worker node IAM role name, you can check IAM setting by running the following commands. This command assumes that you used `eksctl` to create your cluster. If you use other provisioning tools to create your worker node groups, please find the role that is associated with your worker nodes in the Amazon EC2 console. - - ```shell - aws iam list-roles \ - | jq -r ".Roles[] \ - | select(.RoleName \ - | startswith(\"eksctl-$AWS_CLUSTER_NAME\") and contains(\"NodeInstanceRole\")) \ - .RoleName" - - eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6 - ``` - - Change cluster region and worker roles names in your `kfctl_aws.yaml` - ```yaml - region: us-west-2 - roles: - - eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6 - ``` - > If you have multiple node groups, you will see corresponding number of node group roles. In that case, please provide the role names as an array. - -1. Install Kubeflow - - ```shell - kfctl init ${KFAPP} --config=${CONFIG} -V - cd ${KFAPP} - kfctl generate all -v - kfctl apply all -v - ``` - -All rest steps are exact same for both install kubeflow on new cluster and existing cluster. Please come back to [Installing Kubeflow](/docs/aws/deploy/install-kubeflow) to finish post installation step. \ No newline at end of file diff --git a/content/docs/aws/deploy/install-kubeflow.md b/content/docs/aws/deploy/install-kubeflow.md index f1e622300a..53566cd702 100644 --- a/content/docs/aws/deploy/install-kubeflow.md +++ b/content/docs/aws/deploy/install-kubeflow.md @@ -17,50 +17,64 @@ deploy Kubeflow on Amazon Web Services (AWS). * Enter your preferred AWS Region and default output options. * Install [eksctl](https://github.com/weaveworks/eksctl) (version 0.1.31 or newer) and the [aws-iam-authenticator](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html). -You do not need to have an existing Amazon Elastic Container Service for Kubernetes (Amazon EKS) cluster. The deployment process will create a cluster for you. +## EKS cluster +There're many ways to provision EKS cluster, using AWS EKS CLI, CloudFormation or Terraform, AWS CDK or eksctl. +Here, we highly recommend you to create an EKS cluster using [eksctl](https://github.com/weaveworks/eksctl). + +You are required to have an existing Amazon Elastic Container Service for Kubernetes (Amazon EKS) cluster before moving the next step. The installation tool uses the `eksctl` command and doesn't support the `--profile` option in that command. If you need to switch role, use the `aws sts assume-role` commands. See the AWS guide to [using temporary security credentials to request access to AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html). ## Kubeflow installation +In order to deploy Kubeflow on your existing Amazon EKS cluster, you need to provide `AWS_CLUSTER_NAME`, `cluster region` and `worker roles`. + 1. Download the latest `kfctl` golang binary from [Kubeflow release page](https://github.com/kubeflow/kubeflow/releases) and unpack it. - ``` + ```shell # Add kfctl to PATH, to make the kfctl binary easier to use. tar -xvf kfctl__.tar.gz export PATH=$PATH:"" - ``` - -1. Download manifest and config setups - ```shell - # Download config files + # Download config file export CONFIG="/tmp/kfctl_aws.yaml" wget https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_aws.yaml -O ${CONFIG} ``` * `kfctl_aws.yaml` is one of setup manifests, please check [kfctl_aws_cognito.yaml](https://github.com/kubeflow/kubeflow/blob/master/bootstrap/config/kfctl_aws_cognito.yaml) for the template to enable authentication. - - If you plan to use `kfctl` to create a new eks cluster, please remove follow lines in the manifest file. - - If you want to install on existing EKS cluster, please change roles to your worker node group roles. See [existing cluster](/docs/aws/deploy/existing-cluster) for details. +1. Customize your config file. Retrieve the Amazon EKS cluster name, AWS Region, and IAM role name for your worker nodes. - ```shell - roles: - - eksctl-kubeflow-aws-nodegroup-ng-a2-NodeInstanceRole-xxxxxxx + ```shell + export AWS_CLUSTER_NAME= + export KFAPP=${AWS_CLUSTER_NAME} ``` -1. Run the following commands to set up your environment and initialize the cluster. + > Note: To get your Amazon EKS worker node IAM role name, you can check IAM setting by running the following commands. This command assumes that you used `eksctl` to create your cluster. If you use other provisioning tools to create your worker node groups, please find the role that is associated with your worker nodes in the Amazon EC2 console. - Since there're many ways to create your cluster, we highly recommend you to get our own eks cluster ready. + ```shell + aws iam list-roles \ + | jq -r ".Roles[] \ + | select(.RoleName \ + | startswith(\"eksctl-$AWS_CLUSTER_NAME\") and contains(\"NodeInstanceRole\")) \ + .RoleName" - Note: If you would like to install Kubeflow on your existing EKS cluster, - please skip this step and follow the setup instructions for an [existing cluster](/docs/aws/deploy/existing-cluster) instead. + eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6 + ``` + Change cluster region and worker roles names in your `kfctl_aws.yaml` + ```yaml + region: us-west-2 + roles: + - eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6 ``` - export KFAPP=kfaws + > If you have multiple node groups, you will see corresponding number of node group roles. In that case, please provide the role names as an array. +1. Run the following commands to set up your environment and initialize the cluster. + + ```shell kfctl init ${KFAPP} --config=${CONFIG} -V cd ${KFAPP} @@ -71,7 +85,6 @@ If you need to switch role, use the `aws sts assume-role` commands. See the AWS * KFAPP - Use a relative directory name here rather than absolute path, such as `kfapp`. It will be used as eks cluster name. * CONFIG - Path to the configuration file - *Important!!!* By default, these scripts create an AWS Application Load Balancer for Kubeflow that is open to public. This is good for development testing and for short term use, but we do not recommend that you use this configuration for production workloads. To secure your installation, Follow the [instructions](/docs/aws/authentication) to add authentication. diff --git a/content/docs/aws/deploy/uninstall-kubeflow.md b/content/docs/aws/deploy/uninstall-kubeflow.md index b77fd5bba5..0331d77e6c 100644 --- a/content/docs/aws/deploy/uninstall-kubeflow.md +++ b/content/docs/aws/deploy/uninstall-kubeflow.md @@ -5,11 +5,11 @@ weight = 10 +++ -## Uninstall Kubeflow and delete your Amazon EKS cluster. +## Uninstall Kubeflow ``` cd ${KFAPP} kfctl delete all -V ``` -> Note: If you installed Kubeflow on an existing Amazon EKS cluster, these scripts won't tear down your cluster in this step. In this case, you must manually delete your cluster. \ No newline at end of file +> Note: If you installed Kubeflow on an existing Amazon EKS cluster, these scripts won't tear down your cluster in this step. If you want to shutdown EKS cluster, you must manually delete it by yourself. \ No newline at end of file