diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md b/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md index 6ebe19127b86e..6e16e6670b564 100755 --- a/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md @@ -27,7 +27,7 @@ and behind the scene both use the same code. ## kubeadm alpha phase preflight {#cmd-phase-preflight} -You can execute preflight checks both for he tmaster node, like in `kubeadm init`, or for the worker node +You can execute preflight checks both for the master node, like in `kubeadm init`, or for the worker node like in `kubeadm join`. {% capture preflight_master %} @@ -46,7 +46,7 @@ like in `kubeadm join`. ## kubeadm alpha phase certs {#cmd-phase-certs} -You can create all required certificates with the `all` sub command or selectively create certificates. +You can create all required certificates with the `all` subcommand or selectively create certificates. {% capture certs_all %} {% include_relative generated/kubeadm_alpha_phase_certs_all.md %} @@ -84,7 +84,8 @@ You can create all required certificates with the `all` sub command or selective ## kubeadm alpha phase kubeconfig {#cmd-phase-kubeconfig} -You can create all required kubeconfig files with the `all` sub command, or selectively create the files. Additionally, the `user` sub command supports the creation of kubeconfig files for additional users. +You can create all required kubeconfig files with the `all` subcommand, or selectively create the files. +Additionally, the `user` subcommand supports the creation of kubeconfig files for additional users. {% capture kubeconfig_all %} {% include_relative generated/kubeadm_alpha_phase_kubeconfig_all.md %} @@ -118,7 +119,7 @@ You can create all required kubeconfig files with the `all` sub command, or sele ## kubeadm alpha phase controlplane {#cmd-phase-controlplane} -You can create all required static pod files for control plane components with the `all` sub command, +You can create all required static Pod files for the control plane components with the `all` subcommand, or selectively create the files. {% capture controlplane_all %} @@ -145,7 +146,7 @@ or selectively create the files. ## kubeadm alpha phase etcd {#cmd-phase-etcd} -Use the following command to create a self-hosted, local etcd instance based on a static pod file. +Use the following command to create a self-hosted, local etcd instance based on a static Pod file. {% capture etcd-local %} {% include_relative generated/kubeadm_alpha_phase_etcd_local.md %} @@ -159,7 +160,7 @@ Use the following command to create a self-hosted, local etcd instance based on ## kubeadm alpha phase mark-master {#cmd-phase-mark-master} -Use the following command to label the node with the `node-role.kubernetes.io/master` taint. +Use the following command to label and taint the node with the `node-role.kubernetes.io/master=""` key-value pair. {% capture mark-master %} {% include_relative generated/kubeadm_alpha_phase_mark-master.md %} @@ -174,7 +175,7 @@ Use the following command to label the node with the `node-role.kubernetes.io/ma ## kubeadm alpha phase bootstrap-token {#cmd-phase-bootstrap-token} Use the following actions to fully configure bootstrap tokens. -You can fully configure bootstrap tokens with the `all` sub command, +You can fully configure bootstrap tokens with the `all` subcommand, or selectively configure single elements. {% capture bootstrap-token_all %} @@ -205,8 +206,8 @@ or selectively configure single elements. ## kubeadm alpha phase upload-config {#cmd-phase-upload-config} -You can use this command to upload the configuration of your cluster. Alternatively, you -can use [kubeadm config](kubeadm-config.md). +You can use this command to upload the kubeadm configuration to your cluster. +Alternatively, you can use [kubeadm config](kubeadm-config.md). {% capture upload-config %} {% include_relative generated/kubeadm_alpha_phase_upload-config.md %} @@ -220,7 +221,7 @@ can use [kubeadm config](kubeadm-config.md). ## kubeadm alpha phase addon {#cmd-phase-addon} -You can install all the available addons with the `all` sub command, or +You can install all the available addons with the `all` subcommand, or install them selectively. {% capture addon-all %} @@ -258,4 +259,5 @@ install them selectively. ## What's next * [kubeadm init](kubeadm-init.md) to bootstrap a Kubernetes master node +* [kubeadm join](kubeadm-join.md) to connect a node to the cluster * [kubeadm reset](kubeadm-reset.md) to revert any changes made to this host by `kubeadm init` or `kubeadm join` diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-config.md b/docs/reference/setup-tools/kubeadm/kubeadm-config.md index f1d0e09d4671d..5a9ed78ebd8e6 100644 --- a/docs/reference/setup-tools/kubeadm/kubeadm-config.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-config.md @@ -7,9 +7,12 @@ title: kubeadm config --- {% capture overview %} Beginning with v1.8.0, kubeadm uploads the configuration of your cluster to a ConfigMap called -`kubeadm-config` in the `kube-system` namespace, and later reads the ConfigMap when upgrading. This enables correct configuration of system components, and provides a seamless user experience. +`kubeadm-config` in the `kube-system` namespace, and later reads the ConfigMap when upgrading. +This enables correct configuration of system components, and provides a seamless user experience. -You can use `kubeadm config view` to view the ConfigMap. If you initialized your cluster using kubeadm v1.7.x or lower, you can use `kubeadm config upload` to create a ConfigMap before you execute `kubeadm upgrade`. +You can execute `kubeadm config view` to view the ConfigMap. If you initialized your cluster using +kubeadm v1.7.x or lower, you must use `kubeadm config upload` to create the ConfigMap before you +may use `kubeadm upgrade`. {% endcapture %} diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-init.md b/docs/reference/setup-tools/kubeadm/kubeadm-init.md index 30e59549e9e64..96419b5ee26b5 100755 --- a/docs/reference/setup-tools/kubeadm/kubeadm-init.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-init.md @@ -20,15 +20,9 @@ following steps: considered errors and will exit kubeadm until the problem is corrected or the user specifies `--skip-preflight-checks`. -1. Generates the token that additional nodes can use to register - themselves with the master in the future. Optionally, the user can provide a - token via `--token`, as described in the - [kubeadm token](kubeadm-token.md) docs. - -1. Generates a self-signed CA to provision identities for each component - (including nodes) in the cluster. It also generates client certificates to - be used by various components. If the user has provided their own CA by - dropping it in the cert directory configured via `--cert-dir` +1. Generates a self-signed CA (or using an existing one if provided) to set up + identities for each component in the cluster. If the user has provided their + own CA cert and/or key by dropping it in the cert directory configured via `--cert-dir` (`/etc/kubernetes/pki` by default) this step is skipped as described in the [Using custom certificates](#custom-certificates) document. @@ -44,11 +38,16 @@ following steps: Static Pod manifests are written to `/etc/kubernetes/manifests`; the kubelet watches this directory for Pods to create on startup. - Once control plane Pods are up and running, the kubeadm init sequence can continue. + Once control plane Pods are up and running, the `kubeadm init` sequence can continue. 1. Apply labels and taints to the master node so that no additional workloads will run there. +1. Generates the token that additional nodes can use to register + themselves with the master in the future. Optionally, the user can provide a + token via `--token`, as described in the + [kubeadm token](kubeadm-token.md) docs. + 1. Makes all the necessary configurations for allowing node joining with the [Bootstrap Tokens](/docs/admin/bootstrap-tokens/) and [TLS Bootstrap](/docs/admin/kubelet-tls-bootstrapping/) @@ -57,9 +56,9 @@ following steps: - Write a ConfigMap for making available all the information required for joining, and set up related RBAC access rules. - - Ensure access to the CSR signing API for bootstrap tokens. + - Let Bootstrap Tokens access the CSR signing API. - - Configure auto approval for new CSR requests. + - Configure auto-approval for new CSR requests. See [kubeadm join](kubeadm-join.md) for additional info. @@ -145,7 +144,7 @@ will need to look like this: apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration apiServerExtraArgs: - feature-gates: APIResponseCompression=true + feature-gates: APIResponseCompression=true ``` To customize the scheduler or controller-manager, use `schedulerExtraArgs` and `controllerManagerExtraArgs` respectively. @@ -159,7 +158,7 @@ More information on custom arguments can be found here: By default, kubeadm pulls images from `gcr.io/google_containers`, unless the requested Kubernetes version is a CI version. In this case, -`gcr.io/kubernetes-ci-image` is used. +`gcr.io/kubernetes-ci-images` is used. You can override this behavior by using [kubeadm with a configuration file](#config-file). Allowed customization are: @@ -185,11 +184,16 @@ use case. This means you can, for example, copy an existing CA into `/etc/kubern and `/etc/kubernetes/pki/ca.key`, and kubeadm will use this CA for signing the rest of the certs. +#### External CA mode {#external-ca-mode} + It is also possible to provide just the `ca.crt` file and not the `ca.key` file (this is only available for the root CA file, not other cert pairs). If all other certificates and kubeconfig files are in place, kubeadm recognizes -this condition and activates the "ExternalCA" mode, which also -implies the CSR signer controller in the controller manager won't be started. +this condition and activates the "External CA" mode. kubeadm will proceed without the +CA key on disk. + +Instead, run the controller-manager standalone with `--controllers=csrsigner` and +point to the CA certificate and key. ### Managing the kubeadm drop-in file for the kubelet {#kubelet-drop-in} @@ -241,7 +245,7 @@ Here's a breakdown of what/why: `https://{node-ip}:10250/stats/`. If you want to enable cAdvisor to listen on a wide-open port, run: - ``` + ```bash sed -e "/cadvisor-port=0/d" -i /etc/systemd/system/kubelet.service.d/10-kubeadm.conf systemctl daemon-reload systemctl restart kubelet @@ -252,12 +256,11 @@ Here's a breakdown of what/why: ### Use kubeadm with other CRI runtimes -Since the [Kubernetes 1.6 release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#node-components-1), -Kubernetes has started using CRI as the default container runtime. -The container runtime used by kubeadm is Docker, which is enabled through the built-in -`dockershim` in `kubelet` mechanisms. +Since v1.6.0, Kubernetes has enabled the use of CRI, Container Runtime Interface, by default. +The container runtime used by default is Docker, which is enabled through the built-in +`dockershim` CRI implementation inside of the `kubelet`. -Other CRI-based runtimes include:: +Other CRI-based runtimes include: - [cri-o](https://github.com/kubernetes-incubator/cri-o) - [frakti](https://github.com/kubernetes/frakti) @@ -273,15 +276,18 @@ these two additional steps: `RUNTIME_ENDPOINT` to your own value like `/var/run/{your_runtime}.sock`: ```shell - cat > /etc/systemd/system/kubelet.service.d/20-cri.conf < /etc/systemd/system/kubelet.service.d/20-cri.conf < * [kubeadm reset](kubeadm-reset.md) to revert any changes made to this host by `kubeadm init` or `kubeadm join` {% endcapture %} -{% include templates/concept.md %} \ No newline at end of file +{% include templates/concept.md %} diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-reset.md b/docs/reference/setup-tools/kubeadm/kubeadm-reset.md index aabc331139045..1e9f2e862d46a 100644 --- a/docs/reference/setup-tools/kubeadm/kubeadm-reset.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-reset.md @@ -12,15 +12,16 @@ title: kubeadm reset {% include_relative generated/kubeadm_reset.md %} ### External etcd clean up + `kubeadm reset` will not delete any etcd data if external etcd is used. This means that if you run `kubeadm init` again using the same etcd endpoints, you will see state from previous clusters. To wipe etcd data it is recommended you use a client like etcdctl, such as: -``` +```bash etcdctl del "" --prefix ``` -See [etcd documentation](https://github.com/coreos/etcd/tree/master/etcdctl) for more information. +See the [etcd documentation](https://github.com/coreos/etcd/tree/master/etcdctl) for more information. {% endcapture %} {% capture whatsnext %} diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-token.md b/docs/reference/setup-tools/kubeadm/kubeadm-token.md index babd2c2b6123e..24451821535bf 100644 --- a/docs/reference/setup-tools/kubeadm/kubeadm-token.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-token.md @@ -6,11 +6,13 @@ approvers: title: kubeadm token --- {% capture overview %} + Bootstrap tokens are used for establishing bidirectional trust between a node joining the cluster and a the master node, as described in [authenticating with bootstrap tokens](/docs/admin/bootstrap-tokens/). -`kubeadm-init` creates an initial token with 24h TTL. The following commands allow you to manage +`kubeadm init` creates an initial token with a 24-hour TTL. The following commands allow you to manage such a token and also to create and manage new ones. + {% endcapture %} {% capture body %} @@ -31,4 +33,4 @@ such a token and also to create and manage new ones. * [kubeadm join](kubeadm-join.md) to bootstrap a Kubernetes worker node and join it to the cluster {% endcapture %} -{% include templates/concept.md %} \ No newline at end of file +{% include templates/concept.md %} diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-upgrade.md b/docs/reference/setup-tools/kubeadm/kubeadm-upgrade.md index 009bc8d71b58a..323b5568743b1 100755 --- a/docs/reference/setup-tools/kubeadm/kubeadm-upgrade.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-upgrade.md @@ -6,6 +6,19 @@ approvers: title: kubeadm upgrade --- {% capture overview %} + +`kubeadm upgrade` is a user-friendly command that wraps complex upgrading logic behind one command, with support +for both planning an upgrade and actually performing it. `kubeadm upgrade` can also be used for downgrading +cluster if necessary. + +Every upgrade process might be a bit different, so we've documented each minor upgrade process individually. +Please check these documents out for more detailed how-to-upgrade guidance: + + * [1.6 to 1.7 upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/) + * [1.7.x to 1.7.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/) + * [1.7 to 1.8 upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/) + * [1.8.x to 1.8.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/) + {% endcapture %} {% capture body %} @@ -21,4 +34,4 @@ title: kubeadm upgrade * [kubeadm config](kubeadm-config.md) if you initialized your cluster using kubeadm v1.7.x or lower, to configure your cluster for `kubeadm upgrade` {% endcapture %} -{% include templates/concept.md %} \ No newline at end of file +{% include templates/concept.md %} diff --git a/docs/reference/setup-tools/kubeadm/kubeadm-version.md b/docs/reference/setup-tools/kubeadm/kubeadm-version.md index 1395ee15ace9d..059f6892afe5d 100644 --- a/docs/reference/setup-tools/kubeadm/kubeadm-version.md +++ b/docs/reference/setup-tools/kubeadm/kubeadm-version.md @@ -12,4 +12,4 @@ title: kubeadm version {% include_relative generated/kubeadm_version.md %} {% endcapture %} -{% include templates/concept.md %} \ No newline at end of file +{% include templates/concept.md %} diff --git a/docs/setup/independent/create-cluster-kubeadm.md b/docs/setup/independent/create-cluster-kubeadm.md index 4b33f50f4c521..d35d9549b4980 100644 --- a/docs/setup/independent/create-cluster-kubeadm.md +++ b/docs/setup/independent/create-cluster-kubeadm.md @@ -9,59 +9,73 @@ title: Using kubeadm to Create a Cluster {% capture overview %} -This quickstart shows you how to easily install a Kubernetes cluster on machines -running Ubuntu 16.04+, CentOS 7 or HypriotOS v1.0.1+. The installation uses a -tool called _kubeadm_ which is part of Kubernetes. As of v1.6, kubeadm aims to -create a secure cluster out of the box via mechanisms such as RBAC. - -This process works with local VMs, physical servers and/or cloud servers. It is -simple enough that you can easily integrate its use into your own automation -(Terraform, Chef, Puppet, etc). - -See the full [kubeadm reference](/docs/admin/kubeadm/) for information on all -kubeadm command-line flags and for advice on automating kubeadm itself. - -kubeadm assumes you have a set of machines (virtual or real) that are up and -running. It is designed to be part of a large provisioning system - or just for -easy manual provisioning. kubeadm is a great choice where you have your own -infrastructure (e.g. bare metal), or where you have an existing orchestration -system (e.g. Puppet) that you have to integrate with. - -If you are not constrained, there are other higher-level tools built to give you -complete clusters: - -* On GCE, [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) - gives you one-click Kubernetes clusters. -* On Microsoft Azure, [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes) - gives you managed Kubernetes clusters as a service. -* On AWS, [kops](https://github.com/kubernetes/kops) makes cluster installation - and management easy. kops supports building high availability clusters (a - feature that kubeadm is currently lacking but is building toward). +**kubeadm** is a toolkit that help you bootstrap a best-practice Kubernetes +cluster in an easy, reasonably secure and extensible way. It also supports +managing [Bootstrap Tokens](#TODO) for you and upgrading/downgrading clusters. + +kubeadm aims to set up a minimum viable cluster that pass the +[Kubernetes Conformance tests](#TODO), but installing other addons than +really necessary for a functional cluster is out of scope. + +It by design does not install a networking solution for you, which means you +have to install a third-party CNI-compliant networking solution yourself +using `kubectl apply`. + +kubeadm expects the user to bring a machine to execute on, the type doesn't +matter, can be a Linux laptop, virtual machine, physical/cloud server or +Raspberry Pi. This makes kubeadm well suited to integrate with provisioning +systems of different kinds (e.g. Terraform, Ansible, etc.). + +kubeadm is designed to be a good way for new users to start trying +Kubernetes out, possibly for the first time, an way for existing users to +test their application on and stich together a cluster easily and to be +a building block in a larger ecosystem and/or installer tool with a larger +scope. + +You can install _kubeadm_ very easily on operating systems that support +installing deb or rpm packages. The responsible SIG for kubeadm, +[SIG Cluster Lifecycle](#TODO), provides these packages pre-built for you, +but you may also on other OSes. + ### kubeadm Maturity -| Aspect | Maturity Level -|--------|--------------- -| Command line UX | beta -| Config file | alpha -| Self-hosting | alpha -| `kubeadm alpha` commands | alpha -| Implementation | beta - -The experience for the command line is currently in beta and we are trying hard -not to change command line flags and break that flow. Other parts of the -experience are still under active development. The implementation may change -slightly as the tool evolves to support even easier upgrades and high -availability (HA). Any commands under `kubeadm alpha` (not documented here) -are, of course, alpha. - -**Be sure to read the [limitations](#limitations)**. Specifically, configuring -cloud providers is difficult. +| Area | Maturity Level | +|-----------------|--------------- | +| Command line UX | beta | +| Implementation | beta | +| Config file API | alpha | +| Self-hosting | alpha | +| `kubeadm alpha` | alpha | + + +kubeadm's overall feature state is **Beta** and will soon be graduated to +**General Availability (GA)** during 2018. Some sub-features, like self-hosting +or the configuration file API are still under active development. The +implementation of creating the cluster may change slightly as the tool evolves, +but the overall implementation should be pretty stable. Any commands under +`kubeadm alpha` are by definition, supported on an alpha level. + + +### Support timeframes + +Kubernetes releases are generally supported for nine months, and during that +period a patch release may be issued from the release branch if a severe bug or +security issue is found. Here are the latest Kubernetes releases and the support +timeframe; which also applies to `kubeadm`. + +| Kubernetes version | Release date | End-of-life-month | +|--------------------|--------------|-------------------| +| v1.6.x | TODO | December 2017 | +| v1.7.x | TODO | March 2018 | +| v1.8.x | TODO | June 2018 | +| v1.9.x | TODO        | September 2018   | + {% endcapture %} {% capture prerequisites %} -1. One or more machines running Ubuntu 16.04+, CentOS 7 or HypriotOS v1.0.1+ +1. One or more machines running a deb/rpm-compatible OS, e.g. Ubuntu or CentOS 1. 2 GB or more of RAM per machine (any less will leave little room for your apps) 1. 2 CPUs or more on the master @@ -74,9 +88,8 @@ cloud providers is difficult. ## Objectives * Install a secure Kubernetes cluster on your machines -* Install a pod network on the cluster so that application components (pods) can +* Install a Pod network on the cluster so that your Pods can talk to each other -* Install a sample microservices application (a socks shop) on the cluster ## Instructions @@ -89,7 +102,8 @@ apt-get upgrade` or `yum update` to get the latest version of kubeadm. The kubelet is now restarting every few seconds, as it waits in a crashloop for -kubeadm to tell it what to do. +kubeadm to tell it what to do. This crashloop is expected and normal, please +proceed with the next step and the kubelet will start running normally. ### (2/4) Initializing your master @@ -100,15 +114,15 @@ communicates with). To initialize the master, pick one of the machines you previously installed kubeadm on, and run: -``` bash +```bash kubeadm init ``` **Notes:** -- Please refer to the [kubeadm reference doc](/docs/admin/kubeadm/) if you want to +- Please refer to the [kubeadm reference guide](/docs/reference/setup-tools/kubeadm/) if you want to read more about the flags `kubeadm init` provides. You can also specify a -[configuration file](/docs/admin/kubeadm/#sample-master-configuration) instead of using flags. +[configuration file](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file) instead of using flags. - You need to choose a Pod Network Plugin in the next step. Depending on what third-party provider you choose, you might have to set the `--pod-network-cidr` to something provider-specific. The tabs below will contain a notice about what flags @@ -117,9 +131,9 @@ on `kubeadm init` are required. to advertise the master's IP. If you want to use a different network interface, specify `--apiserver-advertise-address=` argument to `kubeadm init`. - If you would like to customise control plane components, you can do so by providing -extra args to each one, as documented [here](/docs/admin/kubeadm#custom-args). +extra args to each one, as documented [here](/docs/reference/setup-tools/kubeadm/kubeadm-init#custom-args). - `kubeadm init` will first run a series of prechecks to ensure that the machine -is ready to run Kubernetes. It will expose warnings and exit on errors. It +is ready to run Kubernetes. It will expose warnings and exit on errors. It will then download and install the cluster database and control plane components. This may take several minutes. - You can't run `kubeadm init` twice without tearing down the cluster in between @@ -180,14 +194,18 @@ as root: kubeadm join --token : --discovery-token-ca-cert-hash sha256: ``` + To make kubectl work for your non-root user, you might want to run these commands (which is also a part of the `kubeadm init` output): -``` + +```bash mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` -Alternatively, if you are the root user, you could run this: -``` + +Alternatively, if you are the `root` user, you could run this: + +```bash export KUBECONFIG=/etc/kubernetes/admin.conf ``` @@ -195,32 +213,31 @@ Make a record of the `kubeadm join` command that `kubeadm init` outputs. You will need this in a moment. The token is used for mutual authentication between the master and the joining -nodes. The token included here is secret, keep it safe — anyone with this -token can add authenticated nodes to your cluster. These tokens can be listed, -created and deleted with the `kubeadm token` command. See the [reference -guide](/docs/admin/kubeadm/#manage-tokens). +nodes. The token included here is secret, keep it safe as anyone with this +token can add authenticated nodes to your cluster. These tokens can be listed, +created and deleted with the `kubeadm token` command. See the +[reference guide](/docs/reference/setup-tools/kubeadm/kubeadm-token/) for more details. ### (3/4) Installing a pod network {#pod-network} -You **must** install a pod network add-on so that your pods can communicate with +You **MUST** install a pod network add-on so that your pods can communicate with each other. -**The network must be deployed before any applications. Also, kube-dns, a -helper service, will not start up before a network is installed. kubeadm only +**The network must be deployed before any applications. Also, kube-dns, an +internal helper service, will not start up before a network is installed. kubeadm only supports Container Network Interface (CNI) based networks (and does not support kubenet).** Several projects provide Kubernetes pod networks using CNI, some of which also -support [Network Policy](/docs/concepts/services-networking/networkpolicies/). See the [add-ons -page](/docs/concepts/cluster-administration/addons/) for a complete list of available network add-ons. +support [Network Policy](/docs/concepts/services-networking/networkpolicies/). See the +[add-ons page](/docs/concepts/cluster-administration/addons/) for a list of available +network add-ons. -**New for Kubernetes 1.6:** kubeadm 1.6 sets up a more secure cluster by -default. As such it uses RBAC to grant limited privileges to workloads running -on the cluster. This includes networking integrations. As such, ensure that -you are using a network system that has been updated to run with 1.6 and RBAC. +**Note:** kubeadm sets up a more secure cluster by default and enforces use of [RBAC](#TODO). +Please make sure that the network manifest of choice supports RBAC. You can install a pod network add-on with the following command: -``` bash +```bash kubectl apply -f ``` @@ -284,7 +301,7 @@ Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information please see [here](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements). -Kube-router relies on kube-controll-manager to allocate pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag. +Kube-router relies on kube-controller-manager to allocate pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag. Kube-router provides pod networking, network policy, and high-performing IP Virtual Server(IPVS)/Linux Virtual Server(LVS) based service proxy. @@ -343,7 +360,7 @@ By default, your cluster will not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, e.g. for a single-machine Kubernetes cluster for development, run: -``` bash +```bash kubectl taint nodes --all node-role.kubernetes.io/master- ``` @@ -367,9 +384,9 @@ The nodes are where your workloads (containers and pods, etc) run. To add new no * Become root (e.g. `sudo su -`) * Run the command that was output by `kubeadm init`. For example: - ``` bash - kubeadm join --token : --discovery-token-ca-cert-hash sha256: - ``` +``` bash +kubeadm join --token : --discovery-token-ca-cert-hash sha256: +``` The output should look something like: @@ -408,67 +425,31 @@ scp root@:/etc/kubernetes/admin.conf . kubectl --kubeconfig ./admin.conf get nodes ``` -**Note:** If you are using GCE, instances disable ssh access for root by default. -If that's the case you can log in to the machine, copy the file someplace that -can be accessed and then use -[`gcloud compute copy-files`](https://cloud.google.com/sdk/gcloud/reference/compute/copy-files). +**Note:** + - The example above assumes SSH access is enabled for root. If that is not the + case, you can copy the `admin.conf` file to be accessible by some other user + and `scp` using that other user instead. + - The `admin.conf` file gives the user _superuser_ privileges over the cluster. + This file should be used sparsingly. For normal users, it's recommended to + generate an unique credential to which you whitelist privileges. You can do + this with the `kubeadm alpha phase kubeconfig user --client-name ` + command. That command will print out a KubeConfig file to STDOUT which you + should save to a file and distribute to your user. After that, whitelist + privileges by using `kubectl create (cluster)rolebinding`. ### (Optional) Proxying API Server to localhost If you want to connect to the API Server from outside the cluster you can use `kubectl proxy`: -``` bash +```bash scp root@:/etc/kubernetes/admin.conf . kubectl --kubeconfig ./admin.conf proxy ``` You can now access the API Server locally at `http://localhost:8001/api/v1` -### (Optional) Installing a sample application - -Now it is time to take your new cluster for a test drive. Sock Shop is a sample -microservices application that shows how to run and connect a set of services on -Kubernetes. To learn more about the sample microservices app, see the [GitHub -README](https://github.com/microservices-demo/microservices-demo). - -Note that the Sock Shop demo only works on `amd64`. - -``` bash -kubectl create namespace sock-shop -kubectl apply -n sock-shop -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true" -``` - -You can then find out the port that the [NodePort feature of -services](/docs/concepts/services-networking/service/) allocated for the front-end service by -running: - -``` bash -kubectl -n sock-shop get svc front-end -``` - -Sample output: - -``` -NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE -front-end 10.110.250.153 80:30001/TCP 59s -``` - -It takes several minutes to download and start all the containers, watch the -output of `kubectl get pods -n sock-shop` to see when they're all up and -running. - -Then go to the IP address of your cluster's master node in your browser, and -specify the given port. So for example, `http://:`. In the -example above, this was `30001`, but it may be a different port for you. - -If there is a firewall, make sure it exposes this port to the internet before -you try to access it. - -To uninstall the socks shop, run `kubectl delete namespace sock-shop` on the -master. - -## Tear down +## Tear down {#tear-down} To undo what kubeadm did, you should first [drain the node](/docs/user-guide/kubectl/{{page.version}}/#drain) and make @@ -476,34 +457,24 @@ sure that the node is empty before shutting it down. Talking to the master with the appropriate credentials, run: -``` bash +```bash kubectl drain --delete-local-data --force --ignore-daemonsets kubectl delete node ``` Then, on the node being removed, reset all kubeadm installed state: -``` bash +```bash kubeadm reset ``` If you wish to start over simply run `kubeadm init` or `kubeadm join` with the appropriate arguments. -**Note**: `kubeadm reset` will not delete any etcd data if external etcd is used. -This means that if you run `kubeadm init` again using the same etcd endpoints, you -will see state from previous clusters. To wipe etcd data after reset, it is -recommended you use a client like `etcdctl`, such as: - -``` -etcdctl del "" --prefix -``` - -See -[their documentation](https://github.com/coreos/etcd/tree/master/etcdctl) for more -information. +More options and information about the +[`kubeadm reset command`](/docs/reference/setup-tools/kubeadm/kubeadm-reset/). -## Upgrading +## Upgrading a kubeadm cluster {#upgrades} Instructions for upgrading kubeadm clusters are available for: @@ -514,32 +485,31 @@ Instructions for upgrading kubeadm clusters are available for: * [1.8 to 1.9 upgrades/downgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/) * [1.9.x to 1.9.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/) -## Explore other add-ons +## Explore other add-ons {#other-addons} See the [list of add-ons](/docs/concepts/cluster-administration/addons/) to explore other add-ons, including tools for logging, monitoring, network policy, visualization & control of your Kubernetes cluster. -## What's next +## What's next {#whats-next} -* Learn about kubeadm's advanced usage on the [advanced reference - doc](/docs/admin/kubeadm/). +* Learn about kubeadm's advanced usage in the [kubeadm reference documentation](/docs/reference/setup-tools/kubeadm/) * Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/user-guide/kubectl-overview/). * Configure log rotation. You can use **logrotate** for that. When using Docker, you can specify log rotation options for Docker daemon, for example `--log-driver=json-file --log-opt=max-size=10m --log-opt=max-file=5`. See [Configure and troubleshoot the Docker daemon](https://docs.docker.com/engine/admin/) for more details. -## Feedback +## Feedback {#feedback} * kubeadm support Slack Channel: - [kubeadm](https://kubernetes.slack.com/messages/kubeadm/) + [#kubeadm](https://kubernetes.slack.com/messages/kubeadm/) * General SIG Cluster Lifecycle Development Slack Channel: - [sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/) -* Mailing List: + [#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/) +* SIG Cluster Lifecycle [SIG information](#TODO) +* SIG Cluster Lifecycle Mailing List: [kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle) -* [GitHub Issues in the kubeadm - repository](https://github.com/kubernetes/kubeadm/issues) +* [kubeadm Github issue tracker](https://github.com/kubernetes/kubeadm/issues) -## Version skew policy +## Version skew policy {#version-skew-policy} The kubeadm CLI tool of version vX.Y may deploy clusters with a control plane of version vX.Y or vX.(Y-1). kubeadm CLI vX.Y can also upgrade an existing kubeadm-created cluster of version vX.(Y-1). @@ -552,7 +522,7 @@ v1.8. Please also check our [installation guide](/docs/setup/independent/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl) for more information on the version skew between kubelets and the control plane. -## kubeadm is multi-platform {#multi-platform} +## kubeadm works on multiple platforms {#multi-platform} kubeadm deb/rpm packages and binaries are built for amd64, arm (32-bit), arm64, ppc64le, and s390x following the [multi-platform @@ -562,22 +532,22 @@ Only some of the network providers offer solutions for all platforms. Please con network providers above or the documentation from each provider to figure out whether the provider supports your chosen platform. -## Limitations +## Limitations {#limitations} Please note: kubeadm is a work in progress and these limitations will be addressed in due course. 1. The cluster created here has a single master, with a single etcd database - running on it. This means that if the master fails, your cluster loses its - configuration data and will need to be recreated from scratch. Adding HA - support (multiple etcd servers, multiple API servers, etc) to kubeadm is + running on it. This means that if the master fails, your cluster may lose + data and may need to be recreated from scratch. Adding HA support + (multiple etcd servers, multiple API servers, etc) to kubeadm is still a work-in-progress. - Workaround: regularly [back up - etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The etcd data - directory configured by kubeadm is at `/var/lib/etcd` on the master. + Workaround: regularly + [back up etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The + etcd data directory configured by kubeadm is at `/var/lib/etcd` on the master. -## Troubleshooting +## Troubleshooting {#troubleshooting} If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/). diff --git a/docs/setup/independent/install-kubeadm.md b/docs/setup/independent/install-kubeadm.md index b07895df7c89c..1e9294ea2948e 100644 --- a/docs/setup/independent/install-kubeadm.md +++ b/docs/setup/independent/install-kubeadm.md @@ -4,19 +4,27 @@ title: Installing kubeadm {% capture overview %} -This page shows how to use install kubeadm. +This page shows how to use install the `kubeadm` toolbox. +For information how to create a cluster with kubeadm once you have performed this installation process, +see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/) page. {% endcapture %} {% capture prerequisites %} -* One or more machines running Ubuntu 16.04+, Debian 9, CentOS 7, RHEL 7, Fedora 25/26 (best-effort) or HypriotOS v1.0.1+ +* One or more machines running one of: + - Ubuntu 16.04+ + - Debian 9 + - CentOS 7 + - RHEL 7 + - Fedora 25/26 (best-effort) + - HypriotOS v1.0.1+ * 2 GB or more of RAM per machine (any less will leave little room for your apps) * 2 CPUs or more * Full network connectivity between all machines in the cluster (public or private network is fine) * Unique hostname, MAC address, and product_uuid for every node * Certain ports are open on your machines. See the section below for more details -* Swap disabled. You must disable swap in order for the kubelet to work properly. +* Swap disabled. You **MUST** disable swap in order for the kubelet to work properly. {% endcapture %} @@ -29,39 +37,42 @@ This page shows how to use install kubeadm. It is very likely that hardware devices will have unique addresses, although some virtual machines may have identical values. Kubernetes uses these values to uniquely identify the nodes in the cluster. -If these values are not unique to each node, the installation processes -[can fail](https://github.com/kubernetes/kubeadm/issues/31). +If these values are not unique to each node, the installation process +[may fail](https://github.com/kubernetes/kubeadm/issues/31). ## Check network adapters -If you have more than one network adapter, and your Kubernetes components are not reachable on the default route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter. +If you have more than one network adapter, and your Kubernetes components are not reachable on the default +route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter. ## Check required ports ### Master node(s) -| Protocol | Direction | Port Range | Purpose | -|----------|-----------|------------|---------------------------------| -| TCP | Inbound | 6443* | Kubernetes API server | -| TCP | Inbound | 2379-2380 | etcd server client API | -| TCP | Inbound | 10250 | Kubelet API | -| TCP | Inbound | 10251 | kube-scheduler | -| TCP | Inbound | 10252 | kube-controller-manager | -| TCP | Inbound | 10255 | Read-only Kubelet API (Heapster)| +| Protocol | Direction | Port Range | Purpose | +|----------|-----------|------------|-------------------------| +| TCP | Inbound | 6443* | Kubernetes API server | +| TCP | Inbound | 2379-2380 | etcd server client API | +| TCP | Inbound | 10250 | Kubelet API | +| TCP | Inbound | 10251 | kube-scheduler | +| TCP | Inbound | 10252 | kube-controller-manager | +| TCP | Inbound | 10255 | Read-only Kubelet API | ### Worker node(s) -| Protocol | Direction | Port Range | Purpose | -|----------|-----------|-------------|---------------------------------| -| TCP | Inbound | 10250 | Kubelet API | -| TCP | Inbound | 10255 | Read-only Kubelet API (Heapster)| -| TCP | Inbound | 30000-32767 | Default port range for [NodePort Services](/docs/concepts/services-networking/service/). Typically, these ports would need to be exposed to external load-balancers, or other external consumers of the application itself. | +| Protocol | Direction | Port Range | Purpose | +|----------|-----------|-------------|-----------------------| +| TCP | Inbound | 10250 | Kubelet API | +| TCP | Inbound | 10255 | Read-only Kubelet API | +| TCP | Inbound | 30000-32767 | NodePort Services** | + +** Default port range for [NodePort Services](/docs/concepts/services-networking/service/). Any port numbers marked with * are overridable, so you will need to ensure any custom ports you provide are also open. Although etcd ports are included in master nodes, you can also host your own -etcd cluster externally on custom ports. +etcd cluster externally or on custom ports. The pod network plugin you use (see below) may also require certain ports to be open. Since this differs with each pod network plugin, please see the @@ -86,7 +97,7 @@ apt-get update apt-get install -y docker.io ``` -or install Docker CE 17.09 from Docker's repositories for Ubuntu or Debian: +or install Docker CE 17.03 from Docker's repositories for Ubuntu or Debian: ```bash apt-get update @@ -97,10 +108,10 @@ apt-get install -y \ software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - add-apt-repository \ - "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ + "deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ $(lsb_release -cs) \ stable" -apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.09 | head -1 | awk '{print $3}') +apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print $3}') ``` {% endcapture %} @@ -155,7 +166,8 @@ need to ensure they match the version of the Kubernetes control panel you want kubeadm to install for you. If you do not, there is a risk of a version skew occurring that can lead to unexpected, buggy behaviour. However, _one_ minor version skew between the kubelet and the control plane is supported, but the kubelet version may never exceed the API -server version. For example, kubelets running 1.7.0 should be fully compatible with a 1.8.0 API server. +server version. For example, kubelets running 1.7.0 should be fully compatible with a 1.8.0 API server, +but not vice versa. For more information on version skews, please read our [version skew policy](/docs/setup/independent/create-cluster-kubeadm/#version-skew-policy). @@ -193,8 +205,10 @@ systemctl enable kubelet && systemctl start kubelet **Note:** - - Disabling SELinux by running `setenforce 0` is required to allow containers to access the host filesystem, which is required by pod networks for example. You have to do this until SELinux support is improved in the kubelet. - - Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure `net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g. + - Disabling SELinux by running `setenforce 0` is required to allow containers to access the host filesystem, which is required by pod networks for example. + You have to do this until SELinux support is improved in the kubelet. + - Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure + `net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g. ``` bash cat < /etc/sysctl.d/k8s.conf @@ -221,8 +235,7 @@ If you are running into difficulties with kubeadm, please consult our [troublesh {% capture whatsnext %} -* [Using kubeadm to Create a - Cluster](/docs/setup/independent/create-cluster-kubeadm/) +* [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/) {% endcapture %} diff --git a/docs/setup/independent/troubleshooting-kubeadm.md b/docs/setup/independent/troubleshooting-kubeadm.md index 884514b32048c..510cd88593898 100644 --- a/docs/setup/independent/troubleshooting-kubeadm.md +++ b/docs/setup/independent/troubleshooting-kubeadm.md @@ -23,7 +23,7 @@ If your cluster is in an error state, you may have trouble in the configuration {% endcapture %} -#### `ebtables` or `ethtool` not found during installation +#### `ebtables` or executable not found during installation If you see the following warnings while running `kubeadm init` @@ -32,7 +32,7 @@ If you see the following warnings while running `kubeadm init` [preflight] WARNING: ethtool not found in system path ``` -Then you may be missing ebtables and ethtool on your Linux machine. You can install them with the following commands: +Then you may be missing `ebtables`, `ethtool` or a similar executable on your Linux machine. You can install them with the following commands: ``` # For ubuntu/debian users, try @@ -141,4 +141,4 @@ The `kubectl describe pod` or `kubectl logs` commands can help you diagnose erro kubectl -n ${NAMESPACE} describe pod ${POD_NAME} kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME} -``` \ No newline at end of file +```