Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tidb-operator: fix documentation usability issues in GCP document #519

Merged
merged 27 commits into from
May 28, 2019
Merged

tidb-operator: fix documentation usability issues in GCP document #519

Changes from 7 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
360adec
tidb-operator: fix documentation usability issues in GCP document
May 24, 2019
713afe8
update wording
May 24, 2019
bbd543c
address hailong's comments
May 27, 2019
ad5e2d6
update configure terraform
May 27, 2019
7fc8b82
Update README.md
May 27, 2019
69f675f
add more descriptions for configure terraform
May 27, 2019
1a9f650
update wording
May 27, 2019
8345618
update wording
May 27, 2019
0ca9498
add descriptions and examples
May 27, 2019
c7545b4
update wording
May 27, 2019
6197efb
add some examples and update wording
May 27, 2019
b12270a
update examples and wording
May 27, 2019
d8da9fc
address hailong's comment
May 27, 2019
c4868b4
improve display
May 27, 2019
02e9e27
update code block
May 27, 2019
05ac1c3
unify wording
May 27, 2019
c223730
update wording
May 27, 2019
eaf5853
update descriptions
May 27, 2019
7a90aa9
Update README.md
May 27, 2019
90d1732
fix messy code
May 27, 2019
b54ae79
Merge remote-tracking branch 'upstream/master' into Fix-documentation…
May 27, 2019
43adae1
fix messy code for dind tutorial
May 27, 2019
03717ef
Update README.md
jlerche May 28, 2019
ad02659
address jacob's comments
May 28, 2019
2a2a8a4
update wording
May 28, 2019
6928911
update descriptions
May 28, 2019
43cf493
Merge branch 'master' into Fix-documentation-usability-issues-found-i…
lilin90 May 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 54 additions & 30 deletions deploy/gcp/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ First of all, make sure the following items are installed on your machine:

## Configure

Before deploying, you need to configure the following items to guarantee a smooth deployment.
Before deploying, you need to configure several items to guarantee a smooth deployment.

### Configure Cloud SDK

After you have installed Google Cloud SDK, you need to run `gcloud init` to [perform initial setup tasks](https://cloud.google.com/sdk/docs/initializing).
After you install Google Cloud SDK, you need to run `gcloud init` to [perform initial setup tasks](https://cloud.google.com/sdk/docs/initializing).

### Configure APIs

Expand All @@ -44,7 +44,9 @@ The terraform script expects three environment variables. You can let Terraform

> *Note*: The service account must have sufficient permissions to create resources in the project. The `Project Editor` primitive will accomplish this.

To set the three environment variables, you can first run `vi ~/.bash_profile` and insert the `export` statements in it. Here is an example in `~/.bash_profile`:
To set the three environment variables, you can first run `vi ~/.bash_profile`, append the `export` statements to it and run `source ~/.bash_profile`.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually type export FOO=var or something like that in the terminal, so setting the environment variables in bash_profile is a little overkill. I would suggest removing the line "To set the three environment variables, you can first run vi ~/.bash_profile, append the export statements to it and run source ~/.bash_profile. "

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If export in the terminal, after customer quit from the session and then open another session to run terraform, the variables need to be exported again, I think it's not convenient

Here is an example in `~/.bash_profile`:

```bash
# Replace the values with the path to the JSON file you have downloaded, the GCP region and your GCP project name.
Expand All @@ -55,14 +57,14 @@ export TF_VAR_GCP_PROJECT="my-project"

## Deploy

The default setup will create a new VPC, two subnetworks, and an f1-micro instance as a bastion machine. The GKE cluster is created with the following instance types as worker nodes:
The default setup creates a new VPC, two subnetworks, and an f1-micro instance as a bastion machine. The GKE cluster is created with the following instance types as worker nodes:

* 3 n1-standard-4 instances for PD
* 3 n1-highmem-8 instances for TiKV
* 3 n1-standard-16 instances for TiDB
* 3 n1-standard-2 instances for monitor

> *NOTE*: The number of nodes created depends on how many availability zones there are in the chosen region. Most have 3 zones, but us-central1 has 4. See [Regions and Zones](https://cloud.google.com/compute/docs/regions-zones/) for more information and see the [Customize](#customize) section on how to customize node pools in a regional cluster.
> *Note*: The number of nodes created depends on how many availability zones there are in the chosen region. Most have 3 zones, but us-central1 has 4. See [Regions and Zones](https://cloud.google.com/compute/docs/regions-zones/) for more information and see the [Customize](#customize) section on how to customize node pools in a regional cluster.

The default setup, as listed above, requires at least 91 CPUs which exceed the default CPU quota of a GCP project. To increase your project's quota, follow the instructions [here](https://cloud.google.com/compute/quotas). You need more CPUs if you need to scale out.

Expand All @@ -75,7 +77,7 @@ terraform init
terraform apply
```

When you run `terraform apply`, you may be asked to set three environment variables for the script to run if you have not exported them in advance. See [Configure Terraform](#configure-terraform) for details.
When you run `terraform apply`, you may be asked to set three environment variables if you have not exported them in advance. See [Configure Terraform](#configure-terraform) for details.

It might take 10 minutes or more to finish the process. A successful deployment gives the output like:

Expand All @@ -87,7 +89,7 @@ Outputs:
cluster_id = my-cluster
cluster_name = my-cluster
how_to_connect_to_mysql_from_bastion = mysql -h 172.31.252.20 -P 4000 -u root
how_to_ssh_to_bastion = gcloud compute ssh bastion --zone us-west1-a
how_to_ssh_to_bastion = gcloud compute ssh bastion --zone us-west1-b
kubeconfig_file = ./credentials/kubeconfig_my-cluster
monitor_ilb_ip = 35.227.134.146
monitor_port = 3000
Expand All @@ -107,76 +109,91 @@ gcloud compute ssh bastion --zone <zone>
mysql -h <tidb_ilb_ip> -P 4000 -u root
```

> *NOTE*: Make sure that you have installed the MySQL client before you connect to TiDB via MySQL.
> *Note*: You need to install the MySQL client before you connect to TiDB via MySQL.

## Interact with the cluster

It is possible to interact with the cluster using `kubectl` and `helm` with the kubeconfig file `credentials/kubeconfig_<cluster_name>`. The default `cluster_name` is `my-cluster`, which can be changed in `variables.tf`:
You can interact with the cluster using `kubectl` and `helm` with the kubeconfig file `credentials/kubeconfig_<cluster_name>`. The default `cluster_name` is `my-cluster`, which can be changed in `variables.tf`:

```bash
# By specifying --kubeconfig argument
# By specifying --kubeconfig argument.
kubectl --kubeconfig credentials/kubeconfig_<cluster_name> get po -n tidb
helm --kubeconfig credentials/kubeconfig_<cluster_name> ls

# Or setting KUBECONFIG environment variable
# Or setting KUBECONFIG environment variable.
export KUBECONFIG=$PWD/credentials/kubeconfig_<cluster_name>
kubectl get po -n tidb
helm ls
```

## Upgrade

To upgrade the TiDB cluster, modify the `tidb_version` variable to a higher version in `variables.tf` and run `terraform apply`.
To upgrade the TiDB cluster, modify the `tidb_version` variable to a higher version in the `variables.tf` file, and run `terraform apply`.

For example, to upgrade the cluster to the 2.1.10 version, modify the `tidb_version` to `v2.1.10`:

```
variable "tidb_version" {
description = "tidb cluster version"
default = "v2.1.10"
description = "TiDB version"
default = "v2.1.10"
}
```

The upgrading does not finish immediately. You can watch the upgrading process by `kubectl --kubeconfig credentials/kubeconfig_<cluster_name> get po -n tidb --watch`.
The upgrading does not finish immediately. You can run `kubectl --kubeconfig credentials/kubeconfig_<cluster_name> get po -n tidb --watch` to verify that all pods are in `Running` state. Then you can [access the database](#access-the-database) and use `tidb_version()` to see whether the cluster has been upgraded successfully:

```sql
MySQL [(none)]> select tidb_version()\G
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "\G" required?

*************************** 1. row ***************************
tidb_version(): Release Version: 2.1.10
Git Commit Hash: v2.1.10
Git Branch: master
UTC Build Time: 2019-05-22 11:12:14
GoVersion: go version go1.12.4 linux/amd64
Race Enabled: false
TiKV Min Version: 2.1.0-alpha.1-ff3dd160846b7d1aed9079c389fc188f7f5ea13e
Check Table Before Drop: false
1 row in set (0.001 sec)
```

## Scale

To scale the TiDB cluster, modify `tikv_count`, `tikv_replica_count`, `tidb_count`, and `tidb_replica_count` to your desired count, and run `terraform apply`.
To scale the TiDB cluster, modify `tikv_count`, `tikv_replica_count`, `tidb_count`, and `tidb_replica_count` in the `variables.tf` file to your desired count, and run `terraform apply`.

Currently, scaling in is not supported since we cannot determine which node to remove. Scaling out needs a few minutes to complete, you can watch the scaling-out process by `kubectl --kubeconfig credentials/kubeconfig_<cluster_name> get po -n tidb --watch`.

For example, to scale out the cluster, you can modify the number of TiDB instances from 2 to 3:
For example, to scale out the cluster, you can modify the number of TiDB instances from 1 to 2:

```
variable "tidb_count" {
default = 3
description = "Number of TiDB nodes per availability zone"
default = 2
}
```

> *Note*: Incrementing the node count will create a node per GCP availability zone.
> *Note*: Incrementing the node count creates a node per GCP availability zone.

## Customize

You can change default values in the `variables.tf` file (such as the cluster name and image versions) as needed.
You can change default values in `variables.tf` (such as the cluster name and the TiDB version) as needed.

### Customize GCP resources

GCP allows attaching a local SSD to any instance type that is `n1-standard-1` or greater. This allows for good customizability.

### Customize TiDB Parameters
### Customize TiDB parameters

Currently, there are not too many parameters exposed to be customized. However, you can modify `templates/tidb-cluster-values.yaml.tpl` before deploying. If you modify it after the cluster is created and then run `terraform apply`, it will not take effect unless the pod(s) is manually deleted.
Currently, there are not too many parameters exposed to be customized. However, you can modify `templates/tidb-cluster-values.yaml.tpl` before deploying. If you modify it after the cluster is created and then run `terraform apply`, it can not take effect unless the pod(s) is manually deleted.

### Customize node pools

The cluster is created as a regional, as opposed to a zonal cluster. This means that GKE will replicate node pools to each availability zone. This is desired to maintain high availability, however for the monitoring services, like Grafana, this is potentially unnecessary. It is possible to manually remove nodes if desired via `gcloud`.
The cluster is created as a regional, as opposed to a zonal cluster. This means that GKE replicates node pools to each availability zone. This is desired to maintain high availability, however for the monitoring services, like Grafana, this is potentially unnecessary. It is possible to manually remove nodes if desired via `gcloud`.

> *NOTE*: GKE node pools are managed instance groups, so a node deleted by `gcloud compute instances delete` will be automatically recreated and added back to the cluster.
> *Note*: GKE node pools are managed instance groups, so a node deleted by `gcloud compute instances delete` will be automatically recreated and added back to the cluster.

Suppose we wish to delete a node from the monitor pool, we can do:
Suppose that you need to delete a node from the monitor pool. You can first do:

```bash
$ gcloud compute instance-groups managed list | grep monitor
gcloud compute instance-groups managed list | grep monitor
```

And the result will be something like this:
Expand All @@ -187,24 +204,31 @@ gke-my-cluster-monitor-pool-7e31100f-grp us-west1-c zone gke-my-cluster-moni
gke-my-cluster-monitor-pool-78a961e5-grp us-west1-a zone gke-my-cluster-monitor-pool-78a961e5 1 1 gke-my-cluster-monitor-pool-78a961e5 no
```

The first column is the name of the managed instance group, and the second column is the zone it was created in. We will also need the name of the instance in that group, we can get it as follows:
The first column is the name of the managed instance group, and the second column is the zone in which it was created. You also need the name of the instance in that group, and you can get it by running:

```bash
gcloud compute instance-groups managed list-instances <the-name-of-the-managed-instance-group> --zone <zone>
```

For example:

```bash
$ gcloud compute instance-groups managed list-instances gke-my-cluster-monitor-pool-08578e18-grp --zone us-west1-b

NAME ZONE STATUS ACTION INSTANCE_TEMPLATE VERSION_NAME LAST_ERROR
gke-my-cluster-monitor-pool-08578e18-c7vd us-west1-b RUNNING NONE gke-my-cluster-monitor-pool-08578e18
```

Now we can delete the instance:
Now you can delete the instance by specifying the name of the managed instance group and the name of the instance, for example:

```bash
$ gcloud compute instance-groups managed delete-instances gke-my-cluster-monitor-pool-08578e18-grp --instances=gke-my-cluster-monitor-pool-08578e18-c7vd --zone us-west1-b
gcloud compute instance-groups managed delete-instances gke-my-cluster-monitor-pool-08578e18-grp --instances=gke-my-cluster-monitor-pool-08578e18-c7vd --zone us-west1-b
```

## Destroy

When you are done, the infrastructure can be torn down by running:

```bash
$ terraform destroy
terraform destroy
```