Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cluster role document #1796

Merged
merged 21 commits into from
Oct 18, 2022
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions assets/docs/troubleshooting/provisioning_flow_chart.drawio
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<mxfile host="app.diagrams.net" modified="2022-10-18T00:18:01.657Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36" etag="TN0chO2VRAjQmLvDBTCH" version="20.4.1" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="main_chart">7V1bk9q4Ev41U+eJKd+w4XEumWTPJqmczW3ztGWwAG+MTWwzM+TXH8lYGCQDQmNbgulJVWZsjGzo/vrerSv7bv78NvUXsw9JgKIrywier+z7K8syHcvFv8iZ1frMwHXWJ6ZpGJQXVSc+h79RedIozy7DAGU7F+ZJEuXhYvfkOIljNM53zvlpmjztXjZJot27Lvwp4k58HvsRf/Z7GOQz+nRGdf4dCqez8s4efWHkj39O02QZl7e7suzJgPxbvzz36VLl9dnMD5KnrVP2myv7Lk2SfP3X/PkOReSrpd/a9z9W36P3P923//1f9sv/evvnl4/feuvFHk55y+YDpijOpZf+Nfr4Ie19+I6yn0+rYH7/9unjU/kW49GPloh+C8VnzVf06y2+IUQWMa7s21k+j/CfJv7zX5Tnq5Id/GWe4FNJms+SaRL70fskWZTXTZI4Ly8zyTGKgxtCdnw8ipLxz/WphzCiC+Oj8vo+PsryNPm5oaxFzlBCkAeK/BGKbjekvEuiJMUvxUmMyFIBZpXys1QP96Y6e4uew/xvstZ1vzz6QR8D/33/XN6mOFiVB+tHorfCfGMYruG5+BVBSpUUzZJlOkYHyGOawxJQfjpF+aEr19eRz7uFipIT3qJkjvJ0hS9IUeTn4eMudPwSgdPNdZu3fkpC/Eksg0oLo3xLKStMCg66xPojle+quBH/sfUY1amCR0/gV7OGX90oL7msEDKUKO6vJUEmJolt+uTf9il3Sn7fFDf8lASFJEJX5MOsVxul9Jp0GcdhPL2yH+hr+MHXN1tfwANmlsxHS/x93D7Nwhx9XvgFjZ+w9N0F0A4fkyUf/HkYkW/2HYoeUR6OfQY+hPv9KJzG+GCMWQyl9QDBtyye+d6tjr4UgOw5dfx7V/yczL+PKM3R80F+K191jF22ccvjpy2BTWXPbEtYD4z9HLrDW6cyksUx0p0fkxuNxyjLyCuhzzPDnwPyEv4yU3IRYQmg/cm035xQRnybI/49WkTJao4KGRISImfLNScAkYWIbBm6IdyRUhXId0euy6uKj4mo+D9ucnRvUb3MhiotJGoT/dixluotJMxd6erv7YOtd5HD6m3F0V7L6s2Ne+u65WfeOu/hF95QWNGve9d4tIv1/JR+VW3YZ4LWmS1onZVg6hnXRp+6ZSsKsPWhrAFHL0kmkwzt2Ga8keftYtm2xYw8biFWKGxcRrrQ+ovjFsLM7K+2LluQC7I27Mk+JyTWKn4cLTMifDkLoNAN/qMfYswQniNWoT8ncj8eZQswB8W0he3qpi1c0BY1/vdAhe44rAO2Zf0+nSCrfRrWDragduiflXbYaIPy1kNJ5dBnLUZDTDk0Jfo9QLwuiG/HWhSRFA0jvi+I+MFZI94SDPrxkLfk7MGmID/gIP8pRYt15I+x+/YGwc3jhhxrok0jnwQStlh2L7YmGJOUTwM08ZeFSGkq3ixuoLGUUm+gDTnaffOj4D8F0WbYySvicry9vsKsWTxiNC/AskggoiNno29Iq4wFqLQ4UWV7WGcjg1fZP1AmHNI/a4181OA4oi03WpVjwsrilsiiVVq/UvQ/dvR8J1a6qM4eCupsBfEak0GqK6mf2YXsjk1yk0/v3QQBPlEr6o0a2f7a9TYrtB1DUGg7rQltS0pog58l6Gd1IXG187OEqyJEyyI2MYjOhLYzbMqpOuadtS20+Wzq1wytTwQ9Ipx7yQKlfk6YAkxvESnuDrQzveWyqWB6d2d6dxFwUyXGJfKnXKVah6Kdlciy9ji7UOf2OJ8e9ReLaFUK96u6erm5H4cTlJFPASU0J0l902AyIpsgnDqxvz8t6pNnSNFkR+TP8pwUfd+Qu1oP0zCfLUfX4wS7Zw+PgY8fD/8uGOchTxEij+Svw60PhWNXvW6vf7F64yF8rvy+LU7zx+MkDQg5Se05kRRYwhfXYP4j5eOVsvEFNE13viL+pA/FjxJv0TRd7TiOT8tVdiQvbTaWZVm1V1TrTkHiyNmZwtGC9sjPp2hOIX8+I44HVvBELAATSDGBBjKAz/WAs6GXs3GGcX5TtByHqiAdI/1sMb109Y3LhJ+4Gs+WXQv6pQHGLwLjTWPVE8WqaCVN91j1+gzCPEmosgs5HVfNWHxWDjI4DWC5gTpXidCfMq0qWuXafbzOtNhSF1mtah/rnGgbqhZAVZei1ospYzdFK2RotFBLiDPIdNiCB+EaGds8slLbGOfzrV3EZJlMrmWHcZb75HFZqfFXSeUiEBsUjbUQf60NvQwc3eJvFnTNggJR5s1Z59UnO2DLJWX7ZDk50LVK4fO8SlTKcjFNfZLG6+Uz1MNyNAuTmJcqX4vL9pQYgaYRCfK7ymdwWNBxq01daJOaRm2MfyCqafSNRrBaxenLahU2bsgu1LZWkeuwhRA/1Aw2AXHhgKMeRYNclF82Bsku1LfxB9v6sboVAnxBx7d16eDNdGva1vaUPRbMULNxdNqWp7w914KaDd2F/RnWbIgmlyx908BcMbisOcctxCqIliU5vf1On8+65JaK8iwM0NiHNh/JZk31UtyuacBlaFs/EQ3090mUZ8draEB5i6P8PdZ9JGhnhPOtEvqDo/CA9EdJz0biNCA9n1D8nCdpQXMDiNwEkQfKGzhtPrFXzT5Cz/irxXZnQdiS8ndXRTk9mbOwXAR+Tq6rOm5efUCdTdloQGE+ifMNpeGkMHvZjqlXTz8WoVXznToCymVEwIWGeOkhrirZ9fiM6hNnZShMtLtsBbRw8ZZxbKW2nWiYOnpB9TUNQ9USTW1QTIumNoxr06EBWhXjyeym4MtOpu8cvnw2A7zkLkJjGpjXfILjS1XmSvBqGQHDC6/eyma9JPXBDvpAQMYXhCvVo9GBnjSwo455MsftKOs0O6pHDCmnr8qQYuewD4zh9XDrZyBnVbFNa0eWbdnEcixA9uUgW5cgiCUqEUSnPOvRgeD2W5EI7HAIxRJhf8MbxEX1iIueY2mRsEgQbUpSP2KiL9uAxC3EzqpoG+LQdnhBSr9ppJ53V2BDOpkNpQw7HvPt7O8RBC0MWviF2D6uhU9s2FKohQdOI4Dnlh2q7ONwoDjhkuDfNIxFawc2CUnD2FXQCnfeaEhBO/sqwrpCKJQWQEO1uuqhEztwNlMFIW7eEPz50gSAvx5TGjWJgYtWF1E9dh7VRa8jBA7dtZdkfeui3EUlQl9UInQP/z6L02acb4fd/9roeOsl+pVvIf7Huonj7tNXvsowWy4WSbEz6s23vy2oMpSrMtSgCaTPFzbdLsNiYAZblmaMaGtPhqIJeZiYXLYsNl8Mc55Jqulo1bmqCWwcLddD2own/OQFG+1XBq+18k0HDrHA0NfU0Fc/jl1YU+uRK2NDccPDs4qEt8TlUNvxQM0+lKyAvd4auI/b64629jo7xWboNYJ4zl43O06P068cuq+lDO8NudSZVXx9Ax1vhIVd4dOAUyVCW3Y2XbVnmTraQvIa1LG6dt2+vrUr7LYo0jsfsQt5HVeQ9vnsdym2LeOtn6Mnf8UHP2B0WQOqWwPxDl3ZimhvK+/n7fOZMbC79xKQm3rUVx7OpBwEtpm2ttkZTj3qi2455IqGSrYCpgqnxHOTkGSnCXMhUg7hLRtsLp/oKgfDv7/l9fXGhCsGUrKv4t/+nMjreJSRX2DSvcxjd9VrBQu0wgVphaalu74ONZvRMuX3EuYcra4ltFwSCxLNnSWazzBoJmyY9U8sKDeH6jBuyGKcS1ttQN8Vxvm0FQ2bsUbWXTJfYAaOawqLII7WgCuuPo7m8ikwmI12coWY+piYC1traq64VdrOonpVzwoxeV3LAdXpuKTbhf0wdfdpJQ1q2962qHvr7ubDZnVx9AmlIeYqYrO0CfXjtrbE1DODbta+aehUV0RWTZh+6aZq1YTproQC5C67sbfZ+Tk62NvQ1XehCkFlhIW69G1IfU/Z7Nvm4qqew2TvHEGRjxm8yEDRyxbkgqzmY+29USlJxJ9s9w2kLat4hkY1EP304C1egreoSf5dWAi55+6OSifgWcuTW6lly9PjE/AQ7TvGflX/pz5mpGdxhIQSOHE/QH281oNE6wVp4KY1qWgQh8rzE2bwOZ42U3Ll7flB/9hKbStSPocKbjy48Z3h3hLEvQbWsjTI2XFc3YNcbvw16Ggti6EaxiqtRzzu7Z64x26XRU2sP2o3llOxO86peFD/oDlWzzCiZQlCXLg2WY+AFtu3Kw97bqWuN+f1+PqKP+JpirKsQHw8Cad7kqnjJE3ROIdUqmS/iKd8eocHafRuaM91JajvFfKgBfglBFQPXrowZCJeUnesHogD2JQZ7O7G42Cimzl755VJviDDe2BJ4R4C4BAAbwT4niDwuwf5kM1ySYOcX6njoNoAktQXpNwbxqowBPXQvUNWYzbXpO913MA7kEs+Ayy1hKUmNrcnmnumruS54J7tubbZFkDpZkK34wnzA9iOGWxudcCnMVewuVsFOZ/IXs9IY3MZ92E2Th5RWje3gx2OBsM6Gsh4DZUXDQ/4bCdkvDpJmGhAez7bCQmTU+lobSZJqKOjXAMwOG9aOm8N22JU+R93wk4Mvqgt/OdqgocsvIR9ME4pd5z3oJ8E4KsdfPVwpwaiww4H55W7rHpVuSKSk0E8ZLWyYAB1fzd+4zCXK2uAUAuEWg5wlScqG4aWoGzQoIbBbWr7INNjgzZt63JrL8h94g6kaLID8FmeL/CT3ZAHsR6mYT5bjq7HyRwf/FyOUBqjHGW9LJxm5HHww4VjfIjSxyJEw4gE7nUqHfytIA7MXG0+jDNQ3rw/hJ1iQbk0rlyoNDtueLrno1wGrHsnrVwGHY9aGfJJ+nu0iBLyxbKy3xiRs5MkipKnYvvR3eh9e0rJDuMs9yNC1yTmJcs7lKJ6zQSRRTZCrIFagXZlXUMT6iOLwh6FHtEGm+0n5vYWla7WGnS8pcqwJqELrUr7+M6j+RF9Mm9DmOGtu7muMmVTE9OxfXf13fvVe/du9nt1Ez1M0i9RT7RsrnthyyVVPOkBh9xKgiNbTp2z2meq+iw6o2/vkx15w1Urc1aHfNIeRIdeouMMPX3RJpnzCSJbsgJH9Rb0w5o+dKjW0yDMW51QZjea9AlA+oP0V1AjRGXTGch/c9hYI/SwYw2w2RgcinI7LsrVQsRbHPGhKvfUMSZaEBJanSF4fkyTHlW5G2VwUl3ubsSzy7pcFogmW4gnHGVnzW+7495I04Cm6AsCsCZN0Rs8CyBfNNqiR4KNa4seNtUWbVmdQ59PfEOGTdyMtpUPAzQNuenNECmBSElD0lt0grMCI40thZCOlfArdR4rgVQ64Fyleybac6U+JGpJ7zJosQre6dwg49PeNzH5WhdpMorQnI+IoucxWpDPkM/W09lnfppDYFQy9+Uor0U1Ddjn+jIlvVJ/3LoAQc+VtsoPIjw60rBtQU+7M7ZQ/r7IecwR/u9njLkeP8woWeaVK17qgOvaOgjRPod/8Ue6ziIMo7LXAd/gAT9F0MOH82Uc5phKDxh4KQr+CeNHrCrw8e+8F4ysn6th4vT++me4sLJvi8HzX0+/82/+YDqJP93wkuczuQk0QYiFEjRQPCZ0bmurePRQIhvlcFyNUPkmns4xHUtdOofRLJa0ZuFW4ubpta5ZLCkYQzpHy70XNQkUnIB8SxT5m1hadzBnx6fIV8+y+rvfdUDQhFZ4bbX1GeNcdG9W6/QZTAO6K7AGIcKuq+ZP7dIxDUbAeM7hLh1WIDHXt9OkY5p1pSMOcWCXIzTOyQLrSr5FEpDgZC9ZeziHavkMMRewzo87DH3G+zOKnxZ9PK5EqDTMtj08t8bDY4eDNejh1fW575JrHC2zHKW9MJ4kr4FIdr9/nEp2t1SqS+k7xe2jeSEvFwRGUXiwKONiKWTbtnIK1SVjeQr5QbC+iPzajcKRk9ePge9n10k6xSeKtE32GuiJdeb1LkUd2+MoatHu4I4oWtduWqfIQrpH8uUTyhzuksn0PPXIk0uOTYof3uvZRNI3MeucZE8rR2jEXnk0CBJF4SJDx3nAzxbFJtr3k/CZ8A2fQCWUCsd+dFO+MA+DIKplhDJYvT+MzbtMbTEN3YeOQpuPatMAdjdBbcqKbTHMmzgAdnmBMrB2+cXq1wwtaYtjDg2b2OKXP7LKLibKPYlqGtOwB5oXA8jIO8kNoABDhP59pUNrahmgpimRTp9bk94oaX3ICHgteUyWft1uE1FLP0tK4ENctP246GlMuB2rPCSpj3dz6zu106HD5Ta3lp3a2R+yQBTc3qGVyfC19II+UU3zku3j8pCiPZO2L8dmLFXpKmNuJZNtIGsRqvgwTQiQqsuxZTL7kASIXPF/</diagram></mxfile>
3 changes: 1 addition & 2 deletions assets/docs/troubleshooting/provisioning_flow_chart.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion assets/docs/troubleshooting/troubleshooting.drawio

This file was deleted.

4 changes: 4 additions & 0 deletions docs/overview/component/discoverer.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,7 @@ For example, Vald LB Gateway creates the new `discoverer client` with config par
When the `initContainer` successes, Vald LB Gateway can get the metrics asynchronously according to its set parameters.

<!-- TODO:image -->

### Cluster role configurations

Please refer [here](../../user-guides/cluster-role-binding.md) for more information about the cluster role configuration.
151 changes: 151 additions & 0 deletions docs/user-guides/cluster-role-binding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Cluster Role Configuration

The [cluster role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) feature provided by Kubernetes contains rules that represent a set of permission to grant access to a specific target depending on the binding rule.

This page describes why we need a cluster role for the Vald cluster and how to configure it.

## What are cluster role and cluster role binding for the Vald cluster?

Vald applies the distributed index system across the Kubernetes cluster depending on the resource usage of the Kubernetes Node, it requires configuration to grant permission to a specific role to retrieve cluster information on Kubernetes.

By default, the cluster role configurations are deployed automatically when using Helm.

The following manifest will be deployed by default.

- [clusterrole.yaml](https://github.com/vdaas/vald/blob/main/k8s/discoverer/clusterrole.yaml)
- [clusterrolebinding.yaml](https://github.com/vdaas/vald/blob/main/k8s/discoverer/clusterrolebinding.yaml)

These configurations allow the service account `discoverer`, which is for the Vald Discoverer components, to access different resources in the Kubernetes cluster.

This service account allows [Vald Discoverer](../overview/component/discoverer.md) to retrieve Node and Pod resource usage from [kube-apiserver](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/), and share it to other components in the Vald cluster.

For example, [Vald LB Gateway](../overview/component/lb-gateway.md) will control which Vald Agent to insert based on the Node and Pod resource usage retrieved by Vald Discoverer.

If you are interested, please refer to the [insert data flow](../overview/data-flow.md#insert) for more detail.

## Configuration for Vald Discoverer

As described in the above section, Vald Discoverer requires configuration on cluster role and cluster role binding to retrieve Node and Pod information from the Kubernetes Cluster.

In this section, we will describe how to configure it and how to customize these configurations.

### Cluster role configuration for Vald Discoverer

By looking at the [cluster role configuration](https://github.com/vdaas/vald/blob/main/k8s/discoverer/clusterrole.yaml), the access right of the following resources are granted to the cluster role `discoverer`.

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: discoverer
...
kevindiu marked this conversation as resolved.
Show resolved Hide resolved
rules:
- apiGroups:
- apps
resources:
- replicasets
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes
- nodes/proxy
- services
- endpoints
- pods
verbs:
- get
- list
- watch
- nonResourceURLs:
- /metrics
verbs:
- get
- apiGroups:
- "metrics.k8s.io"
resources:
- nodes
- pods
verbs:
- get
- list
```

All of these rules are required to retrieve Node and Pod resource usage from [kube-apiserver](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) and also used to discover new Vald Agent Pods or Nodes created on the cluster.

### Cluster role binding configuration for Vald Discoverer

The cluster role binding configuration binds the cluster role `discoverer` described in the previous section to the service account `vald` according to the [configuration file](https://github.com/vdaas/vald/blob/main/k8s/discoverer/clusterrolebinding.yaml).

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: discoverer
...
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: discoverer
subjects:
- kind: ServiceAccount
name: vald
namespace: default
```

When the role binds to the service account, the access right of the role will be granted to the service account.
In this case, all the access rights of the role `discoverer` will be granted to the service account `vald`.

The service account `vald` is required for [Vald Discoverer](https://github.com/vdaas/vald/blob/main/k8s/discoverer/deployment.yaml#L155) to retrieve the required information to operate the Vald cluster.

For more information about Vald Discoverer, please refer [here](../overview/component/discoverer.md).

## Customize cluster role and cluster role binding configuration on Helm chart for Vald Discoverer

To customize the cluster role configuration on the Helm chart for Vald Discoverer, you may need to change the `discoverer.clusterRole` configuration on the Helm chart file. The cluster role configurations are enabled by default.

```yaml
discoverer:
...
clusterRole:
# discoverer.clusterRole.enabled -- creates clusterRole resource
enabled: true
# discoverer.clusterRole.name -- name of clusterRole
name: discoverer
clusterRoleBinding:
# discoverer.clusterRoleBinding.enabled -- creates clusterRoleBinding resource
enabled: true
# discoverer.clusterRoleBinding.name -- name of clusterRoleBinding
name: discoverer
serviceAccount:
# discoverer.serviceAccount.enabled -- creates service account
enabled: true
# discoverer.serviceAccount.name -- name of service account
name: vald
...
```

<div class="warning">
If you disable these configurations, the Vald Discoverer will not work, and the Vald cluster will not be functional.
</div>

If you want to modify or disable these configurations, you need to grant the [cluster role configuration](https://github.com/vdaas/vald/blob/main/k8s/discoverer/clusterrole.yaml) and bind it to the Vald Discoverer to retrieve required information to operate the Vald cluster.

## Customize cluster role configuration on Cloud Providers

Please refer to the official guidelines to configure cluster role configuration for your cloud provider, and configure the service account name for Vald Discoverer.

kevindiu marked this conversation as resolved.
Show resolved Hide resolved
For example:
- [Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html)
- [GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control)

For other cloud providers, you may need to find the related document on their official website, or you can enable the cluster role and the cluster role binding configurations on the Helm chart.

## Related Documents

- [Using RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
kevindiu marked this conversation as resolved.
Show resolved Hide resolved
- [Vald Discoverer](../overview/component/discoverer.md)
- [Data Flow](../overview/data-flow.md)