Skip to content

Commit

Permalink
Merge branch 'main' into inline-exceptions
Browse files Browse the repository at this point in the history
  • Loading branch information
bakito authored Jun 6, 2024
2 parents a17222b + 19c152f commit b51963e
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 25 deletions.
48 changes: 25 additions & 23 deletions content/en/blog/general/introducing-reports-server/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ A high-level overview of the architecture is shown below.

## Performance

Reports server stores policy reports and ephemeral reports outside etcd thus reducing the database size of etcd. In the following tables, we show the database size of etcd of increased workloads with and without reports server. In this test, we installed Kyverno policies to audit the Kubernetes [Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/) using 17 policies. Subsequently, we created workloads (Pods, Deployments, ReplicaSets) that match the installed Kyverno policies and scheduled them on false kwok nodes to measure total size of policy reports in etcd. [kwok](https://kwok.sigs.k8s.io/) is a toolkit that enables setting up a cluster of thousands of Nodes in seconds. `DB_SIZE` in the output of `etcdctl endpoint status -w table` was used to calculate etcd DB size. For more details on these tests, refer to the testing documentation for [the report controller](https://github.com/kyverno/kyverno/tree/main/docs/perf-testing). The version of Kyverno used in this testing was 1.12.1.
The reports server stores policy and ephemeral reports, offloading them from etcd and minimizing its storage footprint. In the following tables, we show the database size of etcd of increased workloads with and without reports server. In this test, we installed Kyverno policies to audit the Kubernetes [Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/) using 17 policies. Subsequently, we created workloads (Pods, Deployments, ReplicaSets) that match the installed Kyverno policies and scheduled them on false kwok nodes to measure total size of policy reports in etcd. [kwok](https://kwok.sigs.k8s.io/) is a toolkit that enables setting up a cluster of thousands of Nodes in seconds. `DB_SIZE` in the output of `etcdctl endpoint status -w table` was used to calculate etcd DB size. For more details on these tests, refer to the testing documentation for [the report controller](https://github.com/kyverno/kyverno/tree/main/docs/perf-testing). The version of Kyverno used in this testing was 1.12.3.

### Without Reports Server

Expand All @@ -55,16 +55,17 @@ $ kubectl get polr -A | wc -l
```
`apiserver_storage_objects` metrics show that there are 10000+ policy reports stored in etcd along with other resources.

Total size of etcd, including all resources in the cluster with respect to amount of policy reports:
| Number of Policy Reports | Total etcd Size |
| --------------- | --------------- |
| 1270 | 134 MB |
| 2470 | 223 MB |
| 3770 | 280 MB |
| 4970 | 334 MB |
| 7370 | 467 MB |
| 9770 | 552 MB |
| 10010 | 552 MB |
Total size of etcd:

| Number of Policy Reports | Number of Pods | Total etcd Size |
| --------------- | --------------- | --------------- |
| 179 | 139 | 46 MB |
| 1199 | 1139 | 71 MB |
| 2219 | 2139 | 100 MB |
| 4259 | 4139 | 149 MB |
| 6299 | 6139 | 167 MB |
| 8339 | 8139 | 220 MB |
| 10379 | 10139 | 255 MB |

### With Reports Server

Expand All @@ -85,18 +86,19 @@ $ kubectl get polr -A | wc -l
```
`apiserver_storage_objects` metric does not find policy reports stored in etcd.

Total size of etcd, including all resources in the cluster with respect to amount of policy reports:
| Number of Policy Reports | Total etcd Size |
| --------------- | --------------- |
| 1204 | 71 MB |
| 2404 | 115 MB |
| 3604 | 152 MB |
| 4804 | 191 MB |
| 7204 | 276 MB |
| 9604 | 343 MB |
| 10250 | 370 MB |

As shown in the benchmark, the size of etcd grows as the number of resources in the cluster grows, but the growth is slower when reports server is installed. As reports server stores policy reports in a separate database, they don't take up any space in etcd. At 10,000 reports, the database size of etcd is 33% smaller compared to when reports server is installed.
Total size of etcd:

| Number of Policy Reports | Number of Pods | Total etcd Size |
| --------------- | --------------- | --------------- |
| 185 | 141 | 38 MB |
| 1205 | 1141 | 38 MB |
| 2225 | 2141 | 41 MB |
| 4265 | 4141 | 55 MB |
| 6305 | 6141 | 58 MB |
| 8345 | 8141 | 67 MB |
| 10385 | 10141 | 76 MB |

As shown in the benchmark, the size of etcd grows as the number of resources in the cluster grows, but the growth is slower when reports server is installed. As reports server stores policy reports in a separate database, they don't take up any space in etcd. At 10,000 reports, the storage size of etcd is 70.1% smaller compared to when reports server is installed.

## Getting Started

Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/installation/methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The Helm chart is the recommended method of installing Kyverno in a production-g

Since Kyverno is comprised of different controllers where each is contained in separate Kubernetes Deployments, high availability is achieved on a per-controller basis. A default installation of Kyverno provides four separate Deployments each with a single replica. Configure high availability on the controllers where you need the additional availability. Be aware that multiple replicas do not necessarily equate to higher scale or performance across all controllers. Please see the [high availability page](../high-availability/_index.md) for more complete details.

The Helm chart offers parameters to configure multiple replicas for each controller. For example, a minimally-viable, highly-available, complete deployment of Kyverno would consist of the following values.
The Helm chart offers parameters to configure multiple replicas for each controller. For example, a highly-available, complete deployment of Kyverno would consist of the following values.

```yaml
admissionController.replicas: 3
Expand All @@ -54,7 +54,7 @@ reportsController.replicas: 2
For all of the available values and their defaults, please see the Helm chart [README](https://github.com/kyverno/kyverno/tree/release-1.10/charts/kyverno). You should carefully inspect all available chart values and their defaults to determine what overrides, if any, are necessary to meet the particular needs of your production environment.
{{% alert title="Note" color="warning" %}}
All Kyverno installations require the admission controller be among the controllers deployed. The admission controller does not support two replicas. For a highly-available installation, the only supported count is three.
All Kyverno installations require the admission controller be among the controllers deployed. For a highly-available installation, at least 2 or more replicas are required. Based on scalability requirements, and cluster topology, additional replicas can be configured for each controller.
{{% /alert %}}
By default, the Kyverno Namespace will be excluded using a namespaceSelector configured with the [immutable label](https://kubernetes.io/docs/concepts/overview/working-with-objects/_print/#automatic-labelling) `kubernetes.io/metadata.name`. Additional Namespaces may be excluded by configuring chart values. Both namespaceSelector and objectSelector may be used for exclusions.
Expand Down
1 change: 1 addition & 0 deletions content/en/docs/releases/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ Release branches and PRs are managed as follows:
- The branch name will contain the version, for example `release-1.5`.
- Patch releases are created from a release branch.
- For critical fixes that need to be included in a patch release, PRs should always be first merged to `main` and then cherry-picked to the release branch. The milestone label (for example `milestone-1.5`) must be added to PRs that are to be merged to a release branch, along with the `cherry-pick-required` label. When the PR is cherry picked (or merged), the `cherry-pick-completed` label is added.
- PRs with no corresponding issues are added to the specific milestone for tracking.
- For complex changes, or when branches diverge significantly, separate PRs may be required for `main` and release branches.
- During PR review, the Assignee selection is used to indicate the reviewer.

Expand Down

0 comments on commit b51963e

Please sign in to comment.