Helm Chart Production Guide #4703
Conversation
hamza-m-masood
added
hold
|
👋 🤖 🤔 Hello, @hamza-m-masood! Did you make your changes in all the right places? These files were changed only in docs/. You might want to duplicate these changes in versioned_docs/version-8.6/.
You may have done this intentionally, but we wanted to point it out in case you didn't. You can read more about the versioning within our docs in our documentation guidelines. |
hamza-m-masood
force-pushed
the
production-guide
branch
from
7f6254c to
aad795a
Compare
|
@conceptualshark, I wanted to add a link to the OpenSearch IRSA guide, but I am having a problem with the docs. I opened an issue here: #4754 |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| ### Keycloak | ||
|
|
||
| - Connect to existing Keycloak: | ||
| - [Configuration of the Camunda Helm Chart](/docs/self-managed/setup/guides/using-existing-keycloak/) |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: 'Configuration of the Camunda Helm Chart'. Please specify the file extension.
|
|
||
| - Connect to existing Keycloak: | ||
| - [Configuration of the Camunda Helm Chart](/docs/self-managed/setup/guides/using-existing-keycloak/) | ||
| - [Configuration of Keycloak](/docs/next/self-managed/identity/user-guide/configuration/configure-external-identity-provider/) |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: 'Configuration of Keycloak'. Please specify the file extension.
hamza-m-masood
force-pushed
the
production-guide
branch
from
c9a6ceb to
9281533
Compare
| kubectl create namespace orchestration | ||
| ``` | ||
|
|
||
| Within the `management` namespace (Web Modeler and Console), we will install Identity, Console, and all the Web Modeler components. Within the `orchestration` namespace, we will install the Camunda Orchestration Core component, along with Connectors and Optimize importer. We do not have to worry about installing each component separately since that will be handled by the Helm chart automatically. For more information on the Orchestration Cluster vs Web Modeler and Console, please review this [guide](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console) |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [guide](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console). Please specify the file extension.
| - **Kubernetes Cluster**: A functioning Kubernetes cluster with kubectl access. We are going to use an AWS EKS cluster. Have a look at the following guides: | ||
| - [Deploy an EKS cluster with Terraform (advanced)](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-terraform/) | ||
| - [Install Camunda 8 on an EKS cluster](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-helm/) | ||
| - **Helm**: Make sure you have the [Helm CLI](/docs/reference/supported-environments/#clients) installed |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [Helm CLI](/docs/reference/supported-environments/#clients). Please specify the file extension.
| - **TLS Certificates**: Obtain valid X.509 certificates for your domain from a trusted Certificate Authority. | ||
| - **External Dependencies**: Provision the following external dependencies: | ||
| - **Amazon Aurora PostgreSQL**: For persistent data storage required for the Web Modeler component. Have a look at the [Set up the Aurora PostgreSQL module](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-terraform/#set-up-the-aurora-postgresql-module) guide. | ||
| - **Amazon OpenSearch**: is used as a datastore for Camunda Orchestration Core components. Have a look at our guide for setting an [OpenSearch domain](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-eksctl/#4-opensearch-domain). |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [OpenSearch domain](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-eksctl/#4-opensearch-domain). Please specify the file extension.
| - **Amazon Aurora PostgreSQL**: For persistent data storage required for the Web Modeler component. Have a look at the [Set up the Aurora PostgreSQL module](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-terraform/#set-up-the-aurora-postgresql-module) guide. | ||
| - **Amazon OpenSearch**: is used as a datastore for Camunda Orchestration Core components. Have a look at our guide for setting an [OpenSearch domain](/docs/self-managed/setup/deploy/amazon/amazon-eks/eks-eksctl/#4-opensearch-domain). | ||
| - **AWS Simple Active Directory**: For simple OIDC authentication in this scenario, we will use [AWS Simple Active Directory](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/directory_simple_ad.html). | ||
| - **Ingress NGINX**: Ensure the [ingress-nginx](https://github.com/kubernetes/ingress-nginx) controller is set up in the cluster. |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
|
|
||
| Within the `management` namespace (Web Modeler and Console), we will install Identity, Console, and all the Web Modeler components. Within the `orchestration` namespace, we will install the Camunda Orchestration Core component, along with Connectors and Optimize importer. We do not have to worry about installing each component separately since that will be handled by the Helm chart automatically. For more information on the Orchestration Cluster vs Web Modeler and Console, please review this [guide](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console) | ||
|
|
||
| #### Installing the Helm Chart |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| secretName: camunda-platform | ||
| ``` | ||
|
|
||
| Optionally, you can configure Ingress for Zeebe GRP endpoints. Here is an example Zeebe GRPC Ingress setup: |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| name: "camunda-credentials" | ||
| ``` | ||
|
|
||
| A secret called `camunda-credentials` will be generated. It will include all the needed secret values for the Camunda Helm Chart. |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| The `camunda-credentials` generated secret will not be deleted if the helm chart is uninstalled | ||
| ::: | ||
|
|
||
| - When upgrading the Camunda Helm Chart, make sure to read the [upgrade guide](/self-managed/operational-guides/update-guide/introduction.md) and corresponding new version elease notes before upgrading and perform the upgrade on a test environment first before attempting in production. |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| - If you have a use case for enabling [Network Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) then it is recommended to do so. | ||
| <!--Maybe link this to customer: https://github.com/ahmetb/kubernetes-network-policy-recipes--> | ||
| - It is possible to have a pod security standard that is suitable to the security constraints you might have. This is possible through modifying the Pod Security Admission. Please refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/security/pod-security-admission/) in order to do so. | ||
| - By default, the Camunda Helm Chart is configured to use a read-only root file system for the pod. It is advisable to retain this default setting, and no modifications are required in your `production-values.yaml`. |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| If you would like to add any other security constraints to your `production-values.yaml` then please refer to the official [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) | ||
|
|
||
| - It is recommended to pull images exclusively from a private registry, such as [Amazon ECR](https://aws.amazon.com/ecr/), rather than directly from Docker Hub. Doing so enhances control over the images, avoids rate limits, and improves performance and reliability. Additionally, you can configure your cluster to pull images only from trusted registries. Tools like [Open Policy Agent](https://blog.openpolicyagent.org/securing-the-kubernetes-api-with-open-policy-agent-ce93af0552c3#3c6e) can be used to enforce this restriction. | ||
| - Open Policy Agent can also be used to [whitelist Ingress hostnames](https://www.openpolicyagent.org/docs/latest/kubernetes-tutorial/#4-define-a-policy-and-load-it-into-opa-via-kubernetes). |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.inclusiveLanguage] Non-inclusive terminology used. Review the WCoE for using inclusive language.
|
|
||
| Here are some points to keep in mind when considering observability: | ||
|
|
||
| - It is possible to enable integration with Prometheus, a popular monitoring solution, in the Camunda Helm Chart. This can be configured by adding the following configuration below to your `production-values.yaml`: |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| Below is the high-level architecture diagram for the base production setup _(click on the image to open the PDF version)_: | ||
| [](./assets/smarch.pdf) | ||
|
|
||
| If you would like to learn more about the architecture setup, please refer to the [About Self-Managed](/docs/self-managed/about-self-managed/#architecture) and [Camunda 8 reference architectures](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console) documents. |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [About Self-Managed](/docs/self-managed/about-self-managed/#architecture). Please specify the file extension.
| Below is the high-level architecture diagram for the base production setup _(click on the image to open the PDF version)_: | ||
| [](./assets/smarch.pdf) | ||
|
|
||
| If you would like to learn more about the architecture setup, please refer to the [About Self-Managed](/docs/self-managed/about-self-managed/#architecture) and [Camunda 8 reference architectures](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console) documents. |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [Camunda 8 reference architectures](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console). Please specify the file extension.
|
|
||
| - Namespace `orchestration`: we will install the Camunda Orchestration Core component, along with Connectors and Optimize importer. | ||
|
|
||
| We do not have to worry about installing each component separately since that will be handled by the Helm chart automatically. For more information on the Orchestration Cluster vs Web Modeler and Console, please review this [guide](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console) |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [guide](/docs/self-managed/reference-architecture/#orchestration-cluster-vs-web-modeler-and-console). Please specify the file extension.
| 2. **TLS certificate**: A TLS certificate created for your domain. The certificate must be an X.509 certificate, issued by a trusted Certificate Authority. Also, the certificate must include the correct domain names (Common Name or Subject Alternative Names) to secure Ingress resources. Please reach out to your DNS provider if you are unsure on how to create a TLS certificate. It is not recommended to use self-signed certificates. | ||
| 3. **TLS secret**: A TLS secret created from your TLS certificate. In our example, we will use a secret called `camunda-platform`. Please refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets) on how to make a TLS secret. | ||
|
|
||
| Here is an example values.yaml configuration using the ingress domain and TLS secret mentioned above: |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[all.glossary] This term is spelled, uppercased, lowercased, hyphenated, or used incorrectly. Review the WCoE glossary.
| policyName: core-record-retention-policy | ||
| ``` | ||
|
|
||
| If you would like more information on configuring ILM policy. Please refer to [the configuration guide on the OpenSearch exporter](/docs/next/self-managed/zeebe-deployment/exporters/opensearch-exporter/#configuration). |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [the configuration guide on the OpenSearch exporter](/docs/next/self-managed/zeebe-deployment/exporters/opensearch-exporter/#configuration). Please specify the file extension.
|
|
||
| ### Upgrade and Maintenance | ||
|
|
||
| - Make sure to follow our [upgrade guide](/docs/next/self-managed/setup/upgrade/) when performing the upgrade on your Helm chart. |
There was a problem hiding this comment.
[docsInstance.hrefDocsToDocs] Improper link format: [upgrade guide](/docs/next/self-managed/setup/upgrade/). Please specify the file extension.
There was a problem hiding this comment.
I left some comments where I found things still either confusing or vague.
I don't know whether we should do expectation management in the overview?
To a customer, the Helm chart is the first product they have in their hands to deploy Camunda 8.
Just because your Helm chart is production-ready does not necessarily mean your Camunda 8 installation is production-ready. From the epic, I also get the focus on helm, but maybe setting expectations could help, as customers don't look at it as two different products, which we internally do.
Customers may not associate the Camunda Helm chart as not being just Camunda 8 and I guess the guidance would be different.
Maybe also to reel in @theburi on the above.
I'm starting to see things come together, well done 🚀!
| - **AWS OpenSearch Snapshot Repository** - This will be a place to store the backups of the Camunda cluster. This repository must be configured with OpenSearch to take backups. Have a look at the [official AWS guide](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-snapshot-registerdirectory.html) for detailed steps. | ||
| - **Amazon S3** - This will be used to store backups of the Camunda cluster. [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) must be configured with OpenSearch to take backups. |
There was a problem hiding this comment.
This part can be confusing as it talks twice about being a place to store backups of the Camunda cluster but also mentions within the S3 part OpenSearch again.
For OS it sounds like two actions have to be taken but afaik it's solved by the first call.
OS Snapshot requires configuring S3.
From reading the backup page, I understand that Operate / Tasklist / Optimize under the hood just use the OS Snapshot store.
Zeebe is saving a snapshot of its partitions to S3.
Coming to a production guide, I wonder what the recommendation is?
- Should it be the same bucket?
- If it's the same bucket, should the path somehow differ or can I put everything in the root?
This could be covered maybe in the Backup section before sending people off to the backup guide.
For this particular part here mentioned we may make it more obvious.
- AWS OpenSearch Snapshot Repository - This will be a place to store the backups of the Camunda WebApps. This repository must be configured with OpenSearch to take backups which are stored in Amazon S3. Have a look at the official AWS guide for detailed steps.
- Amazon S3 - An additional bucket will be used to store backups of the Zeebe brokers.
Just a suggestion. WebApps could also be frontend applications, not sure how we differentiate nowadays.
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Outdated
Show resolved
Hide resolved
| - **Kubernetes Cluster**: A functioning Kubernetes cluster with kubectl access. | ||
| - **Helm**: Helm CLI installed |
There was a problem hiding this comment.
I assume @theburi means https://docs.camunda.io/docs/next/reference/supported-environments/
I think I mentioned it somewhere else as well on this PR where I tagged @conceptualshark and @theburi.
@conceptualshark mentions a ticket in the below comment. Just the question whether we would want to add something like Helm cli as well or not.
The page of supported environments could generally be extended a bit better on what versions are supported and it may include Helm version as well down the line.
We do mention K8s and Helm but not really the versions while we have it quite detailed for the rest.
K8s supported version range.
Helm chart supported version, which would probably be something like x.y.z depending on the docs version.
8.6 --> 11.x.y with Helm 3.10.0+.
Just a rough sketch.
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Outdated
Show resolved
Hide resolved
| - Make sure to not store any state or important, long term business data in the local file system of the container. A pod is transient, if the pod is restarted then the data will get wiped. It is better to create a volume and volume mount instead. Here is some example configuration for the core component to create persistent storage: | ||
|
|
||
| ```yaml | ||
| core: | ||
| extraVolumes: | ||
| extraVolumes: | ||
| - name: persistent-state | ||
| emptyDir: {} | ||
| extraVolumeMounts: | ||
| - name: persistent-state | ||
| mountPath: /mount | ||
| ``` |
There was a problem hiding this comment.
keeping this open for now, not sure the addition was done or is still being considered.
If not feel free to close.
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Outdated
Show resolved
Hide resolved
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Outdated
Show resolved
Hide resolved
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Show resolved
Hide resolved
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Outdated
Show resolved
Hide resolved
docs/self-managed/operational-guides/production-guide/helm-chart-production-guide.md
Show resolved
Hide resolved
…rt-production-guide.md Co-authored-by: Lars Lange <9141483+Langleu@users.noreply.github.com>
Description
closes: https://github.com/camunda/distribution/issues/344
When should this change go live?
holdlabel or convert to draft PR)PR Checklist
/versioned_docsdirectory./docsdirectory (aka/next/).