Skip to content

Commit

Permalink
Merge pull request #50 from seqeralabs/k8s-deployment-updates
Browse files Browse the repository at this point in the history 
Kubernetes deployment page updates
  • Loading branch information
@llewellyn-sl
llewellyn-sl authored May 16, 2024
2 parents 74c1311 + fb1274f commit 6b77246
Show file tree
Hide file tree
Showing 2 changed files with 172 additions and 216 deletions.
198 changes: 88 additions & 110 deletions platform_versioned_docs/version-23.3.0/enterprise/kubernetes.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,39 +5,34 @@ date: "21 Apr 2023"
tags: [kubernetes, deployment]
---

This guide assumes that all prerequisites have been met. Visit the corresponding **Prerequisites** page for your infrastructure provider.

:::note
You may also use this guide for deployments to other cloud platforms. However, you must satisfy any prerequisites for those platforms. Use at your own risk.
:::
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

## Deploy Seqera Platform Enterprise on Kubernetes
This guide assumes that all prerequisites have been met. Visit the corresponding **Prerequisites** page for your infrastructure provider.

Complete the following procedures to install Seqera Enterprise on a Kubernetes cluster:
Complete the following procedures to install Seqera Platform Enterprise on a Kubernetes cluster:

### Create a namespace

Create a namespace for the Seqera resources.

1. Create the namespace (e.g., `seqera-nf`):
Create a namespace for the Seqera resources:

```bash
kubectl create namespace seqera-nf
```

2. Switch to the namespace:
Switch to the namespace:

```bash
kubectl config set-context --current --namespace=seqera-nf
```

### Configure container registry credentials

Seqera Platform Enterprise is distributed as a collection of Docker containers available through the Seqera container registry [`cr.seqera.io`](https://cr.seqera.io). Contact [support](https://support.seqera.io) to get your container access credentials. After you've received your credentials, grant your cluster access to the registry:
Seqera Enterprise is distributed as a collection of Docker containers available through the Seqera container registry [`cr.seqera.io`](https://cr.seqera.io). Contact [support](https://support.seqera.io) to get your container access credentials. After you've received your credentials, grant your cluster access to the registry:
1. Retrieve the `name` and `secret` values from the JSON file that you received from Seqera support.
2. Create a [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) with the `name` and `secret` values retrieved in the previous step:
1. Create a [secret][kubectl-secret]:
```bash
kubectl create secret docker-registry cr.seqera.io \
Expand All @@ -46,9 +41,9 @@ Seqera Platform Enterprise is distributed as a collection of Docker containers a
--docker-password='<YOUR SECRET>'
```
The credential `name` contains a dollar `$` character. To prevent the Linux shell from interpreting this value as an environment variable, you must wrap it in single quotes.
The credential `name` contains a dollar `$` character. Wrap the name in single quotes to prevent the Linux shell from interpreting this value as an environment variable.
3. Configure the Seqera cron service and the application frontend and backend to use the secret created in the previous step (see [tower-cron.yml](./_templates/k8s/tower-cron.yml) and [tower-svc.yml](./_templates/k8s/tower-svc.yml)):
1. Configure the Seqera cron service and the application frontend and backend to use the secret created in the previous step (see [tower-cron.yml](./_templates/k8s/tower-cron.yml) and [tower-svc.yml](./_templates/k8s/tower-svc.yml)):
```yaml
imagePullSecrets:
Expand All @@ -59,101 +54,101 @@ Seqera Platform Enterprise is distributed as a collection of Docker containers a
### Seqera ConfigMap
1. Download and configure a [ConfigMap](_templates/k8s/configmap.yml) as detailed on the [configuration](configuration/overview.mdx) page.
Download and configure a [ConfigMap](_templates/k8s/configmap.yml). See [Configuration](configuration/overview.mdx) for more information.
2. Deploy the ConfigMap to your cluster:
Deploy the ConfigMap to your cluster after it is configured:
```bash
kubectl apply -f configmap.yml
```
:::note
The `configmap.yml` manifest includes both the `tower.env` file and `tower.yml` file. These files are made available to the other containers through volume mounts.
The `configmap.yml` manifest includes both the `tower.env` and `tower.yml` files. These files are made available to the other containers through volume mounts.
:::
### Redis
Seqera Enterprise requires a Redis database for caching purposes. Configure Redis manually by deploying a manifest to your cluster, or configure a managed Redis service.
**Deploy a Redis manifest to your cluster**
#### Deploy a Redis manifest to your cluster
1. Download the appropriate manifest for your infrastructure:
- [Azure AKS](_templates/k8s/redis.aks.yml)
- [Amazon EKS](_templates/k8s/redis.eks.yml)
- [Azure AKS](_templates/k8s/redis.aks.yml)
- [Google Kubernetes Engine](_templates/k8s/redis.gke.yml)
2. Deploy to your cluster:
1. Deploy to your cluster:
```bash
kubectl apply -f redis.*.yml
```
**Managed Redis services**
1. To run the Redis service as a container as part of your Docker or Kubernetes installation, specify the service name as part of the `TOWER_REDIS_URL`:
```bash
TOWER_REDIS_URL=redis://redis:6379
```
Seqera supports managed Redis services such as [Amazon ElastiCache](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.html), [Azure Cache for Redis](https://learn.microsoft.com/en-gb/azure/azure-cache-for-redis/cache-overview), or [Google Memorystore](https://cloud.google.com/memorystore/docs/redis).
#### Managed Redis services
When using a managed Redis service, you must specify the service IP address or DNS name for the `TOWER_REDIS_URL` as described in the following sections.
Seqera supports managed Redis services such as [Amazon ElastiCache][aws-elasticache], [Azure Cache for Redis][azure-cache], or [Google Memorystore][memorystore].
**AWS ElastiCache requirements**
<Tabs>
<TabItem value="AWS ElastiCache" label="AWS ElastiCache" default>
- Use a single-node cluster, as multi-node clusters are not supported
- Use an instance with at least 6GB capacity ([cache.m4.large](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/CacheNodes.SupportedTypes.html) or greater)
- Specify your private ElastiCache instance in the Seqera environment variables:
- Use an instance with at least 6GB capacity ([cache.m4.large][aws-cache-instances] or greater)
- Specify your private ElastiCache instance in the Seqera [environment variables](./configuration/overview.mdx#database-and-redis-manual-configuration):
```bash
TOWER_REDIS_URL=redis://<redis private IP>:6379
```
**Azure Cache for Redis requirements**
</TabItem>
<TabItem value="Azure Cache for Redis" label="Azure Cache for Redis" default>
- Use a single-node cluster, as multi-node clusters are not supported
- Use an instance with at least 6GB capacity ([C3](https://azure.microsoft.com/en-gb/pricing/details/cache/) or greater)
- Specify your private Azure Cache for Redis instance in the Seqera environment variables:
- Use an instance with at least 6GB capacity ([C3][azure-cache-instances] or greater)
- Specify your private Azure Cache for Redis instance in the Seqera [environment variables](./configuration/overview.mdx#database-and-redis-manual-configuration):
```bash
TOWER_REDIS_URL=redis://<redis private IP>:6379
```
**Google Memorystore requirements**
</TabItem>
<TabItem value="Google Memorystore" label="Google Memorystore" default>
- Use a single-node cluster, as multi-node clusters are not supported
- Use an instance with at least 6GB capacity ([M2](https://cloud.google.com/memorystore/docs/redis/pricing#instance_pricing) or greater)
- Specify your private Memorystore instance in the Seqera environment variables:
- Use an instance with at least 6GB capacity ([M2][google-cache-instances] or greater)
- Specify your private Memorystore instance in the Seqera [environment variables](./configuration/overview.mdx#database-and-redis-manual-configuration):
```bash
TOWER_REDIS_URL=redis://<redis private IP>:6379
```
**Self-hosted Redis service**
If you run the Redis service as a container as part of your Docker or Kubernetes installation, you must specify the service name as part of the `TOWER_REDIS_URL`:
```bash
TOWER_REDIS_URL=redis://redis:6379
```
</TabItem>
</Tabs>
### Seqera cron service
1. Download the [cron service manifest](_templates/k8s/tower-cron.yml) file.
Download the [cron service manifest](_templates/k8s/tower-cron.yml) file.
2. Deploy the manifest to your cluster:
To deploy the manifest to your cluster, run the following:
```bash
kubectl apply -f tower-cron.yml
```
```bash
kubectl apply -f tower-cron.yml
```
:::caution
This container creates the required database schema the first time it instantiates. This process can take a few minutes to complete and must finish before you instantiate the Seqera backend. Ensure this container is in the `READY` state before proceeding to the next step.
:::
### Seqera frontend and backend
1. Download the [manifest](_templates/k8s/tower-svc.yml).
Download the [manifest](_templates/k8s/tower-svc.yml).
2. Deploy the manifest to your cluster:
To deploy the manifest to your cluster, run the following:
```bash
kubectl apply -f tower-svc.yml
Expand All @@ -177,7 +172,7 @@ spec:
...
containers:
- name: frontend
image: cr.seqera.io/private/nf-tower-enterprise/frontend:v23.3.0-unprivileged
image: cr.seqera.io/private/nf-tower-enterprise/frontend:v23.4.3-unprivileged
env:
- name: NGINX_LISTEN_PORT # If not defined, defaults to 8000.
value: 8000
Expand Down Expand Up @@ -205,104 +200,87 @@ If further customization of the config file is needed, mount a config map/secret

### Seqera ingress

An ingress is used to make Seqera Enterprise publicly accessible, load-balance traffic, terminate TLS, and offer name-based virtual hosting. The included ingress will create an external IP address and forward HTTP traffic to the Seqera frontend.

Download the appropriate manifest and configure it according to your infrastructure:

- [Azure AKS](_templates/k8s/ingress.aks.yml)
An ingress is used to make Seqera Enterprise publicly accessible, load-balance traffic, terminate TLS, and offer name-based virtual hosting. The included ingress manifest will create an external IP address and forward HTTP traffic to the Seqera frontend.

- [Amazon EKS](_templates/k8s/ingress.eks.yml)
Download and configure the appropriate manifest for your infrastructure:

- [Google Kubernetes Engine](_templates/k8s/ingress.gke.yml)
- [Amazon EKS](_templates/k8s/ingress.eks.yml)
- [Azure AKS](_templates/k8s/ingress.aks.yml)
- [Google Kubernetes Engine](_templates/k8s/ingress.gke.yml)

Deploy the manifest to your cluster:
To deploy the manifest to your cluster, run the following:

```bash
kubectl apply -f ingress.*.yml
```
```bash
kubectl apply -f ingress.*.yml
```

See the Kubernetes documentation on the [ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) resource for more information. If you don't need to make Seqera externally accessible, you can also use a service resource to expose a [node port](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport) or a [load balancer](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) service to make it accessible within your intranet.
See [Kubernetes ingress][k8s-ingress] for more information. If you don't need to make Seqera externally accessible, use a service resource to expose a [node port][k8s-node-port] or a [load balancer][k8s-load-balancer] service to make it accessible within your intranet.
See the relevant documentation for configuring an ingress service on each cloud provider:
See the cloud provider documentation for configuring an ingress service on each cloud provider:
- [Amazon](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/)
- [Azure](https://docs.microsoft.com/en-us/azure/application-gateway/ingress-controller-annotations)
- [Google Cloud](https://cloud.google.com/kubernetes-engine/docs/concepts/ingress)
- [Amazon][aws-configure-ingress]
- [Azure][azure-configure-ingress]
- [Google Cloud][google-configure-ingress]
### Check status
Make sure that all services are up and running:
Check that all services are up and running:
```bash
kubectl get pods
```
## Test the application
To confirm that Seqera Enterprise is properly configured, follow these steps:
1. Log in to your Seqera instance.
2. Create an **organization**.
3. Create a **workspace** within that organization.
4. Create a new **Compute environment**. See [Compute environments](../compute-envs/overview.mdx) for detailed instructions.
5. Select **Quick Launch** from the **Launchpad** tab in your workspace.
6. Enter the repository URL for the `nf-core/rnaseq` pipeline (`https://github.com/nf-core/rnaseq`).
7. In the **Config profiles** dropdown, select the `test` profile.
8. In **Pipeline parameters**, change the output directory to a location based on your compute environment:
```yaml
# save to S3 bucket
outdir: s3://<your-bucket>/results
# save to scratch directory (Kubernetes)
outdir: /scratch/results
```
### Test the application
9. Select **Launch**; you'll be redirected to the **Runs** tab for the workflow. After a few minutes, progress logs will be listed in that workflow's **Execution log** tab.
See [Test deployment](./testing.mdx).
## Optional addons
## Optional features
### Pipeline resource optimization
### Pipeline optimization
Seqera Platform offers a service that optimizes pipeline resource requests. Install the resource optimization service in your Kubernetes cluster with [this manifest](_templates/k8s/groundswell.yml).
Define a set of credentials for the resource optimization database in the `tower-groundswell-cfg` ConfigMap. This can be the same database used for Seqera, but in a different schema.
The initContainers will wait until both the Seqera and resource optimization databases are ready before starting the migration in the Seqera database and finally starting the resource optimization container.
The initContainers will wait until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database and finally starting the resource optimization container.
### Database console
The included [dbconsole.yml](_templates/k8s/dbconsole.yml) manifest can be used to deploy a simple web frontend to the Seqera database. Though not required, this can be useful for administrative purposes.
Use the [dbconsole.yml](_templates/k8s/dbconsole.yml) manifest to deploy a simple web frontend to the Seqera database. Though not required, this can be useful for administrative purposes.
1. Deploy the database console:
```bash
kubectl apply -f dbconsole.yml
```
```bash
kubectl apply -f dbconsole.yml
```
2. Enable a port-forward for the database console to your local machine:
1. Enable a port-forward for the database console to your local machine:
```bash
kubectl port-forward deployment/dbconsole 8080:8080
```
```bash
kubectl port-forward deployment/dbconsole 8080:8080
```
3. Access the database console in a web browser at `http://localhost:8080`.
1. Access the database console in a web browser at `http://localhost:8080`.
### High availability
To configure Seqera Enterprise for high availability, note that:
- The `backend` service can be run in multiple replicas
- The `frontend` service is replicable, however in most scenarios it is not necessary
- The `cron` service may only have a single instance
- The `groundswell` service may only have a single instance
[aws-cache-instances]: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/CacheNodes.SupportedTypes.html
[aws-configure-ingress]: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/
[aws-elasticache]: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.html
[azure-cache]: https://learn.microsoft.com/en-gb/azure/azure-cache-for-redis/cache-overview
[azure-cache-instances]: https://azure.microsoft.com/en-gb/pricing/details/cache/
[azure-configure-ingress]: https://docs.microsoft.com/en-us/azure/application-gateway/ingress-controller-annotations
[google-cache-instances]: https://cloud.google.com/memorystore/docs/redis/pricing#instance_pricing
[google-configure-ingress]: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress
[k8s-ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[k8s-load-balancer]: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer
[k8s-node-port]: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport
[kubectl-secret]: https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/
[memorystore]: https://cloud.google.com/memorystore/docs/redis
Loading

0 comments on commit 6b77246

Please sign in to comment.