Skip to content

Commit

Permalink
feat: add persistentvolumeclaim module type and volumes for containers
Browse files Browse the repository at this point in the history 
This makes it possible to mount persistent volumes in container modules,
including in tasks and tests. With the right storage classes in place,
you can even share volumes between container services, tasks and tests.
These volumes can of course also be referenced in `kubernetes` and
`helm` modules.

See the added section to the container modules guide for details on
how to use the new features.

As a small bonus, in order to be able to use the upstream
PersistentVolumeClaim schema, I've added a
`joi.customObject().jsonSchema()` type, which allows us weave JSON
schemas into our existing Joi schemas. My hope is that we can gradually
get rid of most usages of Joi schemas, at least in our plugin APIs.
  • Loading branch information
@edvald
edvald committed Feb 18, 2020
1 parent 68df10f commit 3b0a9e6
Show file tree
Hide file tree
Showing 55 changed files with 3,339 additions and 734 deletions.
1 change: 1 addition & 0 deletions docs/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
* [`kubernetes`](./module-types/kubernetes.md)
* [`maven-container`](./module-types/maven-container.md)
* [`openfaas`](./module-types/openfaas.md)
* [`persistentvolumeclaim`](./module-types/persistentvolumeclaim.md)
* [`terraform`](./module-types/terraform.md)
* [Reference](./reference/README.md)
* [Glossary](./reference/glossary.md)
Expand Down
89 changes: 89 additions & 0 deletions docs/guides/container-modules.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,3 +216,92 @@ values:
Here, we declare `my-image` as a dependency for the `my-service` Helm chart. In order for the Helm chart to be able to reference the built container image, we must provide the correct image name and version.

For a full list of keys that are available for the `container` module type, take a look at the [outputs reference](../module-types/container.md#outputs).

## Mounting volumes

`container` services, tasks and tests can all mount volumes, using _volume modules_. One such is the [`persistentvolumeclaim` module type](../module-types/persistentvolumeclaim.md), supported by the `kubernetes` provider. To mount a volume, you need to define a volume module, and reference it using the `volumes` key on your services, tasks and/or tests.

Example:

```yaml
kind: Module
name: my-volume
type: persistentvolumeclaim
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 1Gi
---
kind: Module
name: my-module
type: container
services:
- name: my-service
replicas: 1 # <- Important! Unless your volume supports ReadWriteMany, you can't run multiple replicas with it
volumes:
- name: my-volume
module: my-volume
containerPath: /volume
...
```

This will mount the `my-volume` PVC at `/volume` in the `my-service` service when it is run. The `my-volume` module creates a `PersistentVolumeClaim` resource in your project namespace, and the `spec` field is passed directly to the same field on the PVC resource.

{% hint style="warning" %}
Notice the `accessModes` field in the volume module above. The default storage classes in Kubernetes generally don't support being mounted by multiple Pods at the same time. If your volume module doesn't support the `ReadWriteMany` access mode, you must take care not to use the same volume in multiple services, tasks or tests, or multiple replicas. See [Shared volumes](#shared-volumes) below for how to share a single volume with multiple Pods.
{% endhint %}

You can do the same for tests and tasks using the [`tests.volumes`](../module-types/container.md#testsvolumes) and [`tasks.volumes`](../module-types/container.md#tasksvolumes) fields. `persistentvolumeclaim` volumes can of course also be referenced in `kubernetes` and
`helm` modules, since they are deployed as standard PersistentVolumeClaim resources.

Take a look at the [`persistentvolumeclaim` module type](../module-types/persistentvolumeclaim.md) and [`container` module](../module-types/container.md#servicesvolumes) docs for more details.

### Shared volumes

For a volume to be shared between multiple replicas, or multiple services, tasks and/or tests, it needs to be configured with a storage class (using the `storageClassName` field) that supports the `ReadWriteMany` (RWX) access mode. The available storage classes that support RWX vary by cloud providers and cluster setups, and in many cases you need to define a `StorageClass` or deploy a _storage class provisioner_ to your cluster.

You can find a list of storage options and their supported access modes [here](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes). Here are a few commonly used RWX provisioners and storage classes:

* [NFS Server Provisioner](https://github.com/helm/charts/tree/master/stable/nfs-server-provisioner)
* [Azure File](https://docs.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv)
* [AWS EFS Provisioner](https://github.com/helm/charts/tree/master/stable/efs-provisioner)
* [Ceph (via Rook)](https://rook.io/docs/rook/v1.2/ceph-filesystem.html)

Once any of those is set up you can create a `persistentvolumeclaim` module that uses the configured storage class. Here, for example, is how you might use a shared volume with a configured `azurefile` storage class:

```yaml
kind: Module
name: shared-volume
type: persistentvolumeclaim
spec:
accessModes: [ReadWriteMany]
resources:
requests:
storage: 1Gi
storageClassName: azurefile
---
kind: Module
name: my-module
type: container
services:
- name: my-service
volumes:
- &volume # <- using a YAML anchor to re-use the volume spec in tasks and tests
name: shared-volume
module: shared-volume
containerPath: /volume
...
tasks:
- name: my-task
volumes:
- *volume
...
tests:
- name: my-test
volumes:
- *volume
...
```

Here the same volume is used across a service, task and a test in the same module. You could similarly use the same volume across multiple container modules.
1 change: 1 addition & 0 deletions docs/module-types/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,5 @@ title: Module Types
* [`kubernetes`](./kubernetes.md)
* [`maven-container`](./maven-container.md)
* [`openfaas`](./openfaas.md)
* [`persistentvolumeclaim`](./persistentvolumeclaim.md)
* [`terraform`](./terraform.md)
231 changes: 229 additions & 2 deletions docs/module-types/container.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,10 @@ services:
# with hot-reloading enabled, or if the provider doesn't support multiple replicas.
replicas:

# List of volumes that should be mounted when deploying the container.
# List of volumes that should be mounted when deploying the service.
#
# Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
# deploying the container.
volumes:
- # The name of the allocated volume.
name:
Expand All @@ -307,6 +310,18 @@ services:
# module source path (or absolute).
hostPath:

# The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
# depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
# module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
#
# When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
# dependency of this service, as well as a build dependency of this module.
#
# Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
# the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
# services at the same time. Refer to the documentation of the module type in question to learn more.
module:

# A list of tests to run in the module.
tests:
- # The name of the test.
Expand Down Expand Up @@ -345,6 +360,36 @@ tests:
# `GARDEN`) and values must be primitives or references to secrets.
env: {}

# List of volumes that should be mounted when deploying the test.
#
# Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
# deploying the container.
volumes:
- # The name of the allocated volume.
name:

# The path where the volume should be mounted in the container.
containerPath:

# _NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
# and providers. Some providers may not support it at all._
#
# A local path or path on the node that's running the container, to mount in the container, relative to the
# module source path (or absolute).
hostPath:

# The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
# depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
# module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
#
# When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
# dependency of this service, as well as a build dependency of this module.
#
# Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
# the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
# services at the same time. Refer to the documentation of the module type in question to learn more.
module:

# A list of tasks that can be run from this container module. These can be used as dependencies for services (executed
# before the service is deployed) or for other tasks.
tasks:
Expand Down Expand Up @@ -398,6 +443,36 @@ tasks:
# Key/value map of environment variables. Keys must be valid POSIX environment variable names (must not start with
# `GARDEN`) and values must be primitives or references to secrets.
env: {}

# List of volumes that should be mounted when deploying the task.
#
# Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when
# deploying the container.
volumes:
- # The name of the allocated volume.
name:

# The path where the volume should be mounted in the container.
containerPath:

# _NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
# and providers. Some providers may not support it at all._
#
# A local path or path on the node that's running the container, to mount in the container, relative to the
# module source path (or absolute).
hostPath:

# The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will
# depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim
# module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.
#
# When a `module` is specified, the referenced module/volume will be automatically configured as a runtime
# dependency of this service, as well as a build dependency of this module.
#
# Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports
# the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple
# services at the same time. Refer to the documentation of the module type in question to learn more.
module:
```
## Configuration Keys
Expand Down Expand Up @@ -1191,7 +1266,9 @@ Note: This setting may be overridden or ignored in some cases. For example, when

[services](#services) > volumes

List of volumes that should be mounted when deploying the container.
List of volumes that should be mounted when deploying the service.

Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.

| Type | Default | Required |
| --------------- | ------- | -------- |
Expand Down Expand Up @@ -1239,6 +1316,20 @@ services:
- hostPath: "/some/dir"
```

### `services[].volumes[].module`

[services](#services) > [volumes](#servicesvolumes) > module

The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.

When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.

Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.

| Type | Required |
| -------- | -------- |
| `string` | No |

### `tests[]`

A list of tests to run in the module.
Expand Down Expand Up @@ -1408,6 +1499,74 @@ tests:
- {}
```

### `tests[].volumes[]`

[tests](#tests) > volumes

List of volumes that should be mounted when deploying the test.

Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.

| Type | Default | Required |
| --------------- | ------- | -------- |
| `array[object]` | `[]` | No |

### `tests[].volumes[].name`

[tests](#tests) > [volumes](#testsvolumes) > name

The name of the allocated volume.

| Type | Required |
| -------- | -------- |
| `string` | Yes |

### `tests[].volumes[].containerPath`

[tests](#tests) > [volumes](#testsvolumes) > containerPath

The path where the volume should be mounted in the container.

| Type | Required |
| ----------- | -------- |
| `posixPath` | Yes |

### `tests[].volumes[].hostPath`

[tests](#tests) > [volumes](#testsvolumes) > hostPath

_NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
and providers. Some providers may not support it at all._

A local path or path on the node that's running the container, to mount in the container, relative to the
module source path (or absolute).

| Type | Required |
| ----------- | -------- |
| `posixPath` | No |

Example:

```yaml
tests:
- volumes:
- hostPath: "/some/dir"
```

### `tests[].volumes[].module`

[tests](#tests) > [volumes](#testsvolumes) > module

The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.

When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.

Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.

| Type | Required |
| -------- | -------- |
| `string` | No |

### `tasks[]`

A list of tasks that can be run from this container module. These can be used as dependencies for services (executed before the service is deployed) or for other tasks.
Expand Down Expand Up @@ -1598,6 +1757,74 @@ tasks:
- {}
```

### `tasks[].volumes[]`

[tasks](#tasks) > volumes

List of volumes that should be mounted when deploying the task.

Note: If neither `hostPath` nor `module` is specified, an empty ephemeral volume is created and mounted when deploying the container.

| Type | Default | Required |
| --------------- | ------- | -------- |
| `array[object]` | `[]` | No |

### `tasks[].volumes[].name`

[tasks](#tasks) > [volumes](#tasksvolumes) > name

The name of the allocated volume.

| Type | Required |
| -------- | -------- |
| `string` | Yes |

### `tasks[].volumes[].containerPath`

[tasks](#tasks) > [volumes](#tasksvolumes) > containerPath

The path where the volume should be mounted in the container.

| Type | Required |
| ----------- | -------- |
| `posixPath` | Yes |

### `tasks[].volumes[].hostPath`

[tasks](#tasks) > [volumes](#tasksvolumes) > hostPath

_NOTE: Usage of hostPath is generally discouraged, since it doesn't work reliably across different platforms
and providers. Some providers may not support it at all._

A local path or path on the node that's running the container, to mount in the container, relative to the
module source path (or absolute).

| Type | Required |
| ----------- | -------- |
| `posixPath` | No |

Example:

```yaml
tasks:
- volumes:
- hostPath: "/some/dir"
```

### `tasks[].volumes[].module`

[tasks](#tasks) > [volumes](#tasksvolumes) > module

The name of a _volume module_ that should be mounted at `containerPath`. The supported module types will depend on which provider you are using. The `kubernetes` provider supports the [persistentvolumeclaim module](https://docs.garden.io/module-types/persistentvolumeclaim), for example.

When a `module` is specified, the referenced module/volume will be automatically configured as a runtime dependency of this service, as well as a build dependency of this module.

Note: Make sure to pay attention to the supported `accessModes` of the referenced volume. Unless it supports the ReadWriteMany access mode, you'll need to make sure it is not configured to be mounted by multiple services at the same time. Refer to the documentation of the module type in question to learn more.

| Type | Required |
| -------- | -------- |
| `string` | No |


## Outputs

Expand Down
Loading

0 comments on commit 3b0a9e6

Please sign in to comment.