Skip to content

Commit

Permalink
Merge pull request eksctl-io#239 from leakingtapan/readme
Browse files Browse the repository at this point in the history
Update driver README
  • Loading branch information
Cheng Pan authored Mar 15, 2019
2 parents 0c6b853 + 207c65e commit a33d6c7
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 101 deletions.
170 changes: 77 additions & 93 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,29 @@
[![Build Status](https://travis-ci.org/kubernetes-sigs/aws-ebs-csi-driver.svg?branch=master)](https://travis-ci.org/kubernetes-sigs/aws-ebs-csi-driver)
[![Coverage Status](https://coveralls.io/repos/github/kubernetes-sigs/aws-ebs-csi-driver/badge.svg?branch=master)](https://coveralls.io/github/kubernetes-sigs/aws-ebs-csi-driver?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes-sigs/aws-ebs-csi-driver)](https://goreportcard.com/report/github.com/kubernetes-sigs/aws-ebs-csi-driver)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgit.luolix.top%2Fd-nishi%2Faws-ebs-csi-driver.svg?type=shield)](https://app.fossa.io/projects/git%2Bgit.luolix.top%2Fd-nishi%2Faws-ebs-csi-driver?ref=badge_shield)

**WARNING**: This driver is in ALPHA currently. This means that there may potentially be backwards compatible breaking changes moving forward. Do NOT use this driver in a production environment in its current state.
**WARNING**: This driver is currently in Beta release and should not be used in performance critical applications.

**DISCLAIMER**: This is not an officially supported Amazon product

# Amazon Elastic Block Store CSI driver
# Amazon Elastic Block Store (EBS) CSI driver

## Overview

The [Amazon Elastic Block Store](https://aws.amazon.com/ebs/) Container Storage Interface (CSI) Driver provides a [CSI](https://github.com/container-storage-interface/spec/blob/master/spec.md) interface used by Container Orchestrators to manage the lifecycle of Amazon EBS volumes.

This driver is in alpha stage. Basic volume operations that are functional include CreateVolume/DeleteVolume, ControllerPublishVolume/ControllerUnpublishVolume, NodeStageVolume/NodeUnstageVolume, NodePublishVolume/NodeUnpublishVolume and [Volume Scheduling](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode).

## Container Images:

|AWS EBS CSI Driver Version | Image |
|---------------------------|-------------------------------------|
|v0.1.0 |amazon/aws-ebs-csi-driver:0.1.0-alpha|
|v0.2.0 |amazon/aws-ebs-csi-driver:0.2.0 |
|master branch |amazon/aws-ebs-csi-driver:latest |

## CSI Specification Compability Matrix
| AWS EBS CSI Driver \ CSI Version | v0.3.0| v1.0.0 |
|----------------------------------------|-------|--------|
| v0.1.0 | yes | no |
| v0.2.0 | no | yes |
| master branch | no | yes |

## Kubernetes Version Compability Matrix
| AWS EBS CSI Driver \ Kubernetes Version| v1.12 | v1.13 |
|----------------------------------------|-------|-------|
| v0.1.0 | yes | yes |
| v0.2.0 | no | yes |
| master branch | no | yes |
| v0.2.0 | no | yes |
| v0.1.0 | yes | no |

## Features
### Capabilities
The list of supported driver capabilities:
* Identity Service: **CONTROLLER_SERVICE** and **ACCESSIBILITY_CONSTRAINTS**
* Controller Service: **CREATE_DELETE_VOLUME** and **PUBLISH_UNPUBLISH_VOLUME**
* Node Service: **STAGE_UNSTAGE_VOLUME**
The following CSI gRPC calls are implemented:
* **Controller Service**: CreateVolume, DeleteVolume, ControllerPublishVolume, ControllerUnpublishVolume, ControllerGetCapabilities, ValidateVolumeCapabilities, CreateSnapshot, DeleteSnapshot
* **Node Service**: NodeStageVolume, NodeUnstageVolume, NodePublishVolume, NodeUnpublishVolume, NodeGetCapabilities, NodeGetInfo
* **Identity Service**: GetPluginInfo, GetPluginCapabilities, Probe

### CreateVolume Parameters
There are several optional parameters that could be passed into `CreateVolumeRequest.parameters` map:
Expand All @@ -55,51 +36,75 @@ There are several optional parameters that could be passed into `CreateVolumeReq
| "encrypted" | | | Whether the volume should be encrypted or not. Valid values are "true" or "false" |
| "kmsKeyId" | | | The full ARN of the key to use when encrypting the volume. When not specified, the default KMS key is used |

## Prerequisites
### Kubernetes
1. Kubernetes 1.12+ is required. Although this driver should work with any other container orchestration system that implements the CSI specification, so far it has only been tested in Kubernetes.

2. Enable the flag `--allow-privileged=true` in the manifest entries of kubelet and kube-apiserver.

3. Add `--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,CSIBlockVolume=true,VolumeSnapshotDataSource=true` in the manifest entries for `kube-apiserver` and `--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,CSIBlockVolume=true` in the `kubelets`. This is required to enable topology support of EBS volumes in Kubernetes, using `volumeMode: Block` and restoring volumes from snapshots.
# EBS CSI Driver on Kubernetes
Following sections are Kubernetes specific. If you are Kubernetes user, use followings for driver features, installation steps and examples.

4. Install the `CSINodeInfo` CRD on the cluster using the instructions provided here: [Enabling CSINodeInfo](https://kubernetes-csi.github.io/docs/csi-node-info-object.html#enabling-csinodeinfo).

5. Please refer to [kubernetes CSI docs](https://kubernetes-csi.github.io/docs/) for general CSI driver setup instructions on kubernetes.
## Kubernetes Version Compability Matrix
| AWS EBS CSI Driver \ Kubernetes Version| v1.12 | v1.13 |
|----------------------------------------|-------|-------|
| master branch | no | yes |
| v0.2.0 | no | yes |
| v0.1.0 | yes | yes |

## Setup
### Kubernetes
1. Use the manifest files under the directory [deploy/kubernetes](../deploy/kubernetes), needed to deploy the CSI driver and sidecar containers.
## Container Images:
|AWS EBS CSI Driver Version | Image |
|---------------------------|-------------------------------------|
|master branch |amazon/aws-ebs-csi-driver:latest |
|v0.2.0 |amazon/aws-ebs-csi-driver:0.2.0 |
|v0.1.0 |amazon/aws-ebs-csi-driver:0.1.0-alpha|

2. The driver can use the EC2 instance roles, otherwise add AWS credentials of the IAM user to the [deploy/kubernetes/secret.yaml](../deploy/kubernetes/secret.yaml) file.
## Features
* **Static Provisioning** - create a new or migrating existing EBS volumes, then create persistence volume (PV) from the EBS volume and consume the PV from container using persistence volume claim (PVC).
* **Dynamic Provisioning** - uses persistence volume claim (PVC) to request the Kuberenetes to create the EBS volume on behalf of user and consumes the volume from inside container.
* **Mount Option** - mount options could be specified in persistence volume (PV) to define how the volume should be mounted.
* **Block Volume** - consumes the EBS volume as a raw block device for latency sensitive application eg. MySql
* **Volume Snapshot** - creating volume snapshots and restore volume from snapshot.
* **NVMe** - consume NVMe EBS volume from EC2 [Nitro instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#ec2-nitro-instances).

## Prerequisites
* Get yourself familiar with how to setup Kubernetes on AWS
* If you are managing EBS volumes using static provisioning, get yourself familiar with [EBS volume](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html).
* Have a working Kubernetes cluster:
* Enable flag `--allow-privileged=true` for `kubelet` and `kube-apiserver`
* Enable `kube-apiserver` feature gates `--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,CSIBlockVolume=true,VolumeSnapshotDataSource=true`
* Enable `kubelet` feature gates `--feature-gates=CSINodeInfo=true,CSIDriverRegistry=true,CSIBlockVolume=true`

## Installation
Checkout the project:
```sh
git clone https://github.com/kubernetes-sigs/aws-ebs-csi-driver.git
cd aws-ebs-csi-driver
```
apiVersion: v1
kind: Secret
metadata:
name: aws-secret
stringData:
key_id: [aws_access_key_id]
access_key: [aws_secret_access_key]
```

3. Apply the secret using `kubectl apply -f deploy/kubernetes/secret.yaml` if required.

4. Grant only required permissions to the CSI driver. Use this sample [IAM policy](example-iam-policy.json) and add it to the worker nodes in the cluster.

5. Deploy the csi-provisioner, csi-attacher and csi-node manifests to the cluster in one step:
Edit the [secret manifest](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/deploy/kubernetes/secret.yaml) using your favorite text editor. The secret should have [enough IAM permission](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/example-iam-policy.json) to create EBS volume. Then deploy the secret:
```sh
kubectl apply -f deploy/kubernetes/secret.yaml
```

If your cluster is v1.14+, you can skip this step. Install the `CSINodeInfo` CRD on the cluster:
```sh
kubectl create -f https://raw.githubusercontent.com/kubernetes/csi-api/release-1.13/pkg/crd/manifests/csinodeinfo.yaml
```
kubectl apply -f deploy/kubernetes

Then deploy the driver:
```sh
kubectl apply -f deploy/kubernetes/controller.yaml
kubectl apply -f deploy/kubernetes/node.yaml
```

Now any user can start creating and using EBS volumes with the CSI driver.
Verify driver is running:
```sh
kubectl get pods -n kube-system
```

6. Apply `examples/kubernetes/pod-single-volume` that uses the recently deployed driver:
## Examples
Make sure you follow the [Prerequisites](README.md#Prerequisites) before the examples:
* [Dynamic Provisioning](../examples/kubernetes/pod-single-volume)
* [Block Volume](../examples/kubernetes/block-volume)
* [Volume Snapshot](../examples/kubernetes/snapshot)

```
kubectl apply -f examples/kubernetes/pod-single-volume
```
## Migrating from in-tree EBS plugin
Starting from Kubernetes 1.14, CSI migration is supported as alpha feature. If you have persistence volumes that are created with in-tree `kubernetes.io/aws-ebs` plugin, you could migrate to use EBS CSI driver. To turn on the migration, set `CSIMigration` and `CSIMigrationAWS` feature gates to `true` for `kube-controller-manager` and `kubelet`.

## Development
Please go through [CSI Spec](https://github.com/container-storage-interface/spec/blob/master/spec.md) and [General CSI driver development guideline](https://kubernetes-csi.github.io/docs/Development.html) to get some basic understanding of CSI driver before you start.
Expand All @@ -109,44 +114,23 @@ Please go through [CSI Spec](https://github.com/container-storage-interface/spec
* [Ginkgo](https://github.com/onsi/ginkgo) in your PATH for integration testing and end-to-end testing
* Docker 17.05+ for releasing

### Testing

To execute all unit tests, run: `make test`

To execute sanity test run: `make test-sanity`

**Notes**:
* Sanity tests make sure that the driver complies with the CSI specification

To execute integration tests, run: `make test-integration`

**Notes**:
* EC2 instance is required to run integration test, since it is exercising the actual flow of creating EBS volume, attaching it and read/write on the disk.
* See [Ingetration Testing](../tests/integration/README.md) for more details.
### Dependency
Dependencies are managed through go module. To build the project, first turn on go mod using `export GO111MODULE=on`, then build the project using: `make`

To execute e2e tests, run:

```
make test-e2e-single-az // executes single az test suite
make test-e2e-multi-az // executes multi az test suite
```
### Testing
* To execute all unit tests, run: `make test`
* To execute sanity test run: `make test-sanity`
* To execute integration tests, run: `make test-integration`
* To execute e2e tests, run: `make test-e2e-single-az` and `make test-e2e-multi-az`

**Notes**:
* See [E2E Testing](../tests/e2e/README.md) for more details.
* Sanity tests make sure the driver complies with the CSI specification
* EC2 instance is required to run integration test, since it is exercising the actual flow of creating EBS volume, attaching it and read/write on the disk. See [Ingetration Testing](../tests/integration/README.md) for more details.
* E22 tests exercises various driver functionalities in Kubernetes cluster. See [E2E Testing](../tests/e2e/README.md) for more details.

### Build and Publish Container Image

Build image and push it with latest tag:

```
make image && make push
```

Build image and push it with release tag:

```
make image-release && make push-release
```
* Build image and push it with latest tag: `make image && make push`
* Build image and push it with release tag: `make image-release && make push-release`

## Milestone
[Milestones page](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/milestones)
16 changes: 8 additions & 8 deletions examples/kubernetes/snapshot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ kubectl describe pv
```

4. Validate the pod successfully wrote data to the volume, taking note of the timestamp of the first entry:
```
kubectl exec -it app cat /data/out.txt
```
```
kubectl exec -it app cat /data/out.txt
```

5. Create a `VolumeSnapshot` referencing the `PersistentVolumeClaim` name:
```
Expand Down Expand Up @@ -60,8 +60,8 @@ kubectl exec -it app cat /data/out.txt
```

10. Cleanup resources:
```
kubectl delete -f specs/snapshot-restore
kubectl delete -f specs/snapshot
kubectl delete -f specs/classes
```
```
kubectl delete -f specs/snapshot-restore
kubectl delete -f specs/snapshot
kubectl delete -f specs/classes
```

0 comments on commit a33d6c7

Please sign in to comment.