Skip to content
This repository has been archived by the owner on Nov 7, 2019. It is now read-only.

Commit

Permalink
- Renamed k8s resources to reflect new project name
Browse files Browse the repository at this point in the history
- Renamed all resources to "example-*" for consistency
- Modified readme so that the step by step getting started is a little easier to
follow
- Added example gif

Signed-off-by: Alexander Tanton <tantonat@amazon.com>
  • Loading branch information
Tanton committed Sep 18, 2018
1 parent 0067efc commit e3d1bd6
Show file tree
Hide file tree
Showing 10 changed files with 98 additions and 99 deletions.
Binary file added aws-service-operator-example.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 19 additions & 23 deletions configs/aws-operator.yaml → configs/aws-service-operator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ items:
- kind: Namespace
apiVersion: v1
metadata:
name: aws-operator
name: aws-service-operator

- kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: aws-operator
name: aws-service-operator
rules:
- apiGroups:
- ""
Expand Down Expand Up @@ -56,47 +56,43 @@ items:
- kind: ServiceAccount
apiVersion: v1
metadata:
name: aws-operator
namespace: aws-operator
name: aws-service-operator
namespace: aws-service-operator

- kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: aws-operator
name: aws-service-operator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: aws-operator
name: aws-service-operator
subjects:
- kind: ServiceAccount
name: aws-operator
namespace: aws-operator
name: aws-service-operator
namespace: aws-service-operator

- kind: Deployment
apiVersion: apps/v1beta1
metadata:
name: aws-operator
namespace: aws-operator
name: aws-service-operator
namespace: aws-service-operator
spec:
replicas: 1
template:
metadata:
annotations:
iam.amazonaws.com/role: arn:aws:iam::<ACCOUNT_ID>:role/aws-service-operator
labels:
app: aws-operator
app: aws-service-operator
spec:
serviceAccountName: aws-operator
serviceAccountName: aws-service-operator
containers:
- name: aws-operator
image: christopherhein/test-operator:v0.0.1-alpha4
- name: aws-service-operator
image: awsserviceoperator/aws-service-operator:v0.0.1-alpha4
imagePullPolicy: Always
env:
- name: AWS_ACCESS_KEY_ID
value:
- name: AWS_SECRET_ACCESS_KEY
value:
args:
- server
- --cluster-name=<cluster-name>
- --region=<region>
- --bucket=<bucket-name>
- --account-id=<account-id>
- --cluster-name=<CLUSTER_NAME>
- --region=<REGION>
- --account-id=<ACCOUNT_ID>
2 changes: 1 addition & 1 deletion examples/dynamodb.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: operator.aws/v1alpha1
kind: DynamoDB
metadata:
name: dynamodb-table
name: example-table-name
spec:
hashAttribute:
name: user_id
Expand Down
2 changes: 1 addition & 1 deletion examples/ecrrepository.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
apiVersion: operator.aws/v1alpha1
kind: ECRRepository
metadata:
name: aws-operator-codegen
name: example-repository-name
2 changes: 1 addition & 1 deletion examples/s3bucket.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: operator.aws/v1alpha1
kind: S3Bucket
metadata:
name: s3bucket.aws-operator.com
name: Private
spec:
versioning: true
accessControl: PublicRead
Expand Down
10 changes: 5 additions & 5 deletions examples/snssqssub.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
apiVersion: operator.aws/v1alpha1
kind: SNSTopic
metadata:
name: chrishein-sns-topic-3
name: example-sns-topic

---
apiVersion: operator.aws/v1alpha1
kind: SQSQueue
metadata:
name: chrishein-test-sqs-3
name: example-sqs-queue-name
spec:
contentBasedDeduplication: true
delaySeconds: 5
Expand All @@ -21,8 +21,8 @@ spec:
apiVersion: operator.aws/v1alpha1
kind: SNSSubscription
metadata:
name: chrishein-sns-subscription-3
name: example-subscription-name
spec:
topicName: chrishein-sns-topic-3
topicName: example-sns-topic
protocol: sqs
endpoint: chrishein-test-sqs-3
endpoint: example-sqs-queue-name
6 changes: 3 additions & 3 deletions examples/snssubscription.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
apiVersion: operator.aws/v1alpha1
kind: SNSSubscription
metadata:
name: chrishein-sns-subscription-2
name: example-sns-subscription-name
spec:
topicName: chrishein-sns-topic-2
topicName: example-sns-topic-name
protocol: sqs
endpoint: chrishein-test-sqs-2
endpoint: example-sqs-queue-name
2 changes: 1 addition & 1 deletion examples/snstopic.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: operator.aws/v1alpha1
kind: SNSTopic
metadata:
name: chrishein-sns-topic-2
name: example-sns-topic-name
spec:

2 changes: 1 addition & 1 deletion examples/sqsqueue.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: operator.aws/v1alpha1
kind: SQSQueue
metadata:
name: chrishein-test-sqs-2
name: example-sqs-queue-name
spec:
contentBasedDeduplication: true
delaySeconds: 5
Expand Down
129 changes: 66 additions & 63 deletions readme.adoc
Original file line number Diff line number Diff line change
@@ -1,89 +1,92 @@
= AWS Operator
= AWS Service Operator

The AWS Operator allows you to create custom AWS specific resources using
Kubernetes CRDs or Custom Resource Definitions. These help with defining your
applications including all the necessary components such as Amazon RDS
databases, Amazon ElasticCache resources, Amazon SQS queues and many more.
The AWS Service Operator allows you to manage AWS resources using
Kubernetes Custom Resource Definitions.

Using the AWS Operator allows your to use processes like `gitops` to keep your
clusters in a specified state with the internal control loop managing the
lifecycle of those components.
Using the AWS Service Operator enables a `gitops` workflow to drive your infrastructure to the desired state leveraging Kubernetes Custom Resource Definitions (CRD), the Kubernetes internal control loop, and AWS cloudformation orchestration. Read more about "operators" link:https://coreos.com/operators/[here].

To make this all possible we merge together Kubernetes CRDs with an operator
which interacts with Amazon Cloudformation to keep the AWS resources in-sync
with the control loop.
image::aws-service-operator-example.gif[]

// TODO: Add demo screen capture
== Prerequisites

To get started you will need

- a Kubernetes cluster running in AWS. Check out link:https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html[EKS] or link:https://github.com/kubernetes/kops[kops] to get started
- link:https://kubernetes.io/docs/tasks/tools/install-kubectl/[kubectl]
- link:https://docs.aws.amazon.com/cli/latest/userguide/installing.html[awscli]

== Getting Started

First thing that you will need to do is install an Pod to IAM management layer
such as `kube2iam`. This will allow you to create an AWS IAM role that gives
access to create resources. For an example policy check the
`examples/iam-policy.json`.
Make sure your Kubernetes cluster is up and running and you've configured your awscli for the approriate account and region you'll be working in.

=== IAM permissions management

You will need to install an IAM management layer
such as `kube2iam`. This will allow you to use an AWS IAM role to manage a pod's
access to AWS resources.

To get started with `kube2iam` go [here](https://github.com/jtblin/kube2iam)
To get started with `kube2iam` go link:https://github.com/jtblin/kube2iam[here] or check out the
link:https://github.com/helm/charts/tree/master/stable/kube2iam[helm chart]

After you have installed `kube2iam` we need to do is deploy the `aws-operator`
this runs as a pod in your Kubernetes cluster and listen for new CRD's of the
`aws` type and creates the resource and allocates a kubernetes `Service` to
connect to it.
The `aws-service-operator` runs as a pod in your Kubernetes cluster and listens for new `aws` type CRDs.
When a new CRD is created the operator will create the resource in AWS via cloudformation and
create a Kubernetes `Service` for access within the cluster.

You'll want to download the operator file like so
=== Create an IAM role for the `aws-service-operator`

The `K8S_WORKER_NODE_IAM_ROLE` is the IAM role assigned to your kubernetes worker instances.

[source,shell]
----
wget https://raw.githubusercontent.com/christopherhein/aws-operator/master/configs/aws-operator.yaml
----
aws cloudformation create-stack \
--stack-name aws-service-operator-role \
--capabilities CAPABILITY_NAMED_IAM \
--template-body file://configs/aws-service-operator-role.yaml \
--parameters \
ParameterKey=WorkerArn,ParameterValue=<K8S_WORKER_NODE_IAM_ROLE>

Then edit the file and replace `{{POD-ARN}}` with the pod ARN you created for
`kube2iam`
Your resulting IAM role arn should look something like `arn:aws:iam::<ACCOUNT_ID>:role/aws-service-operator`

Last install the manifest with the pod ARN specified.
=== Deploy the aws-service-operator

[source,shell]
----
# Install RBAC
kubectl apply -f aws-operator.yaml
----
Before applying these resources make sure to replace the following placeholders with the approriate information in `configs/aws-service-operator.yaml`

To test this create a file like this
- `<ACCOUNT_ID>` - Your AWS Account ID
- `<REGION>` - The AWS Region you're working in
- `<CLUSTER_NAME>` - The name of your cluster
- `<BUCKET_NAME>` - (optional) The operator stores certain things in s3 create a bucket or provide an existing bucket for the operator to use `i.e. aws s3 mb s3://foobar`

[source,yaml]
----
# s3bucket.yml
apiVersion: operator.aws/v1alpha1
kind: S3Bucket
metadata:
name: some-name-for-your-bucket
spec:
bucketName: some-name-for-your-bucket
versioning: false
logging:
enabled: true
prefix: "archive"
tags:
- key: service
value: kube
----

Then install like you would any other manifest file.

.1. Create the operator
[source,shell]
kubectl apply -f configs/aws-service-operator.yaml

.2. Create the cloudformation templates (cft) used by the operator
[source,shell]
----
kubectl apply -f s3bucket.yml
----
kubectl apply -f examples/cloudformationtemplates

This will communicate directly with Cloudformation to create the S3 bucket using
.3. (optional) Follow the operator logs
[source,shell]
kubectl logs -f -n aws-service-operator deploy/aws-service-operator

.4. Create an ecr repository with the operator
[source,yaml]
kubectl apply -f examples/ecrrepository.yaml

The operator will communicate directly with Cloudformation to create the ecr repository using
the parameters you have passed in. If you'd like to see the progress you can
view the status directly via `kubectl`.

.5. Check the ecr resource in Kubernetes
[source,shell]
----
kubectl get s3buckets <bucket-name> -w
// TODO: add output.
----
kubectl describe ecr example-repository-name

To learn more about the other resources please visit the documentation at
link:/docs/readme.adoc[Documentation]
== Removing everything

If you would like to tear everything down - run the following commands.

*IMPORTANT* this will not remove anything you created with the AWS cli (ecr repo for the operator itself, IAM roles etc.)

[source,shell]
kubectl delete ecr example-repository-name
kubectl get crd | grep ".operator.aws" | awk '{print $1}' | xargs kubectl delete crd
kubectl delete -f configs/aws-service-operator.yaml

0 comments on commit e3d1bd6

Please sign in to comment.