Securely access AWS Services from GKE cluster

Ever wanted to access AWS services from Google Kubernetes cluster (GKE) without using AWS IAM credentials?

This solution can help you to get and exchange Google OIDC token for temporary AWS IAM security credentials are generated by AWS STS service. This approach allows you to access AWS services form a GKE cluster without pre-generated long-living AWS credentials.

Read more about this solution on DoiT Securely Access AWS Services from Google Kubernetes Engine (GKE) blog post.

gtoken tool

The gtoken tool can get Google Cloud ID token when running with under GCP Service Account (for example, GKE Pod with Workload Identity).

gtoken command syntax

NAME:
   gtoken - generate ID token with current Google Cloud service account

USAGE:
   gtoken [global options] command [command options] [arguments...]

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --refresh      auto refresh ID token before it expires (default: true)
   --file value   write ID token into file (stdout, if not specified)
   --help, -h     show help (default: false)
   --version, -v  print the version

gtoken-webhook Kubernetes webhook

The gtoken-webhook is a Kubernetes mutating admission webhook, that mutates any K8s Pod running under specially annotated Kubernetes Service Account (see details below).

gtoken-webhook mutation

The gtoken-webhook injects a gtoken initContainer into a target Pod and an additional gtoken sidekick container (to refresh an ID OIDC token a moment before expiration), mounts token volume and injects three AWS-specific environment variables. The gtoken container generates a valid GCP OIDC ID Token and writes it to the token volume.

Injected AWS environment variables:

• AWS_WEB_IDENTITY_TOKEN_FILE - the path to the web identity token file (OIDC ID token)
• AWS_ROLE_ARN - the ARN of the role to assume by Pod containers
• AWS_ROLE_SESSION_NAME - the name applied to this assume-role session

The AWS SDK will automatically make the corresponding AssumeRoleWithWebIdentity calls to AWS STS on your behalf. It will handle in memory caching as well as refreshing credentials as needed.

skip injection

The gtoken-webhook can be configured to skip injection for all Pods in the specific Namespace by adding the admission.gtoken/ignore label to the Namespace.

gtoken-webhook deployment

1. Create a new gtoken namespace:

kubectl create -f deployment/namespace.yaml

1. To deploy the `gtoken-webhook` server, we need to create a webhook service and a deployment in our Kubernetes cluster. It’s pretty straightforward, except one thing, which is the server’s TLS configuration. If you’d care to examine the [deployment.yaml](https://github.com/doitintl/gtoken/blob/master/deployment/deployment.yaml) file, you’ll find that the certificate and corresponding private key files are read from command line arguments, and that the path to these files comes from a volume mount that points to a Kubernetes secret:

```yaml
[...]
      args:
      [...]
      - --tls-cert-file=/etc/webhook/certs/cert.pem
      - --tls-private-key-file=/etc/webhook/certs/key.pem
      volumeMounts:
      - name: webhook-certs
        mountPath: /etc/webhook/certs
        readOnly: true
[...]
   volumes:
   - name: webhook-certs
     secret:
       secretName: gtoken-webhook-certs

The most important thing to remember is to set the corresponding CA certificate later in the webhook configuration, so the apiserver will know that it should be accepted. For now, we’ll reuse the script originally written by the Istio team to generate a certificate signing request. Then we’ll send the request to the Kubernetes API, fetch the certificate, and create the required secret from the result.

First, run webhook-create-signed-cert.sh script and check if the secret holding the certificate and key has been created:

./deployment/webhook-create-signed-cert.sh

creating certs in tmpdir /var/folders/vl/gxsw2kf13jsf7s8xrqzcybb00000gp/T/tmp.xsatrckI71
Generating RSA private key, 2048 bit long modulus
.........................+++
....................+++
e is 65537 (0x10001)
certificatesigningrequest.certificates.k8s.io/gtoken-webhook-svc.