Allow service account annotations to set the AWS_REGION/AWS_DEFAULT_REGION

[archoversight]

What would you like to be added:

I would like to be able to add an annotation to the service account that specifies the AWS_REGION/AWS_DEFAULT_REGION so that I can use the same OIDC provider while assuming roles using a web hook identity file across partitions.

It can however and probably should check the ARN and automatically set up the appropriate AWS_REGION/AWS_DEFAULT_REGION based upon the ARN partition identifier (so that there is a sane default, and the user can use the annotation to override it if necessary):

• aws-us-gov: US govCloud AWS regions

• aws: AWS Commercial regions

• aws-cn: AWS China regions

In this case, defaulting to the primary region would be fine for me (us-gov-east-1, us-east-1, etc...) but with an override so that if you are using the role with resources in another region you can do so easily.

Why is this needed:

More specifically, I am deploying in govCloud, however govCloud still does not have public Route53 so when I want to update records in Route53 I need to use an AWS commercial account.

This works now, and I can use the OIDC provider setup, and a role in commercial and use the IRSA to inject the AWS_WEB_IDENTITY_TOKEN_FILE, however by default it injects:

AWS_DEFAULT_REGION=us-gov-west-1
AWS_REGION=us-gov-west-1

As well. This off course fails:

# aws sts get-caller-identity

An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: No OpenIDConnect provider found in your account for https://oidc.eks.us-gov-west-1.amazonaws.com/id/0F1216F44E25B48EC173C6B7309C7B14

However as soon as I setup the right variables:

# export AWS_REGION=us-east-1
# export AWS_DEFAULT_REGION=us-east-1
# aws sts get-caller-identity
{
    "UserId": "AROA33ONKA6TKBXV5VLSG:botocore-session-1648585171",
    "Account": "[accountid]",
    "Arn": "arn:aws:sts::[accountid]:assumed-role/manage-route53--test.example.com/botocore-session-1648585171"
}

[archoversight]

As a work-around, set the environment variables explicitly in your pod spec, and the mutating web hook will not override them:

apiVersion: v1
kind: Pod
metadata:
  name: admin-23bf8e77
  namespace: default
spec:
  containers:
    - name: admin
      image: ubuntu:20.04
      env:
        - name: AWS_REGION
          value: us-east-1
        - name: AWS_DEFAULT_REGION
          value: us-east-1
      imagePullPolicy: IfNotPresent
      stdin: true
      stdinOnce: true
      tty: true
  restartPolicy: Never
  serviceAccountName: my-service-account

[vumdao]

I also face this issue and it took my huge time to investigate why