refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TFSec compliance rules compliance. #1372

rodrigobersa · 2023-01-25T03:03:32Z

What does this PR do?

Adjustments in data.aws_iam_policy_document.aws_efs_csi_driver, on modules/kubernetes-addons/aws-efs-csi-driver/data.tf to implement least privilege access policies for IAM Roles for Service Accounts (IRSA).

Motivation

Partially resolves Improve security contexts promoting TFSec integration and usage. #1256
Grant least privilege access for IRSAs
Be compliant with TFSec rules, fixing CRITICAL and HIGH impact findings.

More

Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
Yes, I have added a new example under examples to support my PR
Yes, I have created another PR for add-ons under add-ons repo (if applicable)
Yes, I have updated the docs for this feature
Yes, I ran pre-commit run -a with this PR

Note: Not all the PRs require a new example and/or doc page. In general:

Use an existing example when possible to demonstrate a new addons usage
A new docs page under docs/add-ons/* is required for new a new addon

For Moderators

E2E Test successfully complete before merge?

Additional Notes

Pre Commit & TFSec check

terraform-aws-eks-blueprints git:(2023-01-23-tfsec-kubernetes-addons-aws-efs-csi-driver) pre-commit run --file modules/kubernetes-addons/aws-efs-csi-driver/*
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check for merge conflicts................................................Passed
detect private key.......................................................Passed
detect aws credentials...................................................Passed
Terraform fmt............................................................Passed
Terraform docs...........................................................Passed
Terraform validate with tflint...........................................Passed
Terraform validate.......................................................Passed
Terraform validate with tfsec........................(no files to check)Skipped

terraform-aws-eks-blueprints git:(2023-01-23-tfsec-kubernetes-addons-aws-efs-csi-driver) tfsec modules/kubernetes-addons/aws-efs-csi-driver                  
  timings
  ──────────────────────────────────────────
  disk i/o             347.794µs
  parsing              18.427291ms
  adaptation           1.654917ms
  checks               2.865458ms
  total                23.29546ms

  counts
  ──────────────────────────────────────────
  modules downloaded   0
  modules processed    3
  blocks processed     53
  files read           13

  results
  ──────────────────────────────────────────
  passed               17
  ignored              0
  critical             0
  high                 0
  medium               0
  low                  0


No problems detected!

AWS EFS CSI Driver validation

Created EFS FileSystem
Opened port 2049/TCP
Created StorageClass
Created a Persistent Volume Claim (PVC), with dynamic provisioning.

cat <<EOF | kubectl apply -f -                                                        
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi
EOF
persistentvolumeclaim/efs-claim created

Capture logs from the container for volume creation on EFS Filesystem.

kubectl -n kube-system logs efs-csi-controller-7687f4bc7b-bcz5t -c csi-provisioner

I0120 21:42:46.695872       1 controller.go:1332] provision "default/efs-claim" class "efs-sc": started
I0120 21:42:46.697235       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"41746908-c850-4fff-9e35-a04974b56de2", APIVersion:"v1", ResourceVersion:"7156624", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/efs-claim"
I0120 21:42:46.821168       1 controller.go:838] successfully created PV pvc-41746908-c850-4fff-9e35-a04974b56de2 for PVC efs-claim and csi volume name fs-0c0f7c2cfe4f15941::fsap-0ae8b3d2d12ad9c9f
I0120 21:42:46.821209       1 controller.go:1439] provision "default/efs-claim" class "efs-sc": volume "pvc-41746908-c850-4fff-9e35-a04974b56de2" provisioned
I0120 21:42:46.821232       1 controller.go:1456] provision "default/efs-claim" class "efs-sc": succeeded
I0120 21:42:46.835713       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"41746908-c850-4fff-9e35-a04974b56de2", APIVersion:"v1", ResourceVersion:"7156624", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-41746908-c850-4fff-9e35-a04974b56de2

Validate created PVC and PV

kubectl describe pvc efs-claim 

Name:          efs-claim
Namespace:     default
StorageClass:  efs-sc
Status:        Bound
Volume:        pvc-41746908-c850-4fff-9e35-a04974b56de2
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: efs.csi.aws.com
               volume.kubernetes.io/storage-provisioner: efs.csi.aws.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      5Gi
Access Modes:  RWX
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                 Age   From                                                                                      Message
  ----    ------                 ----  ----                                                                                      -------
  Normal  ExternalProvisioning   51s   persistentvolume-controller                                                               waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
  Normal  Provisioning           51s   efs.csi.aws.com_efs-csi-controller-7687f4bc7b-bcz5t_a7045b3d-aa40-4557-91d9-dbda508367c6  External provisioner is provisioning volume for claim "default/efs-claim"
  Normal  ProvisioningSucceeded  51s   efs.csi.aws.com_efs-csi-controller-7687f4bc7b-bcz5t_a7045b3d-aa40-4557-91d9-dbda508367c6  Successfully provisioned volume pvc-41746908-c850-4fff-9e35-a04974b56de2

kubectl describe pv pvc-41746908-c850-4fff-9e35-a04974b56de2 

Name:            pvc-41746908-c850-4fff-9e35-a04974b56de2
Labels:          <none>
Annotations:     pv.kubernetes.io/provisioned-by: efs.csi.aws.com
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    efs-sc
Status:          Bound
Claim:           default/efs-claim
Reclaim Policy:  Delete
Access Modes:    RWX
VolumeMode:      Filesystem
Capacity:        5Gi
Node Affinity:   <none>
Message:         
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            efs.csi.aws.com
    FSType:            
    VolumeHandle:      fs-0c0f7c2cfe4f15941::fsap-0ae8b3d2d12ad9c9f
    ReadOnly:          false
    VolumeAttributes:      storage.kubernetes.io/csiProvisionerIdentity=1673639079972-8081-efs.csi.aws.com
Events:                <none>

Validate resource created on AWS Account

aws efs describe-file-systems
{
    "FileSystems": [
        {
            "OwnerId": "978045894046",
            "CreationToken": "6c7e4adb-f797-4129-967a-3cd50d39a8bb",
            "FileSystemId": "fs-0c0f7c2cfe4f15941",
            "FileSystemArn": "arn:aws:elasticfilesystem:us-west-2:978045894046:file-system/fs-0c0f7c2cfe4f15941",
            "CreationTime": "2023-01-12T18:24:54-05:00",
            "LifeCycleState": "available",
            "NumberOfMountTargets": 0,
            "SizeInBytes": {
                "Value": 6144,
                "Timestamp": "2023-01-20T16:35:31-05:00",
                "ValueInIA": 0,
                "ValueInStandard": 6144
            },
            "PerformanceMode": "generalPurpose",
            "Encrypted": false,
            "ThroughputMode": "bursting",
            "Tags": []
        }
    ]
}

Mounted on Test POD
Validated access inside the POD creating a test file with DD.

kubectl describe pod nginx-cd46b5fd5-8v8zn 
Name:             nginx-cd46b5fd5-8v8zn
Namespace:        default
Priority:         0
Service Account:  default
Node:             ip-10-0-12-157.us-west-2.compute.internal/10.0.12.157
Start Time:       Thu, 26 Jan 2023 19:03:08 -0500
Labels:           app=nginx
                  pod-template-hash=cd46b5fd5
Annotations:      kubernetes.io/psp: eks.privileged
Status:           Running
IP:               10.0.12.196
IPs:
  IP:           10.0.12.196
Controlled By:  ReplicaSet/nginx-cd46b5fd5
Containers:
  nginx:
    Container ID:   containerd://3a21501c8b40a9b82c77a7a47788245af0605c2d30b643611607f87e8d19fc5d
    Image:          nginx
    Image ID:       docker.io/library/nginx@sha256:b8f2383a95879e1ae064940d9a200f67a6c79e710ed82ac42263397367e7cc4e
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 26 Jan 2023 19:03:11 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /data from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mwwfd (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  efs-claim
    ReadOnly:   false
  kube-api-access-mwwfd:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  18m   default-scheduler  Successfully assigned default/nginx-cd46b5fd5-8v8zn to ip-10-0-12-157.us-west-2.compute.internal
  Normal  Pulling    18m   kubelet            Pulling image "nginx"
  Normal  Pulled     18m   kubelet            Successfully pulled image "nginx" in 554.362297ms
  Normal  Created    18m   kubelet            Created container nginx
  Normal  Started    18m   kubelet            Started container nginx

kubectl exec -ti nginx-cd46b5fd5-8v8zn -- bash
root@nginx-cd46b5fd5-8v8zn:/# df
Filesystem            1K-blocks    Used        Available Use% Mounted on
overlay                52416492 6515356         45901136  13% /
tmpfs                     65536       0            65536   0% /dev
tmpfs                   3965792       0          3965792   0% /sys/fs/cgroup
127.0.0.1:/    9007199254739968       0 9007199254739968   0% /data
/dev/nvme0n1p1         52416492 6515356         45901136  13% /etc/hosts
shm                       65536       0            65536   0% /dev/shm
tmpfs                   7241408      12          7241396   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs                   3965792       0          3965792   0% /proc/acpi
tmpfs                   3965792       0          3965792   0% /sys/firmware
root@nginx-cd46b5fd5-8v8zn:/# cd /data/
root@nginx-cd46b5fd5-8v8zn:/data# dd if=/dev/zero of=teste.dd bs=4k count=1M
1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 41.2244 s, 104 MB/s

Delete provisioned PVC

kubectl describe pvc efs-claim 
persistentvolumeclaim "efs-claim" deleted

Capture container logs for volume deletion

kubectl -n kube-system logs efs-csi-controller-7687f4bc7b-bcz5t -c csi-provisioner
I0120 21:46:57.371375       1 controller.go:1472] delete "pvc-41746908-c850-4fff-9e35-a04974b56de2": started
I0120 21:46:57.432278       1 controller.go:1487] delete "pvc-41746908-c850-4fff-9e35-a04974b56de2": volume deleted
I0120 21:46:57.443959       1 controller.go:1537] delete "pvc-41746908-c850-4fff-9e35-a04974b56de2": persistentvolume deleted
I0120 21:46:57.443984       1 controller.go:1542] delete "pvc-41746908-c850-4fff-9e35-a04974b56de2": succeeded

…ons/aws-efs-csi-driver.

…s-efs-csi-driver

…river`

…Sec compliance rules compliance. (#1372) Co-authored-by: Rodrigo Bersa <bersr@amazon.com> resolves undefined

…Sec compliance rules compliance. (aws-ia#1372) Co-authored-by: Rodrigo Bersa <bersr@amazon.com> resolves undefined

bersr-aws and others added 3 commits January 23, 2023 15:02

IAM Policy adjustments for TFSec compliance on modules/kubernetes-add…

83f715a

…ons/aws-efs-csi-driver.

Merge branch 'aws-ia:main' into 2023-01-23-tfsec-kubernetes-addons-aw…

d6aa746

…s-efs-csi-driver

pre-commit: fix end of files `modules/kubernetes-addons/aws-efs-csi-d…

88f9d2a

…river`

rodrigobersa temporarily deployed to EKS Blueprints Test January 25, 2023 03:03 — with GitHub Actions Inactive

rodrigobersa marked this pull request as ready for review January 27, 2023 00:27

rodrigobersa requested a review from a team as a code owner January 27, 2023 00:27

bryantbiggs approved these changes Jan 27, 2023

View reviewed changes

bryantbiggs merged commit c514a4a into aws-ia:main Jan 27, 2023

vara-bonthu pushed a commit that referenced this pull request Feb 2, 2023

refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TF…

7a8b20c

…Sec compliance rules compliance. (#1372) Co-authored-by: Rodrigo Bersa <bersr@amazon.com> resolves undefined

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TFSec compliance rules compliance. #1372

refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TFSec compliance rules compliance. #1372

rodrigobersa commented Jan 25, 2023 •

edited

Loading

refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TFSec compliance rules compliance. #1372

refactor: IAM Policy improvements on AWS EFS CSI Driver Add-on for TFSec compliance rules compliance. #1372

Conversation

rodrigobersa commented Jan 25, 2023 • edited Loading

What does this PR do?

Motivation

More

For Moderators

Additional Notes

Pre Commit & TFSec check

AWS EFS CSI Driver validation

rodrigobersa commented Jan 25, 2023 •

edited

Loading