Param store poc

SSG-API-Testing-Application-v2/lambda/README.md

@@ -1,50 +1,82 @@
 # Introduction
-This directory is still **WIP**
+This directory contain some samples of obtaining secrets from AWS Systems Manager Parameter Store.
 
-It will contain POC of obtaining secrets from AWS Systems Manager Parameter Store.
-
-This POC is meant to showcase how you can put your authentication secrets in AWS Systems Manager Parameter Store and retrieve them to use when calling our APIs
+This sample is meant to showcase how you can put your authentication secrets in AWS Systems Manager Parameter Store and retrieve them to use when calling our APIs.
 
 There are also samples in the `secrets-manager-samples` folder if you wish to use AWS Secrets Manager instead.
 
-# Description of files
-
-`lambda_function.py` contains the lambda function POC
-
-- Permissions given to the lambda function:
-    - AWSLambdaBasicExecutionRole (AWS managed)
-    - KmsDecrypt 
-    ```
-    {
-        "Version": "2012-10-17",
-        "Statement": [
-            {
-                "Sid": "VisualEditor0",
-                "Effect": "Allow",
-                "Action": "kms:Decrypt",
-                "Resource": "*"
-            }
-        ]
-    }
-    ```
-    - SecretsPolicy  
-    ```
-    {
-        "Version": "2012-10-17",
-        "Statement": [
-            {
-                "Sid": "VisualEditor0",
-                "Effect": "Allow",
-                "Action": [
-                    "secretsmanager:GetSecretValue"
-                ],
-                "Resource": "*"
-            }
-        ]
-    }
-    ```
-
-To import dependencies into AWS Lambda, you can add them via custom layers. The steps to do so are described [here](https://stackoverflow.com/questions/65975883/aws-lambda-python-error-runtime-importmoduleerror)
-
-The minimum dependancy required to run the lambda function provided is:
-- requests 
+# Explaination
+
+## Files
+
+`deploy-secrets.yml` is the workflow file for github actions to **DEPLOY** the secret to AWS Parameter Store.
+
+`remove-secrets.yml` is the workflow file for github actions to **REMOVE** the secret to AWS Parameter Store.
+> [!NOTE]
+> While `remove-secrets.yml` works, some investigation is still needed to determine if all the variables passed via `TF_VAR_secrets` is required.
+
+`lambda_function.py` contains the code necessary to securely retrieve secrets and interact with the View Course Run API to obtain a response.
+
+### Running the lambda_function.py
+There are some dependencies required to run this file and they are added via layers.
+
+You only need to install the `requests` package into the custom layer.
+
+To import dependencies into AWS Lambda, you can add them by following the steps described [here](https://stackoverflow.com/questions/65975883/aws-lambda-python-error-runtime-importmoduleerror)
+
+### Roles and Permissions
+
+The lambda function is given the role `SampleAppLambda` with the policies below:
+- AWSLambdaBasicExecutionRole (AWS managed)
+- StsAssumeRole (Customer inline) - to assume the following role to retrieve secrets 
+
+```
+{
+"Version": "2012-10-17","Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": "sts:AssumeRole",
+            "Resource": "arn:aws:iam::YourAccountNumber:role/SampleAppRetrieveSecret"
+        }
+    ]
+}
+```
+
+The role that the lambda function will assume when retrieving the secret from AWS Parameter Store `SampleAppRetrieveSecret` with the policies below:
+- AWSLambdaBasicExecutionRole (AWS managed) 
+- KmsDecrypt 
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": "kms:Decrypt",
+            "Resource": "*"
+        }
+    ]
+}
+```
+- SecretsPolicy  
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": "ssm:GetParametersByPath",
+            "Resource": "arn:aws:ssm:ap-southeast-1:767397936445:parameter/SampleApp/*"
+        }
+    ]
+}
+```
+
+> [!NOTE]
+> You can choose to remove the `AWSLambdaBasicExecutionRole` policy from the `SampleAppRetrieveSecret` role if logging to CloudWatch is not required. This ensures the role adheres to the principle of least privilege by only granting the permissions necessary for its task.
+
+# Additional References:
+- https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html
+- https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm/client/get_parameters_by_path.html

SSG-API-Testing-Application-v2/lambda/deploy-secrets.yml

@@ -0,0 +1,78 @@
+name: "Deploy Secrets to AWS Systems Manager Parameter Store"
+
+on: workflow_dispatch
+
+jobs:
+  deploy-secrets:
+    name: "Deploy to AWS Systems Manager Parameter Store"
+    runs-on: ubuntu-latest
+    environment: dev
+
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@v4
+
+      - name: Set up Terraform
+        uses: hashicorp/setup-terraform@v3
+
+      - name: Verify Terraform Script
+        id: create-backend-verify
+        working-directory: ./SSG-API-Testing-Application-v2/lambda/deploy/
+        run: |
+          terraform fmt
+          terraform fmt -check
+
+      - name: Initialise Backend
+        id: init-backend
+        working-directory: ./SSG-API-Testing-Application-v2/lambda/deploy/
+        run: terraform init
+
+      - name: Validate Terraform Script
+        id: create-backend-validate
+        working-directory: ./SSG-API-Testing-Application-v2/lambda/deploy/
+        run: terraform validate
+
+      # Generates an execution plan for Terraform
+      - name: Terraform Plan
+        id: plan
+        working-directory: ./SSG-API-Testing-Application-v2/lambda/deploy/
+        env: 
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_REGION: ${{ env.AWS_REGION }}
+          # this is the secret value that will be placed in aws parameter store
+          TF_VAR_secrets: |
+            {
+              "/SampleApp/example_secrets/example": {
+                "value": "${{ secrets.example }}",
+                "description": "An example secret"
+              },
+              "/SampleApp/example_secrets/another_example": {
+                "value": "${{ secrets.another_example }}",
+                "description": "Another example secret"
+              }
+            }
+        run: terraform plan
+
+      # On push to "main", build or change infrastructure according to Terraform configuration files
+      - name: Terraform Apply
+        working-directory: ./SSG-API-Testing-Application-v2/lambda/deploy/
+        id: apply
+        env: 
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_REGION: ${{ env.AWS_REGION }}
+          # this is the secret value that will be placed in aws parameter store
+          TF_VAR_secrets: |
+            {
+              "/SampleApp/example_secrets/example": {
+                "value": "${{ secrets.example }}",
+                "description": "An example secret"
+              },
+              "/SampleApp/example_secrets/another_example": {
+                "value": "${{ secrets.another_example }}",
+                "description": "Another example secret"
+              }
+            }
+        run: terraform apply -auto-approve
+        continue-on-error: true

SSG-API-Testing-Application-v2/lambda/deploy/ssm-parameter-store.tf

@@ -0,0 +1,35 @@
+# Define input variables for the secret value, 
+# env variable should have key value pairs like below
+# {
+#   "db_password": {
+#     "value": "${{ secrets.DB_PASSWORD }}",
+#     "description": "The database password for the application"
+#   },
+#   "api_key": {
+#     "value": "${{ secrets.API_KEY }}",
+#     "description": "The API key for accessing external services"
+#   }
+# }
+
+variable "secrets" {
+  description = "A map of secrets with their values and descriptions"
+  type = map(object({
+    value       = string
+    description = string
+  }))
+  sensitive = true
+}
+
+resource "aws_ssm_parameter" "secrets" {
+  count = length(keys(var.secrets))
+
+  name        = element(keys(var.secrets), count.index)
+  value       = var.secrets[element(keys(var.secrets), count.index)].value
+  description = var.secrets[element(keys(var.secrets), count.index)].description
+
+  type = "SecureString"
+}
+
+provider "aws" {
+  region = "ap-southeast-1"
+}

SSG-API-Testing-Application-v2/lambda/lambda_function.py

@@ -1,3 +1,4 @@
+# This code is meant to run in AWS Lambda and will retrieve secrets from AWS Systems Manager Parameter Store.
 # Use this code snippet in your app.
 # If you need more information about configurations
 # or implementing the sample code, visit the AWS docs:
@@ -11,7 +12,20 @@
 import boto3
 from botocore.exceptions import ClientError
 
-course_run_id = "35423"
+# the path where the items are stored
+secret_path = "/SampleApp/testing/"
+# path to the certificate parameter
+cert_path = "/SampleApp/testing/cert"
+# path to the key parameter
+key_path = "/SampleApp/testing/key"
+
+# the role with the permissions to obtain and decrypt secrets
+role_arn = "arn:aws:iam::767397936445:role/SampleAppRetrieveSecret"
+region_name = "ap-southeast-1"
+
+# parameters that the view courses api call requires
+course_run_id = "340121"
+
 endpoint = f"https://uat-api.ssg-wsg.sg/courses/courseRuns/id/{course_run_id}"
 params = {"includeExpiredCourses": "true"}
 header = {
@@ -20,45 +34,49 @@
 
 
 def lambda_handler(event, context):
-    secrets = json.loads(get_secret())
-    cert_pem = create_temp_file(secrets["cert"])
-    key_pem = create_temp_file(secrets["key"])
+    secrets = get_secret()
+
+    cert_value = extract_secret(secrets,cert_path)
+    key_value = extract_secret(secrets,key_path)    
+
+    cert_pem = create_temp_file(cert_value)
+    key_pem = create_temp_file(key_value)
     response = view_course_run(cert_pem, key_pem)
-    print(response)
+    return(response.content)
 
 
 def get_secret():
-
-    secret_name = "SampleApp/testing"
-    region_name = "ap-southeast-1"
-
+    ''' 
+    returns a list of parameters found by query 
+    search for your secret using the value stored in the "Name" and "Value" key 
+    see full response syntax at https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameters.html#API_GetParameters_ResponseSyntax 
+    '''
     # Create a Secrets Manager client
     retrieve_secrets_session = assume_role()
-    session = boto3.session.Session()
-    client = retrieve_secrets_session.client(
-        service_name='secretsmanager',
+    ssm = retrieve_secrets_session.client(
+        service_name='ssm',
         region_name=region_name
     )
-
+    
     try:
-        get_secret_value_response = client.get_secret_value(
-            SecretId=secret_name
+        # query the parameters under the same path
+        response = ssm.get_parameters_by_path(
+            Path=secret_path,
+            WithDecryption=True,
         )
     except ClientError as e:
         # For a list of exceptions thrown, see
-        # https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
+        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm/client/get_parameters_by_path.html
         raise e
 
-    secret = get_secret_value_response['SecretString']
-
-    # Your code goes here.
-    return secret
+    return response['Parameters']
 
 
 def assume_role():
+    ''' returns a session of the temporary role to obtain the secret '''
     sts_client = boto3.client('sts')
     response = sts_client.assume_role(
-        RoleArn="arn:aws:iam::767397936445:role/SampleAppRetrieveSecret",
+        RoleArn=role_arn,
         RoleSessionName="retrieve-secret-session"
     )
 
@@ -69,6 +87,21 @@ def assume_role():
     return new_session
 
 
+def extract_secret(parameters,secret_name):
+    '''
+    iterate through the list of parameters 
+    returns the secret value if exists and none if not found
+    '''
+    if parameters is None:
+        return None
+
+    for parameter in parameters:
+        if parameter["Name"] == secret_name:
+            return parameter["Value"]
+
+    return None
+
+
 def create_temp_file(inputStuff):
     ''' save input into temporary file and return file name '''
     temp_file = NamedTemporaryFile(