Reformat all AWS-related README files to have a consistent style

DAGWorks-Inc · Apr 1, 2024 · 2a5d266 · 2a5d266
1 parent a2c5235
commit 2a5d266
Show file tree

Hide file tree

Showing 3 changed files with 116 additions and 50 deletions.
diff --git a/examples/aws/glue/README.md b/examples/aws/glue/README.md
@@ -1,3 +1,5 @@
+# Deploy Hamilton Functions as an AWS Glue Job
+
 [AWS Glue](https://aws.amazon.com/glue/) is a serverless data integration service. This guide demonstrates deploying a "hello-world" [processing job](https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html) using Hamilton functions on AWS Glue.
 
 ## Prerequisites
@@ -21,7 +23,9 @@ First things first, AWS Glue jobs run a single python script, but you can includ
 - **Build python wheel:**
 
     ```shell
-    cd app && python -m build --wheel --skip-dependency-check && cd ..
+    cd app \
+        && python -m build --wheel --skip-dependency-check \
+        && cd ..
     ```
 
 ### 2. Upload all necessary files to S3
@@ -31,40 +35,52 @@ First things first, AWS Glue jobs run a single python script, but you can includ
     Replace `<YOUR_PATH_TO_WHL>` with your specific S3 bucket and path:
 
     ```shell
-    aws s3 cp app/dist/hamilton_functions-0.1-py3-none-any.whl s3://<YOUR_PATH_TO_WHL>/hamilton_functions-0.1-py3-none-any.whl
+    aws s3 cp \
+        app/dist/hamilton_functions-0.1-py3-none-any.whl \
+        s3://<YOUR_PATH_TO_WHL>/hamilton_functions-0.1-py3-none-any.whl
     ```
 
 - **Upload main python script to s3:**
 
     Replace `<YOUR_PATH_TO_SCRIPT>` with your specific S3 bucket and path:
 
     ```shell
-    aws s3 cp processing.py s3://<YOUR_PATH_TO_SCRIPT>/processing.py
+    aws s3 cp \
+        processing.py \
+        s3://<YOUR_PATH_TO_SCRIPT>/processing.py
     ```
 
 - **Upload input data to s3:**
 
     Replace `<YOUR_PATH_TO_INPUT_DATA>` with your specific S3 bucket and path:
 
     ```shell
-    aws s3 cp data/input_table.csv s3://<YOUR_PATH_TO_INPUT_DATA>
+    aws s3 cp \
+        data/input_table.csv \
+        s3://<YOUR_PATH_TO_INPUT_DATA>
     ```
 
 ### 3. Create a simple role for AWS Glue job execution
 
 - **Create the Role**:
 
     ```shell
-    aws iam create-role --role-name GlueProcessorRole --assume-role-policy-document '{"Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "glue.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
+    aws iam create-role \
+        --role-name GlueProcessorRole \
+        --assume-role-policy-document '{"Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "glue.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
     ```
 
 - **Attach Policies to the Role**:
 
     Here we grant full access to S3 as an example. For production environments it's important to restrict access appropriately.
 
     ```shell
-    aws iam attach-role-policy --role-name GlueProcessorRole --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
-    aws iam attach-role-policy --role-name GlueProcessorRole --policy-arn arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole
+    aws iam attach-role-policy \
+        --role-name GlueProcessorRole \
+        --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
+    aws iam attach-role-policy \
+        --role-name GlueProcessorRole \
+        --policy-arn arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole
     ```
 
 ### 4. Create and run the job
@@ -74,15 +90,22 @@ First things first, AWS Glue jobs run a single python script, but you can includ
     Ensure all paths are correctly replaced with the actual ones:
 
     ```shell
-    aws glue create-job --name test_hamilton_script --role GlueProcessorRole --command '{"Name" :  "pythonshell", "PythonVersion": "3.9", "ScriptLocation" : "s3://<YOUR_PATH_TO_SCRIPT>/processing.py"}' --max-capacity 0.0625 --default-arguments '{"--extra-py-files" : "s3://<YOUR_PATH_TO_WHL>/hamilton_functions-0.1-py3-none-any.whl", "--additional-python-modules" : "sf-hamilton"}'
+    aws glue create-job \
+        --name test_hamilton_script \
+        --role GlueProcessorRole \
+        --command '{"Name" :  "pythonshell", "PythonVersion": "3.9", "ScriptLocation" : "s3://<YOUR_PATH_TO_SCRIPT>/processing.py"}' \
+        --max-capacity 0.0625 \
+        --default-arguments '{"--extra-py-files" : "s3://<YOUR_PATH_TO_WHL>/hamilton_functions-0.1-py3-none-any.whl", "--additional-python-modules" : "sf-hamilton"}'
     ```
 
 - **Run the job:**
 
     Ensure all paths are correctly replaced with the actual ones:
 
     ```shell
-    aws glue start-job-run --job-name test_hamilton_script --arguments '{"--input-table" : "s3://<YOUR_PATH_TO_INPUT_DATA>", "--output-table" : "s3://<YOUR_PATH_TO_OUTPUT_DATA>"}'
+    aws glue start-job-run \
+        --job-name test_hamilton_script \
+        --arguments '{"--input-table" : "s3://<YOUR_PATH_TO_INPUT_DATA>", "--output-table" : "s3://<YOUR_PATH_TO_OUTPUT_DATA>"}'
     ```
 
     Once you've run the job, you should see an output file at `s3://<YOUR_PATH_TO_OUTPUT_DATA>`.
diff --git a/examples/aws/lambda/README.md b/examples/aws/lambda/README.md
@@ -5,63 +5,92 @@
 Here we have an example how to deploy "hello-world" AWS Lambda with Hamilton functions.
 This example is based on the official instruction: https://docs.aws.amazon.com/lambda/latest/dg/python-image.html#python-image-instructions
 
-0. Set up AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html
+## Prerequisites
 
-1. Docker image build:
+- **AWS CLI Setup**: Make sure the AWS CLI is set up on your machine. If you haven't done this yet, no worries! You can follow the [Quick Start guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html) for easy setup instructions.
 
-```shell
-docker build --platform linux/amd64 -t aws-lambda-hamilton .
-```
+## Step-by-Step Guide
 
-2. Local tests:
+### 1. Build Docker image:
 
-```shell
-docker run -p 9000:8080 aws-lambda-hamilton
-```
+- **Build Docker image for deploy in AWS ECR**
 
-```shell
-curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"body": {"columns":["signups","spend"],"index":[0,1,2,3,4,5],"data":[[1,10],[10,10],[50,20],[100,40],[200,40],[400,50]]}}'
-```
+  ```shell
+  docker build --platform linux/amd64 -t aws-lambda-hamilton .
+  ```
 
-3. Create AWS ECR repository:
+- **Local tests:**
 
+  Run Docker container:
 
-```shell
-aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com
-```
+  ```shell
+  docker run -p 9000:8080 aws-lambda-hamilton
+  ```
 
-```shell
-aws ecr create-repository --repository-name aws-lambda-hamilton --region us-east-1 --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE
-```
+  Send test request to check if Docker container executes it correctly:
 
-4. Deploy image to AWS ECR:
+  ```shell
+  curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"body": {"columns":["signups","spend"],"index":[0,1,2,3,4,5],"data":[[1,10],[10,10],[50,20],[100,40],[200,40],[400,50]]}}'
+  ```
 
-```shell
-docker tag aws-lambda-hamilton 111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-lambda-hamilton:latest
-```
+### 2. Create AWS ECR repository:
+
+Ensure the AWS account number (`111122223333`) is correctly replaced with yours:
+
+- **Authenticate Docker to Amazon ECR**:
+
+    Retrieve an authentication token to authenticate your Docker client to your Amazon Elastic Container Registry (ECR):
+
+    ```shell
+    aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com
+    ```
+
+- **Create the ECR Repository**:
+
+    ```shell
+    aws ecr create-repository --repository-name aws-lambda-hamilton \
+        --region us-east-1 \
+        --image-scanning-configuration scanOnPush=true \
+        --image-tag-mutability MUTABLE
+  ```
+
+### 3. Deploy the Image to AWS ECR
+
+Ensure the AWS account number (`111122223333`) is correctly replaced with yours:
 
 ```shell
+docker tag aws-lambda-hamilton 111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-lambda-hamilton:latest
 docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-lambda-hamilton:latest
 ```
 
-4.5. Create simple AWS Lambda role (if needed):
+### 4. Create a simple AWS Lambda role:
+
+Example of creating an AWS Role for Lambda execution:
 
 ```shell
-aws iam create-role --role-name lambda-ex --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
+aws iam create-role \
+    --role-name lambda-ex \
+    --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
 ```
 
-5. Create AWS Lambda
+### 5. Create AWS Lambda
+
+Ensure the AWS account number (`111122223333`) is correctly replaced with yours:
 
 ```shell
 aws lambda create-function \
-  --function-name aws-lambda-hamilton \
-  --package-type Image \
-  --code ImageUri=111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-lambda-hamilton:latest \
-  --role arn:aws:iam::111122223333:role/lambda-ex
+    --function-name aws-lambda-hamilton \
+    --package-type Image \
+    --code ImageUri=111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-lambda-hamilton:latest \
+    --role arn:aws:iam::111122223333:role/lambda-ex
 ```
 
-6. Test AWS Lambda
+### 6. Test AWS Lambda
 
 ```shell
-aws lambda invoke --function-name aws-lambda-hamilton --cli-binary-format raw-in-base64-out --payload '{"body": {"columns":["signups","spend"],"index":[0,1,2,3,4,5],"data":[[1,10],[10,10],[50,20],[100,40],[200,40],[400,50]]}}' response.json
+aws lambda invoke \
+    --function-name aws-lambda-hamilton \
+    --cli-binary-format raw-in-base64-out \
+    --payload '{"body": {"columns":["signups","spend"],"index":[0,1,2,3,4,5],"data":[[1,10],[10,10],[50,20],[100,40],[200,40],[400,50]]}}' \
+    response.json
 ```
diff --git a/examples/aws/sagemaker/README.md b/examples/aws/sagemaker/README.md
@@ -1,4 +1,4 @@
-# Deploying Hamilton Functions as an AWS SageMaker Processing Job
+# Run Hamilton Functions as an AWS SageMaker Processing Job
 
 [AWS SageMaker](https://aws.amazon.com/sagemaker/) is a comprehensive platform that facilitates the creation, training, and deployment of machine learning (ML) models. This guide demonstrates deploying a "hello-world" [processing job](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html) using Hamilton functions on SageMaker.
 
@@ -13,12 +13,14 @@
 Navigate to the container directory and build the Docker image:
 
 ```shell
-cd container/ && docker build --platform linux/amd64 -t aws-sagemaker-hamilton . && cd ..
+cd container/ \
+    && docker build --platform linux/amd64 -t aws-sagemaker-hamilton . \
+    && cd ..
 ```
 
 ### 2. Create AWS ECR repository.
 
-Replace `111122223333` with your AWS account number.
+Ensure the AWS account number (`111122223333`) is correctly replaced with yours:
 
 - **Authenticate Docker to Amazon ECR**:
 
@@ -31,12 +33,16 @@ Replace `111122223333` with your AWS account number.
 - **Create the ECR Repository**:
 
     ```shell
-    aws ecr create-repository --repository-name aws-sagemaker-hamilton --region us-east-1 --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE
+    aws ecr create-repository \
+        --repository-name aws-sagemaker-hamilton \
+        --region us-east-1 \
+        --image-scanning-configuration scanOnPush=true \
+        --image-tag-mutability MUTABLE
     ```
 
 ### 3. Deploy the Image to AWS ECR
 
-Ensure the AWS account number is correctly replaced with yours:
+Ensure the AWS account number (`111122223333`) is correctly replaced with yours:
 
 ```shell
 docker tag aws-sagemaker-hamilton 111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-sagemaker-hamilton:latest
@@ -50,17 +56,25 @@ docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/aws-sagemaker-hamilton:
     Example of creating an AWS Role with full permissions for ECR and S3.
 
     ```shell
-    aws iam create-role --role-name SageMakerScriptProcessorRole --assume-role-policy-document '{"Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "sagemaker.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
+    aws iam create-role \
+        --role-name SageMakerScriptProcessorRole \
+        --assume-role-policy-document '{"Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "sagemaker.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
     ```
 
 - **Attach Policies to the Role**:
 
     Here we grant full access to ECR, S3 and SageMaker as an example. For production environments it's important to restrict access appropriately.
 
     ```shell
-    aws iam attach-role-policy --role-name SageMakerScriptProcessorRole --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
-    aws iam attach-role-policy --role-name SageMakerScriptProcessorRole --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess
-    aws iam attach-role-policy --role-name SageMakerScriptProcessorRole --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess
+    aws iam attach-role-policy \
+        --role-name SageMakerScriptProcessorRole \
+        --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
+    aws iam attach-role-policy \
+        --role-name SageMakerScriptProcessorRole \
+        --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess
+    aws iam attach-role-policy \
+        --role-name SageMakerScriptProcessorRole \
+        --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess
     ```
 
 ### 5. Install additional requirements