Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): GovCloud Layer Workflow #5261

Merged
merged 12 commits into from
Oct 11, 2024
Merged

feat(ci): GovCloud Layer Workflow #5261

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
297e2ab
feat(ci): GovCloud Layer Workflow
sthulb Sep 27, 2024
4bdc6c7
Merge branch 'develop' into feat/govcloud
leandrodamascena Sep 27, 2024
f3aba6b
add description
sthulb Sep 30, 2024
b5757a2
Add metadata and checks
sthulb Oct 1, 2024
bc86ba3
set correct arch types
sthulb Oct 1, 2024
82de2b2
remove env prefix from layer name
sthulb Oct 1, 2024
f694994
fix description
Oct 3, 2024
3ea4214
Merge branch 'develop' into feat/govcloud
sthulb Oct 3, 2024
38a95f6
verify sha after upload
Oct 3, 2024
c059153
Merge branch 'develop' into feat/govcloud
leandrodamascena Oct 3, 2024
2e3506e
Merge branch 'develop' into feat/govcloud
leandrodamascena Oct 3, 2024
9d129c3
Merge branch 'develop' into feat/govcloud
sthulb Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
203 changes: 203 additions & 0 deletions .github/workflows/layer_govcloud.yml
View file
Open in desktop
sthulb marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
# GovCloud Layer Publish
# ---
# This workflow publishes a specific layer version in an AWS account based on the environment input.
#
# Using a matrix, we pull each architecture and python version of the layer and store them as artifacts
# we upload them to each of the GovCloud AWS accounts.
#
# A number of safety checks are performed to ensure safety.

on:
workflow_dispatch:
inputs:
environment:
description: Deployment environment
type: choice
options:
- Gamma
- Prod
sthulb marked this conversation as resolved.
Show resolved Hide resolved
default: Gamma
required: true
version:
description: Layer version to duplicate
type: number
required: true
workflow_call:
inputs:
environment:
description: Deployment environment
type: string
default: Gamma
required: true
version:
description: Layer version to duplicate
type: number
required: true

name: Layer Deployment (GovCloud)
run-name: Layer Deployment (GovCloud) - ${{ inputs.environment }}

jobs:
download:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
strategy:
matrix:
layer:
- AWSLambdaPowertoolsPythonV3-python38
- AWSLambdaPowertoolsPythonV3-python39
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
arch:
- arm64
- x86_64
environment: Prod (Readonly)
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
aws-region: us-east-1
dreamorosi marked this conversation as resolved.
Show resolved Hide resolved
mask-aws-account-id: true
- name: Grab Zip
sthulb marked this conversation as resolved.
Show resolved Hide resolved
run: |
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} --query 'Content.Location' | xargs curl -L -o ${{ matrix.layer }}_${{ matrix.arch }}.zip
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} > ${{ matrix.layer }}_${{ matrix.arch }}.json
- name: Store Zip
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.zip
path: ${{ matrix.layer }}_${{ matrix.arch }}.zip
retention-days: 1
if-no-files-found: error
- name: Store Metadata
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.json
path: ${{ matrix.layer }}_${{ matrix.arch }}.json
retention-days: 1
if-no-files-found: error

copy_east:
name: Copy (East)
needs: download
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
strategy:
matrix:
layer:
- AWSLambdaPowertoolsPythonV3-python38
- AWSLambdaPowertoolsPythonV3-python39
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
arch:
- arm64
- x86_64
environment: GovCloud ${{ inputs.environment }} (East)
steps:
- name: Download Zip
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.zip
sthulb marked this conversation as resolved.
Show resolved Hide resolved
- name: Download Metadata
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.json
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' ${{ matrix.layer }}_${{ matrix.arch }}.json)
test $(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64) == $SHA && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
aws-region: us-gov-east-1
mask-aws-account-id: true
- name: Create Layer
run: |
aws --region us-gov-east-1 lambda publish-layer-version \
--layer-name ${{ matrix.layer }}-${{ matrix.arch }} \
--zip-file fileb://./${{ matrix.layer }}_${{ matrix.arch }}.zip \
--compatible-runtimes $(jq -r ".CompatibleRuntimes[0]" ${{ matrix.layer }}_${{ matrix.arch }}.json) \
--compatible-architectures $(jq -r ".CompatibleArchitectures[0]" ${{ matrix.layer }}_${{ matrix.arch }}.json) \
--license-info "MIT-0" \
--description "$(jq -r '.Description' ${{ matrix.layer }}_${{ matrix.arch }}.json)" \
--query 'Version' | \
xargs aws --region us-gov-east-1 lambda add-layer-version-permission \
--layer-name ${{ matrix.layer }}-${{ matrix.arch }} \
--statement-id 'PublicLayer' \
--action lambda:GetLayerVersion \
--principal '*' \
--version-number
- name: Verify Layer
run: |
REMOTE_SHA=$(aws --region us-gov-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-gov-east-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' ${{ matrix.layer }}_${{ matrix.arch }}.json)
test $($REMOTE_SHA == $SHA && echo "SHA OK: ${SHA}" || exit 1

copy_west:
dreamorosi marked this conversation as resolved.
Show resolved Hide resolved
name: Copy (West)
needs: download
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
strategy:
matrix:
layer:
- AWSLambdaPowertoolsPythonV3-python38
- AWSLambdaPowertoolsPythonV3-python39
- AWSLambdaPowertoolsPythonV3-python310
- AWSLambdaPowertoolsPythonV3-python311
- AWSLambdaPowertoolsPythonV3-python312
arch:
- arm64
- x86_64
environment:
name: GovCloud ${{ inputs.environment }} (West)
steps:
- name: Download Zip
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.zip
- name: Download Metadata
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.json
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' ${{ matrix.layer }}_${{ matrix.arch }}.json)
test $(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64) == $SHA && echo "SHA OK: ${SHA}" || exit 1
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
aws-region: us-gov-west-1
mask-aws-account-id: true
- name: Create Layer
run: |
aws --region us-gov-west-1 lambda publish-layer-version \
--layer-name ${{ matrix.layer }}-${{ matrix.arch }} \
--zip-file fileb://./${{ matrix.layer }}_${{ matrix.arch }}.zip \
--compatible-runtimes $(jq -r ".CompatibleRuntimes[0]" ${{ matrix.layer }}_${{ matrix.arch }}.json) \
--compatible-architectures $(jq -r ".CompatibleArchitectures[0]" ${{ matrix.layer }}_${{ matrix.arch }}.json) \
--license-info "MIT-0" \
--description "$(jq -r '.Description' ${{ matrix.layer }}_${{ matrix.arch }}.json)" \
--query 'Version' | \
xargs aws --region us-gov-west-1 lambda add-layer-version-permission \
--layer-name ${{ matrix.layer }}-${{ matrix.arch }} \
--statement-id 'PublicLayer' \
--action lambda:GetLayerVersion \
--principal '*' \
--version-number
- name: Verify Layer
run: |
REMOTE_SHA=$(aws --region us-gov-west-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-gov-west-1:${{ secrets.AWS_ACCOUNT_ID }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} --query 'Content.CodeSha256' --output text)
SHA=$(jq -r '.Content.CodeSha256' ${{ matrix.layer }}_${{ matrix.arch }}.json)
test $($REMOTE_SHA == $SHA && echo "SHA OK: ${SHA}" || exit 1