Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update cmft templates with stackrefs #276

Merged
merged 24 commits into from
Feb 2, 2023
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
544d18b
update cmft templates with stackrefs
deathtenk Jan 27, 2023
915d4d1
readded CORS config stuff
deathtenk Jan 27, 2023
0e3302e
made VPCId and DBSubnets optional parameters
deathtenk Jan 27, 2023
469608f
fixed missing comma in launch script
deathtenk Jan 27, 2023
1a9d2f8
updated DBSubnets condition with list
deathtenk Jan 27, 2023
aeaebef
fixed broken params in templates
deathtenk Jan 27, 2023
c806516
added information to VPCStackName doc string
deathtenk Jan 27, 2023
934c4dc
fixed db version to 12.9
deathtenk Jan 27, 2023
3c7f48d
removed stackrefs for VPC
deathtenk Jan 31, 2023
33bd462
removed stackrefs for vpc resources on 2_lrs
deathtenk Jan 31, 2023
f73b9d4
made DBMassterUserPasswordPath a ref
deathtenk Jan 31, 2023
2a4a068
removed unecessary instructions from docs
deathtenk Jan 31, 2023
127046c
formatting fixes
deathtenk Jan 31, 2023
6dea98f
more formatting fixes
deathtenk Jan 31, 2023
37205e0
updated doc strings
deathtenk Jan 31, 2023
867614e
added DBStackName to docs
deathtenk Feb 1, 2023
8c33026
added InstanceSubnets as own param
deathtenk Feb 1, 2023
9e1f69c
Merge branch 'main' into stackref_update
deathtenk Feb 1, 2023
f4112ab
set dbdeletion default to true
deathtenk Feb 1, 2023
c2acf07
reverted default db instance type to r4.large
deathtenk Feb 1, 2023
e0d190c
removed default DBStackName
deathtenk Feb 1, 2023
84a3fd8
updated docs on DBSubnets
deathtenk Feb 1, 2023
8c3944b
removed text fragment from cors stuff
deathtenk Feb 1, 2023
0347f18
Merge branch 'main' into stackref_update
deathtenk Feb 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 36 additions & 6 deletions dev-resources/template/1_db.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
AWSTemplateFormatVersion: "2010-09-09"
Description: "Postgres DB for SQL LRS"
Parameters:
# Networking
VPCId:
Description: "VPC on which to run SQL LRS"
Type: AWS::EC2::VPC::Id
DBSubnets:
Description: Subnets on which to run services
Description: Subnets on which the DB is running
milt marked this conversation as resolved.
Show resolved Hide resolved
Type: List<AWS::EC2::Subnet::Id>

# Postgres
Expand All @@ -28,6 +27,7 @@ Parameters:
Default: "12.9"
AllowedValues:
- "12.9"
- "14.3"
DBInstanceClass:
Description: For provisioned aurora, the instance class to use
Type: String
Expand Down Expand Up @@ -60,9 +60,18 @@ Parameters:
- false
Default: true

Mappings:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat!

DBClusterParameterGroupMap:
"12.9":
Name: default.aurora-postgresql12
"14.3":
Name: default.aurora-postgresql14

Conditions:
DBProvisioned: !Equals [!Ref DBEngineMode, "provisioned"]
DBSnapshotIdentifierProvided: !Not [!Equals [!Ref DBSnapshotIdentifier, ""]]
DBProvisioned:
!Equals [!Ref DBEngineMode, "provisioned"]
DBSnapshotIdentifierProvided:
!Not [!Equals [!Ref DBSnapshotIdentifier, ""]]

Resources:
# PostgreSQL Database
Expand All @@ -71,7 +80,8 @@ Resources:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: SQL LRS App Instance Subnet Group
SubnetIds: !Ref DBSubnets
SubnetIds:
!Ref DBSubnets

DBInstanceSG:
Type: AWS::EC2::SecurityGroup
Expand Down Expand Up @@ -108,7 +118,7 @@ Resources:
VpcSecurityGroupIds:
- !Ref DBInstanceSG
DBSubnetGroupName: !Ref DBSubnetGroup
DBClusterParameterGroupName: default.aurora-postgresql12
DBClusterParameterGroupName: !FindInMap [DBClusterParameterGroupMap, !Ref "DBEngineVersion", Name]
EnableCloudwatchLogsExports: !If
- DBProvisioned
- [postgresql]
Expand Down Expand Up @@ -145,3 +155,23 @@ Outputs:
Value: !Ref DBInstanceSG
Export:
Name: !Sub "${AWS::StackName}:DBInstanceSG"
DBName:
Description: Name of the DB
Value: !Ref DBName
Export:
Name: !Sub "${AWS::StackName}:DBName"
DBMasterUserName:
Description: DB Master username
Value: !Ref DBMasterUserName
Export:
Name: !Sub "${AWS::StackName}:DBMasterUserName"
DBMasterUserPasswordPath:
Description: Path to the DB Master password
Value: !Ref DBMasterUserPasswordPath
Export:
Name: !Sub "${AWS::StackName}:DBMasterUserPasswordPath"
DBMasterUserPasswordVersion:
Description: Path to the DB Master password
Value: !Ref DBMasterUserPasswordVersion
Export:
Name: !Sub "${AWS::StackName}:DBMasterUserPasswordVersion"
118 changes: 66 additions & 52 deletions dev-resources/template/2_lrs.yml
Original file line number Diff line number Diff line change
@@ -1,44 +1,21 @@
AWSTemplateFormatVersion: "2010-09-09"
Description: "SQL LRS and DB Init Script"
Parameters:
# Networking
VPCId:
Description: "VPC on which to run SQL LRS"
Description: VPC on which to run SQL LRS
Type: AWS::EC2::VPC::Id
DBSubnets:
Description: Subnets on which the DB is running
Type: List<AWS::EC2::Subnet::Id>
# DB Details
DBName:
Description: PG Database name. Ignored if DBSnapshotIdentifier is provided
Type: String
Default: "lrsql_demo"
DBHost:
Description: Write hostname of PG db
Type: String
DBPort:
Description: Port of PG db
Type: String
Default: 3306
DBInstanceSG:
Type: AWS::EC2::SecurityGroup::Id
Description: DB instance security group
DBMasterUserName:
Description: Name of admin user
milt marked this conversation as resolved.
Show resolved Hide resolved
DBStackName:
Description: DB Stack Reference
Type: String
Default: yetadmin
milt marked this conversation as resolved.
Show resolved Hide resolved
DBAppUserName:
Description: Name of app db user
Type: String
Default: lrsqldbuser
DBMasterUserPasswordPath:
Description: SSM Path to the secret password
Type: String
Default: "/lrsql/demo/DB_MASTER_USER_PASSWORD"
DBMasterUserPasswordVersion:
Description: SSM version
Type: Number
Default: 1
DBAppUserPasswordPath:
Description: SSM Path to the secret password
Type: String
Expand Down Expand Up @@ -75,7 +52,7 @@ Parameters:
InstanceSSHCidr:
Description: CIDR Range for SSH Access to instances (Typically VPC CIDR Range)
Type: String
Default: "173.147.0.0/16"
Default: "172.147.0.0/16"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i sorta get updating these but in general its totally deployment specific.

InstanceSubnets:
Description: Subnet on which to run the lrsql server
Type: List<AWS::EC2::Subnet::Id>
Expand Down Expand Up @@ -110,28 +87,27 @@ Parameters:
ASGCPUPolicyTargetValue:
Type: String # according to docs a Double
Description: Target CPU utilization of instances. Leave blank to disable CPU autoscaling.
Default: "" # '60.0'
Default: "" # "60.0"
ASGALBRequestCountTargetValue:
Type: String # according to docs a Double
Description: Target requests per instance. Leave blank to disable request count autoscaling.
Default: "" # '10000.0'
Default: "" # "10000.0"

# Front-end (ALB) specific settings
ALBSubnets:
Description: Subnets on which to run the ALB.
Type: List<AWS::EC2::Subnet::Id>
ALBHostName:
Type: String
Description: The hostname to use for the Elastic Load Balancer.
Default: "sqllrs.yetanalytics.io"
ALBSubnets:
Description: Subnets that are used by the Elastic Load Balancer.
Type: List<AWS::EC2::Subnet::Id>
ALBCertArn:
Type: String
Description: The ARN of an ACM cert to use on the ALB
ALBHostedZone:
Type: String
Description: Route53 Hosted Zone in which to set a DNS record. If unset no record will be updated
Default: ""

# CORS Settings
CORSAllowedOrigins:
Type: CommaDelimitedList
Expand All @@ -148,13 +124,16 @@ Parameters:
Default: "/yet/lrsql/"

Conditions:
SetDNS: !Not [!Equals [!Ref ALBHostedZone, ""]]
SetCORS: !Not [!Equals [!Join ["", !Ref CORSAllowedOrigins], ""]]
SetDNS:
!Not [!Equals [!Ref ALBHostedZone, ""]]
SetCORS:
!Not [!Equals [!Join ["", !Ref CORSAllowedOrigins], ""]]
ASGCPUPolicyTargetValueProvided:
!Not [!Equals [!Ref ASGCPUPolicyTargetValue, ""]]
ASGALBRequestCountTargetValueProvided:
!Not [!Equals [!Ref ASGALBRequestCountTargetValue, ""]]
InstanceKeyNameProvided: !Not [!Equals [!Ref InstanceKeyName, ""]]
InstanceKeyNameProvided:
!Not [!Equals [!Ref InstanceKeyName, ""]]

Resources:
# DB Initialization Function and custom resource to run it
Expand Down Expand Up @@ -184,10 +163,15 @@ Resources:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from the init fn to RDS instance
GroupId: !Ref DBInstanceSG
GroupId:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBInstanceSG"]]
IpProtocol: tcp
FromPort: !Ref DBPort
ToPort: !Ref DBPort
FromPort:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBPort"]]
ToPort:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBPort"]]
SourceSecurityGroupId: !Ref DBInitFnSG

DBInitPolicy:
Expand All @@ -205,7 +189,11 @@ Resources:
- "secretsmanager:GetSecretValue"
Resource:
- !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter${DBAppUserPasswordPath}"
- !Sub "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter${DBMasterUserPasswordPath}"
- !Sub
- "arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter${DBMasterUserPasswordPath}"
- DBMasterUserPasswordPath:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBMasterUserPasswordPath"]]
- Effect: Allow
Action:
- ec2:DescribeNetworkInterfaces
Expand Down Expand Up @@ -235,19 +223,26 @@ Resources:
DependsOn: DBInitFn
Properties:
ServiceToken: !GetAtt DBInitFn.Arn
DBMasterUsername: !Ref DBMasterUserName
DBMasterUsername:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBMasterUserName"]]
DBMasterPasswordPath: !Join
- ":"
- - !Ref DBMasterUserPasswordPath
- !Ref DBMasterUserPasswordVersion
- - Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBMasterUserPasswordPath"]]
- Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBMasterUserPasswordVersion"]]
DBUsername: !Ref DBAppUserName
DBPasswordPath: !Join
- ":"
- - !Ref DBAppUserPasswordPath
- !Ref DBAppUserPasswordVersion
DBHost: !Ref DBHost
DBPort: !Ref DBPort
DBName: !Ref DBName
DBHost:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBEndpoint"]]
DBPort:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBPort"]]
DBName:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBName"]]

# Servers
LogGroup:
Expand Down Expand Up @@ -338,10 +333,15 @@ Resources:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from the LRS instance to RDS instance
GroupId: !Ref DBInstanceSG
GroupId:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBInstanceSG"]]
IpProtocol: tcp
FromPort: !Ref DBPort
ToPort: !Ref DBPort
FromPort:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBPort"]]
ToPort:
Fn::ImportValue:
!Join [":", [!Ref "DBStackName", "DBPort"]]
SourceSecurityGroupId: !Ref InstanceSG

LrsInstances:
Expand Down Expand Up @@ -493,7 +493,7 @@ Resources:
"dbPort": ${DBPort},
"dbName": "${DBName}",
"dbUser": "${DBAppUserName}",
"dbPassword": "${InitDBCustomResource.dbAppPass}"
"dbPassword": "${DBPass}"
},
"lrs" : {
"adminUserDefault": "${DefaultAdminUser}",
Expand All @@ -506,10 +506,21 @@ Resources:
"allowedOrigins": ${AllowedOrigins}
}
}
- AllowedOrigins: !If
- DBName:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBName"]]
DBHost:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBEndpoint"]]
DBAppUserName: !Ref DBAppUserName
DBPass: !GetAtt InitDBCustomResource.dbAppPass
DefaultAdminUser: !Ref DefaultAdminUser
DefaultAdminPass: !Ref DefaultAdminPass
InstanceHttpPort: !Ref InstanceHttpPort
DBPort:
Fn::ImportValue: !Join [":", [!Ref "DBStackName", "DBPort"]]
AllowedOrigins: !If
- SetCORS
- !Sub
- '["${JoinedAllowedOrigins}"]'
- '["${JoinedAllowedOrigins}}"]'
- JoinedAllowedOrigins: !Join ['","', !Ref CORSAllowedOrigins]
- !If
- SetDNS
Expand Down Expand Up @@ -605,6 +616,7 @@ Resources:
- !GetAtt TargetGroup.TargetGroupFullName
TargetValue: !Ref ASGALBRequestCountTargetValue


# ASG
AutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Expand Down Expand Up @@ -751,3 +763,5 @@ Outputs:
- ""
- - "https://"
- !Ref ALBHostName
Export:
Name: !Sub "${AWS::StackName}:LrsAddress"
7 changes: 1 addition & 6 deletions doc/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,9 @@ This template deploys the application servers, the load balancer, and also a sma
- ALBHostedZone: (Optional) Set the Hosted Zone ID if the domain registrar is Route53 to enable automatic DNS management
- ALBSubnets: Choose the two Public Subnets from Step 2
- CORSAllowedOrigins: If you are using your own DNS and do not provide ALBHostName and ALBHostedZone above, put the HTTPS address of your LRS here, ie. `https://mydomain.com` to allow CORS requests.
- DBStackName: Choose the name of the stack deployed in Step 3
- DBAppUserName: Choose a desired database username for the application
- DBAppUserPasswordPath: Use the name selected in Systems Manager for the App Password in Step 1
- DBHost: Copy and paste the DBEndpoint Output from Step 3
- DBInstanceSG: Select the DBInstanceSG Output from Step 3
- DBMasterUserName: Must be the same value as in Step 3
- DBMasterUserPasswordPath: Must be the same value as in Step 3
- DBName: Must be the same value as in Step 3
- DBPort: 3306
milt marked this conversation as resolved.
Show resolved Hide resolved
- DBSubnets: Select the two Private Subnets from Step 2
- DefaultAdminPass: Enter a temporary seed password for the LRS Admin login (for first login).
- DefaultAdminUser: Enter initial seed username for LRS Admin Login
Expand Down