Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dynamodb): global table L2 #26563

Merged
merged 120 commits into from
Aug 29, 2023
Merged
Show file tree
Hide file tree
Changes from 118 commits
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
e8326b9
initial global table code structure
colifran Jul 14, 2023
f87fc2a
initial work on BillingMode and Capacity classes
colifran Jul 14, 2023
cd69914
implementation for Capacity and BillingMode classes
colifran Jul 15, 2023
3e826ec
code refactor and TableEncryption implementation
colifran Jul 15, 2023
6c66654
added global table attributes to IGlobalTable, GlobalTableBase, and G…
colifran Jul 15, 2023
3207515
moved addReplica, replica, addGlobalSecondaryIndex, addLocalSecondary…
colifran Jul 15, 2023
c339bb9
added enums for CapacityType, BillingModeType, and TableEncryptionType
colifran Jul 15, 2023
9273cdc
initial structure for global table test suite
colifran Jul 15, 2023
a58538f
refactor in progress
colifran Jul 16, 2023
11933b9
completed refactor and added initial logic to configure write provisi…
colifran Jul 16, 2023
f02f2e5
naming clean-up
colifran Jul 16, 2023
b067b39
refactor
colifran Jul 17, 2023
893c945
refactor
colifran Jul 17, 2023
e16e615
refactor
colifran Jul 17, 2023
d0467c4
finished refactor and added implementation details for configuring gl…
colifran Jul 17, 2023
c0ccb3b
added todo note
colifran Jul 18, 2023
2fca8cc
added code for configuring local secondary indexes
colifran Jul 24, 2023
d576b1e
Merge branch 'main' into colifran/global-table
colifran Jul 24, 2023
54dd0c5
reorganized code, moved shared code into common shared file, added in…
colifran Jul 25, 2023
dd2e8e1
added getters with error handling for optional capacity and billing a…
colifran Jul 25, 2023
d094547
Merge branch 'main' into colifran/global-table
colifran Jul 26, 2023
79fe29e
replica table configuration code
colifran Jul 26, 2023
6b98115
replica table configuration code
colifran Jul 26, 2023
9ec699b
wip
colifran Jul 26, 2023
bfc18d8
refactored logic for adding replicas, gsi, and lsi
colifran Jul 26, 2023
095d286
refactored logic for lazily generating gsi and lsi and added private …
colifran Jul 26, 2023
03fe4b4
code for configuring gsi on replicas
colifran Jul 27, 2023
855233d
table property unit tests
colifran Jul 27, 2023
e8fdb31
some cleanup and code organization
colifran Jul 28, 2023
81e8dd9
updated replica gsi options
colifran Jul 28, 2023
ea97764
code to configure replicas, gsi, lsi added with table
colifran Jul 28, 2023
b2172ed
updated logic for generating gsi on replicas
colifran Jul 28, 2023
9fcbdf1
updated unit test
colifran Jul 28, 2023
b96cda7
secondary index unit tests
colifran Jul 28, 2023
1de7113
small bug fix and additional unit tests
colifran Jul 28, 2023
30fbf23
Merge branch 'main' into colifran/global-table
colifran Aug 1, 2023
ae1bfb8
unit tests for secondary indexes
colifran Jul 31, 2023
437b9b7
secondary index unit tests
colifran Jul 31, 2023
bdb554c
table encryption and encryption unit tests
colifran Aug 1, 2023
acfc2fa
Merge branch 'main' into colifran/global-table
colifran Aug 1, 2023
55e45fa
encryption unit tests
colifran Aug 1, 2023
185d62d
billing and capacity unit tests
colifran Aug 1, 2023
a1740f5
added docstrings to public methods and interface properties
colifran Aug 1, 2023
48fdf47
Merge branch 'main' into colifran/global-table
colifran Aug 5, 2023
4316419
Merge branch 'main' into colifran/global-table
colifran Aug 8, 2023
cf0e5a2
refactor to support replica method on global table base class
colifran Aug 2, 2023
4fbc350
replica table unit tests
colifran Aug 2, 2023
6db4b0e
replica table configuration unit tests
colifran Aug 3, 2023
721a110
reverted change to billing mode
colifran Aug 3, 2023
119f08a
unit test for billing mode
colifran Aug 3, 2023
4d4fe99
grants and metrics, refactored table encryption class, replica method
colifran Aug 4, 2023
baedfc5
refactored billing, capacity, and table encryption
colifran Aug 5, 2023
c25843f
replica method
colifran Aug 5, 2023
6e8f370
docstrings for global table attributes
colifran Aug 5, 2023
5cc3f2f
replica and IGlobalTable implementation
colifran Aug 6, 2023
efb586c
refactoring and code cleanup
colifran Aug 8, 2023
ba82e23
updated unit tests to match updated error messages
colifran Aug 8, 2023
2c09a74
updated encryption class to pass repliceKeyArns
colifran Aug 8, 2023
93d6758
unit tests
colifran Aug 8, 2023
1a0791a
removed _renderReplicaKeys from previous implementation
colifran Aug 8, 2023
9c25cf7
further simplification of replica gsi options config method
colifran Aug 8, 2023
a6c5de2
Merge branch 'main' into colifran/global-table
colifran Aug 8, 2023
c85f5fb
stream view type only set when replica outside of deployment region i…
colifran Aug 8, 2023
a95d4c9
updated stream arn logic for grants
colifran Aug 8, 2023
75ffd68
Merge branch 'main' into colifran/global-table
colifran Aug 10, 2023
744fff7
updated naming of some methods and added more unit tests
colifran Aug 10, 2023
8bc55a0
unit tests
colifran Aug 10, 2023
2e510a6
Merge branch 'main' into colifran/global-table
colifran Aug 11, 2023
fcba6cb
grant unit tests
colifran Aug 10, 2023
8968c81
grant unit tests and added test files and tests for capacity, encrypt…
colifran Aug 11, 2023
9c0217f
Merge branch 'main' into colifran/global-table
colifran Aug 14, 2023
bd13edc
updated encryption tests for table key and replica key arns to valida…
colifran Aug 11, 2023
390e763
separated out global table base added more grant unit tests
colifran Aug 14, 2023
af2fafd
Merge branch 'main' into colifran/global-table
colifran Aug 16, 2023
503a270
added IGlobalTable to support tableId and added unit tests for import…
colifran Aug 15, 2023
a39ecf1
updated metric implementation and added metric unit tests
colifran Aug 16, 2023
5a17fb6
unit tests for metrics
colifran Aug 16, 2023
ead6d41
Merge branch 'main' into colifran/global-table
colifran Aug 16, 2023
f1f0ccf
added code to validate that local indexes can only be added when the …
colifran Aug 16, 2023
edd96e2
Merge branch 'main' into colifran/global-table
colifran Aug 17, 2023
dcecc3f
updated mode in capacity and billing to be CapacityMode and BillingMo…
colifran Aug 17, 2023
77eb26e
updated grants for global tables
colifran Aug 17, 2023
39fe59e
added validation to capacity constructor for autoscaled capacity
colifran Aug 17, 2023
510741a
updated docstrings for grants
colifran Aug 17, 2023
62af22b
default min capacity
colifran Aug 17, 2023
cb677a2
Merge branch 'main' into colifran/global-table
colifran Aug 18, 2023
f7b4533
adr for grants
colifran Aug 18, 2023
3bf823f
Merge branch 'main' into colifran/global-table
colifran Aug 23, 2023
8079ce1
removed aws no value and updated integ tests. added env from arn to i…
colifran Aug 21, 2023
be146a7
updated gsi logic to inherit read capacity from global table just lik…
colifran Aug 21, 2023
28d6376
unit test for gsi inheriting read capacity from global table
colifran Aug 21, 2023
d26c73b
initial code for global table integ test
colifran Aug 21, 2023
1ae2b1d
global table integ tests and snapshots
colifran Aug 22, 2023
d8e81ea
updated global table in integ test to have destroy as removal policy
colifran Aug 22, 2023
0f5099c
updated replica integ test and snapshots
colifran Aug 23, 2023
4b214cf
readme updates for global table
colifran Aug 23, 2023
96c1114
finished readme for global table
colifran Aug 23, 2023
3b1dd20
re-running build
colifran Aug 23, 2023
3c0223e
Merge branch 'main' into colifran/global-table
colifran Aug 23, 2023
b3c1688
readme updates
colifran Aug 23, 2023
98eac2f
more readme updates
colifran Aug 23, 2023
2d88365
Merge branch 'main' into colifran/global-table
colifran Aug 23, 2023
e645d11
changed grantRead to grantReadData in readme
colifran Aug 23, 2023
ffb4eb0
fix for kinesis streams - they should not be inherited from the table…
colifran Aug 23, 2023
0d09f48
updates to unit tests and readme for kinesis streams
colifran Aug 23, 2023
f2acf17
added kinesis stream to integ test and updated snapshots
colifran Aug 23, 2023
c757425
updated replica integ test to correctly assert for 200 status respons…
colifran Aug 23, 2023
9db758b
minor readme update
colifran Aug 23, 2023
67ee1e4
Merge branch 'main' into colifran/global-table
colifran Aug 24, 2023
da9a83c
added tagging functionality, updated readme, added unit tests for tag…
colifran Aug 24, 2023
2656e2b
changed back unit test after making changes
colifran Aug 24, 2023
7cd8d36
Merge branch 'main' into colifran/global-table
colifran Aug 25, 2023
a83c953
readme updates, removed tagging, updated adr
colifran Aug 25, 2023
e365c29
updated README and updated dynamo streams
colifran Aug 25, 2023
72a7f1c
removed new and old images constant - replaced with new and old image…
colifran Aug 25, 2023
8a097b8
fixed readme rosetta issues
colifran Aug 25, 2023
0de5d47
fixed readme rosetta issues
colifran Aug 25, 2023
8be052e
Merge branch 'main' into colifran/global-table
colifran Aug 29, 2023
534222a
readme updates
colifran Aug 29, 2023
5264ab7
Merge branch 'main' into colifran/global-table
mergify[bot] Aug 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"version": "33.0.0",
"files": {
"7605c78d81246ac4a63b646e0f39ae3e697626ced7d652a031ea3f9dba4c613e": {
"source": {
"path": "asset.7605c78d81246ac4a63b646e0f39ae3e697626ced7d652a031ea3f9dba4c613e",
"packaging": "zip"
},
"destinations": {
"current_account-us-east-1": {
"bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-us-east-1",
"objectKey": "7605c78d81246ac4a63b646e0f39ae3e697626ced7d652a031ea3f9dba4c613e.zip",
"region": "us-east-1",
"assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-us-east-1"
}
}
},
"ae0dd0d32332147cd7ad825eef90902d72051b1cb41575b1da051ae1432ceeb3": {
"source": {
"path": "BarStack.template.json",
"packaging": "file"
},
"destinations": {
"current_account-us-east-1": {
"bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-us-east-1",
"objectKey": "ae0dd0d32332147cd7ad825eef90902d72051b1cb41575b1da051ae1432ceeb3.json",
"region": "us-east-1",
"assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-us-east-1"
}
}
}
},
"dockerImages": {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
{
"Resources": {
"FunctionServiceRole675BB04A": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
}
}
],
"Version": "2012-10-17"
},
"ManagedPolicyArns": [
{
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
]
]
}
]
}
},
"FunctionServiceRoleDefaultPolicy2F49994A": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": [
"dynamodb:BatchWriteItem",
"dynamodb:DeleteItem",
"dynamodb:DescribeTable",
"dynamodb:PutItem",
"dynamodb:UpdateItem"
],
"Effect": "Allow",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:dynamodb:us-west-1:",
{
"Ref": "AWS::AccountId"
},
":table/",
{
"Ref": "Resource"
}
]
]
}
}
],
"Version": "2012-10-17"
},
"PolicyName": "FunctionServiceRoleDefaultPolicy2F49994A",
"Roles": [
{
"Ref": "FunctionServiceRole675BB04A"
}
]
}
},
"Function76856677": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-us-east-1"
},
"S3Key": "7605c78d81246ac4a63b646e0f39ae3e697626ced7d652a031ea3f9dba4c613e.zip"
},
"FunctionName": "global-table-lambda",
"Handler": "index.handler",
"Role": {
"Fn::GetAtt": [
"FunctionServiceRole675BB04A",
"Arn"
]
},
"Runtime": "python3.11"
},
"DependsOn": [
"FunctionServiceRoleDefaultPolicy2F49994A",
"FunctionServiceRole675BB04A"
]
},
"Resource": {
"Type": "AWS::DynamoDB::GlobalTable",
"Properties": {
"AttributeDefinitions": [
{
"AttributeName": "pk",
"AttributeType": "S"
}
],
"BillingMode": "PAY_PER_REQUEST",
"KeySchema": [
{
"AttributeName": "pk",
"KeyType": "HASH"
}
],
"Replicas": [
{
"Region": "us-west-1"
},
{
"Region": "us-east-2"
},
{
"Region": "us-east-1"
}
],
"StreamSpecification": {
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"TableName": "global-table"
},
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete"
}
},
"Parameters": {
"BootstrapVersion": {
"Type": "AWS::SSM::Parameter::Value<String>",
"Default": "/cdk-bootstrap/hnb659fds/version",
"Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
}
},
"Rules": {
"CheckBootstrapVersion": {
"Assertions": [
{
"Assert": {
"Fn::Not": [
{
"Fn::Contains": [
[
"1",
"2",
"3",
"4",
"5"
],
{
"Ref": "BootstrapVersion"
}
]
}
]
},
"AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
}
]
}
}
}
Loading