Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schedule for the re-enqueue lambda #80

Merged
merged 2 commits into from
Apr 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions elastio-s3-changelog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,21 @@ Then, the Elastio `iscan` job reads those events to perform the scanning of new

1. First, you need to enable Amazon EventBridge for your S3 buckets by following these instructions:
[Enabling Amazon EventBridge](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-event-notifications-eventbridge.html).
2. You can use
[this quick-create link](https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
to create the stack.

> You might need to switch to the region where your Elastio CloudFormation stack is deployed.

> **Important!** You can change the stack name, but it **MUST** start with `elastio-`. Otherwise, Elastio won't be able to access the created resources.
2. Use one of the following quick-create links. Choose the region where your Elastio Cloud Connector is deployed.

**Important!** You can change the stack name, but it **MUST** start with `elastio-`. Otherwise, Elastio won't be able to access the created resources.

* [us-east-1](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-east-2](https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-1](https://us-west-1.console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-2](https://us-west-2.console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-central-1](https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-1](https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-2](https://eu-west-2.console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-3](https://eu-west-3.console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ca-central-1](https://ca-central-1.console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-south-1](https://ap-south-1.console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-southeast-2](https://ap-southeast-2.console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v1/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)

3. Fill in the main parameters:
* *BucketNames* - comma-separated list of S3 bucket names;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Parameters:
Default: 'false'
QrtsBatchSize:
Type: Number
Default: 0
MinValue: 0
QrtsMaxDelay:
Type: Number
MinValue: 60
Expand Down
22 changes: 22 additions & 0 deletions elastio-s3-changelog/cloudformation-single-bucket.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,26 @@ Resources:
Tags:
'elastio:resource': 'true'

queueRefreshSchedule:
Type: AWS::Scheduler::Schedule
Properties:
Description: A schedule to invoke the periodic changelog queue refresh lambda
ScheduleExpression: cron(0/15 * * * ? *)
FlexibleTimeWindow:
Mode: 'OFF'
State: ENABLED
Target:
Arn: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:elastio-s3-changelog-handler-refresh'
RoleArn: !Sub 'arn:aws:iam::${AWS::AccountId}:role/ElastioS3ChangelogSchedulerExecution'
Input: !Sub >
{
"queue_url": "${queue}",
Veetaha marked this conversation as resolved.
Show resolved Hide resolved
"queue_name": "${queue.QueueName}"
}
RetryPolicy:
MaximumEventAgeInSeconds: 60
MaximumRetryAttempts: 3

queueDepthAlarm:
Type: AWS::CloudWatch::Alarm
Condition: EnableQrts
Expand Down Expand Up @@ -284,3 +304,5 @@ Outputs:
Value: !GetAtt dlq.Arn
templateVersion:
Value: "{{VERSION}}"
templatePatch:
Value: 1