Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: back off on cfn event pull #11624

Merged
merged 4 commits into from
Dec 20, 2022
Merged

fix: back off on cfn event pull #11624

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
73342f6
fix: back off on cfn event pull
sobolk Dec 18, 2022
3ab7b97
fix: try this
sobolk Dec 18, 2022
8443a24
fix: try this
sobolk Dec 18, 2022
e959c47
chore: pr feedback
sobolk Dec 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.base.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1306,7 +1306,7 @@ commands:
source $BASH_ENV
amplify version
retry runE2eTest
no_output_timeout: 60m
no_output_timeout: 90m
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has to go up until searchable migration is faster.
image

scan_e2e_test_artifacts:
description: 'Scan And Cleanup E2E Test Artifacts'
parameters:
Expand Down
2 changes: 2 additions & 0 deletions packages/amplify-e2e-tests/src/__tests__/transformer-migrations/searchable-migration.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ import AWSAppSyncClient, { AUTH_TYPE } from 'aws-appsync';

(global as any).fetch = require('node-fetch');

jest.setTimeout(120 * 60 * 1000); // Set timeout to 2 hour because of creating/deleting searchable instance

describe('transformer model searchable migration test', () => {
let projRoot: string;
let projectName: string;
Expand Down
24 changes: 19 additions & 5 deletions packages/amplify-provider-awscloudformation/src/aws-utils/aws-cfn.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const { printer } = require('amplify-prompts');

const CFN_MAX_CONCURRENT_REQUEST = 5;
const CFN_POLL_TIME = 5 * 1000; // 5 secs wait to check if new stacks are created by root stack
const CFN_POLL_TIME_MAX = 30 * 1000; // 30 seconds
let CFNLOG = [];
const CFN_SUCCESS_STATUS = ['UPDATE_COMPLETE', 'CREATE_COMPLETE', 'DELETE_COMPLETE', 'DELETE_SKIPPED'];

Expand Down Expand Up @@ -85,7 +86,7 @@ class CloudFormation {
}
cfnModel.waitFor(cfnCompleteStatus, cfnStackCheckParams, async (completeErr, waitForStackdata) => {
if (self.pollForEvents) {
clearInterval(self.pollForEvents);
clearTimeout(self.pollForEvents);
}

this.progressBar?.stop();
Expand Down Expand Up @@ -124,7 +125,7 @@ class CloudFormation {
Promise.reject(e);
} finally {
if (this.pollForEvents) {
clearInterval(this.pollForEvents);
clearTimeout(this.pollForEvents);
}
}
});
Expand Down Expand Up @@ -153,7 +154,20 @@ class CloudFormation {
}

readStackEvents(stackName) {
this.pollForEvents = setInterval(() => this.addToPollQueue(stackName, 3), CFN_POLL_TIME);
const self = this;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this rename?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup.
image

let delay = CFN_POLL_TIME;
let readStackEventsCalls = 0;
const invoker = () => {
self.addToPollQueue(stackName, 3);
if (delay < CFN_POLL_TIME_MAX) {
delay = Math.min(Math.pow(2, readStackEventsCalls) * CFN_POLL_TIME, CFN_POLL_TIME_MAX);
}
self.pollForEvents = setTimeout(invoker, delay);
readStackEventsCalls++;
}

// start it off
self.pollForEvents = setTimeout(invoker, delay);
}

pollStack(stackName) {
Expand Down Expand Up @@ -357,13 +371,13 @@ class CloudFormation {
const cfnCompleteStatus = 'stackUpdateComplete';
if (updateErr) {
if (self.pollForEvents) {
clearInterval(self.pollForEvents);
clearTimeout(self.pollForEvents);
}
return reject(updateErr);
}
cfnModel.waitFor(cfnCompleteStatus, cfnStackCheckParams, completeErr => {
if (self.pollForEvents) {
clearInterval(self.pollForEvents);
clearTimeout(self.pollForEvents);
}
this.progressBar?.stop();

Expand Down