Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): upgrade aws-sdk to sdkv3 #31702

Merged
merged 52 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
d6ed395
update configuration files and package versions
TheRealAmazonKendra Oct 4, 2024
509c2bd
cli changes
TheRealAmazonKendra Oct 4, 2024
b475621
other resulting changes
TheRealAmazonKendra Oct 4, 2024
4d1c009
changes from rebase and build
TheRealAmazonKendra Oct 7, 2024
3bb1c21
fixed most integ tests - migrate and sam still issue
TheRealAmazonKendra Oct 8, 2024
28a5722
skip tests in regression testing
TheRealAmazonKendra Oct 9, 2024
c17db32
add mocks to context provider test
TheRealAmazonKendra Oct 9, 2024
4bc7513
remove unnecessary log statement
TheRealAmazonKendra Oct 15, 2024
ed49c42
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Oct 22, 2024
2f1eaa3
Adapted garbage collection to SDK v3
otaviomacedo Oct 23, 2024
e473a3d
Using the latest cdk-assets v3
otaviomacedo Oct 23, 2024
7e89697
Calling getPromise if v2 credentials coming from the plugin
otaviomacedo Oct 23, 2024
814ea9c
withAssumedRole inside the `try` block
otaviomacedo Oct 31, 2024
16b7f4f
Remove error handling wrapper
otaviomacedo Oct 31, 2024
1e918dd
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Nov 1, 2024
44c42d0
Adapted ECR garbage collection to SDK v3
otaviomacedo Nov 1, 2024
245e00b
Resolved conflicts
otaviomacedo Nov 1, 2024
0fc353a
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Nov 4, 2024
8db1220
lint
otaviomacedo Nov 4, 2024
813d275
yarn.lock
otaviomacedo Nov 4, 2024
ce8c025
Suppress warnings when running lerna ls -ap to not interfere with the…
otaviomacedo Nov 4, 2024
b76d6b1
Upgrade nx to 20.0.8
otaviomacedo Nov 5, 2024
cd8a3a7
yarn lock
otaviomacedo Nov 5, 2024
2ea928f
@nrwl -> @nx
otaviomacedo Nov 5, 2024
00650d7
rm -rf .nx
otaviomacedo Nov 5, 2024
d0222b4
Revert "rm -rf .nx"
otaviomacedo Nov 5, 2024
11ccf0c
Reset dependency changes
rix0rrr Nov 5, 2024
9f335fe
Merge remote-tracking branch 'origin/TheRealAmazonKendra/sdkv3-for-aw…
otaviomacedo Nov 5, 2024
e150880
Fixed import and THIRD_PARTY_LICENSES
otaviomacedo Nov 5, 2024
6f9d517
Roll back THIRD_PARTY_LICENSES
otaviomacedo Nov 5, 2024
4048296
THIRD_PARTY_LICENSES again
otaviomacedo Nov 5, 2024
250c6fa
THIRD_PARTY_LICENSES again
otaviomacedo Nov 5, 2024
d8d4d79
New test
otaviomacedo Nov 5, 2024
378aeca
New tests
otaviomacedo Nov 6, 2024
ec49cca
lint
otaviomacedo Nov 6, 2024
75894cd
merge main and resolve conflicts
otaviomacedo Nov 7, 2024
80edd5a
Removed unused import
otaviomacedo Nov 7, 2024
3864a0c
Updated integ test snapshots
otaviomacedo Nov 7, 2024
777984f
Using SHA256 for S3 operations
otaviomacedo Nov 7, 2024
db50140
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Nov 7, 2024
5657b85
setDefaultSTSMocks()
otaviomacedo Nov 7, 2024
d0c82a3
Missing comma
otaviomacedo Nov 7, 2024
7693785
Calling `setup.setupHotswapTests()` last in the setup of hotswap-depl…
otaviomacedo Nov 7, 2024
d725237
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Nov 8, 2024
f2ccdc9
Fixed backoff computation
otaviomacedo Nov 8, 2024
cb23d21
Undo some unnecessary formatting changes
rix0rrr Nov 8, 2024
0a1c87c
Merge branch 'TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two' of gith…
rix0rrr Nov 8, 2024
5517a11
Updated the version to skip a CLI integ test
otaviomacedo Nov 9, 2024
e6ae8b4
Merge remote-tracking branch 'origin/TheRealAmazonKendra/sdkv3-for-aw…
otaviomacedo Nov 9, 2024
de5a539
Updated the version to skip a CLI integ test
otaviomacedo Nov 9, 2024
b2383fc
Merge branch 'main' into TheRealAmazonKendra/sdkv3-for-aws-cdk-take-two
otaviomacedo Nov 11, 2024
275e6b4
SSMContextProviderPlugin tests
otaviomacedo Nov 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 40 additions & 33 deletions packages/@aws-cdk-testing/cli-integ/lib/integ-test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@ import * as fs from 'fs';
import * as path from 'path';
import { MemoryStream } from './corking';

const SKIP_TESTS = fs.readFileSync(path.join(__dirname, '..', 'skip-tests.txt'), { encoding: 'utf-8' })
const SKIP_TESTS = fs
.readFileSync(path.join(__dirname, '..', 'skip-tests.txt'), { encoding: 'utf-8' })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☹️ on all these formatting changes

.split('\n')
.map(x => x.trim())
.filter(x => x && !x.startsWith('#'));
.map((x) => x.trim())
.filter((x) => x && !x.startsWith('#'));

if (SKIP_TESTS) {
process.stderr.write(`ℹ️ Skipping tests: ${JSON.stringify(SKIP_TESTS)}\n`);
Expand Down Expand Up @@ -38,37 +39,41 @@ export function integTest(
const testKind = process.env.JEST_TEST_CONCURRENT === 'true' ? test.concurrent : test;
const runner = shouldSkip(name) ? testKind.skip : testKind;

runner(name, async () => {
const output = new MemoryStream();
runner(
name,
async () => {
const output = new MemoryStream();

output.write('================================================================\n');
output.write(`${name}\n`);
output.write('================================================================\n');
output.write('================================================================\n');
output.write(`${name}\n`);
output.write('================================================================\n');

const now = Date.now();
process.stderr.write(`[INTEG TEST::${name}] Starting (pid ${process.pid})...\n`);
try {
return await callback({
output,
randomString: randomString(),
log(s: string) {
output.write(`${s}\n`);
},
});
} catch (e: any) {
process.stderr.write(`[INTEG TEST::${name}] Failed: ${e}\n`);
output.write(e.message);
output.write(e.stack);
// Print output only if the test fails. Use 'console.log' so the output is buffered by
// jest and prints without a stack trace (if verbose: false).
// eslint-disable-next-line no-console
console.log(output.buffer().toString());
throw e;
} finally {
const duration = Date.now() - now;
process.stderr.write(`[INTEG TEST::${name}] Done (${duration} ms).\n`);
}
}, timeoutMillis);
const now = Date.now();
process.stderr.write(`[INTEG TEST::${name}] Starting (pid ${process.pid})...\n`);
try {
return await callback({
output,
randomString: randomString(),
log(s: string) {
output.write(`${s}\n`);
},
});
} catch (e: any) {
process.stderr.write(`[INTEG TEST::${name}] Failed: ${e}\n`);
output.write(e.message);
output.write(e.stack);
// Print output only if the test fails. Use 'console.log' so the output is buffered by
// jest and prints without a stack trace (if verbose: false).
// eslint-disable-next-line no-console
console.log(output.buffer().toString());
throw e;
} finally {
const duration = Date.now() - now;
process.stderr.write(`[INTEG TEST::${name}] Done (${duration} ms).\n`);
}
},
timeoutMillis,
);
}

function shouldSkip(testName: string) {
Expand All @@ -77,5 +82,7 @@ function shouldSkip(testName: string) {

export function randomString() {
// Crazy
return Math.random().toString(36).replace(/[^a-z0-9]+/g, '');
return Math.random()
.toString(36)
.replace(/[^a-z0-9]+/g, '');
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The output of this test is changed by the sdk upgrade. The type and content of the error have changed from sdkv2 to sdkv3. We now receive more specific information about the error type.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Skipping test due to the incompatibility in error output between sdkv2 and sdkv3
hotswap deployment for ecs service detects failed deployment and errors
Original file line number Diff line number Diff line change
Expand Up @@ -2160,11 +2160,12 @@ integTest(
const functionName = response.Stacks?.[0].Outputs?.[0].OutputValue;

// THEN

// The deployment should not trigger a full deployment, thus the stack's status must remains
// "CREATE_COMPLETE"
expect(response.Stacks?.[0].StackStatus).toEqual('CREATE_COMPLETE');
expect(deployOutput).toContain(`Lambda Function '${functionName}' hotswapped!`);
// The entire string fails locally due to formatting. Making this test less specific
expect(deployOutput).toMatch(/hotswapped!/);
expect(deployOutput).toContain(functionName);
}),
);

Expand Down Expand Up @@ -2205,7 +2206,9 @@ integTest(
// The deployment should not trigger a full deployment, thus the stack's status must remains
// "CREATE_COMPLETE"
expect(response.Stacks?.[0].StackStatus).toEqual('CREATE_COMPLETE');
expect(deployOutput).toContain(`Lambda Function '${functionName}' hotswapped!`);
// The entire string fails locally due to formatting. Making this test less specific
expect(deployOutput).toMatch(/hotswapped!/);
expect(deployOutput).toContain(functionName);
} finally {
// Ensure cleanup in reverse order due to use of import/export
await fixture.cdkDestroy('lambda-hotswap');
Expand Down Expand Up @@ -2244,7 +2247,9 @@ integTest(
// The deployment should not trigger a full deployment, thus the stack's status must remains
// "CREATE_COMPLETE"
expect(response.Stacks?.[0].StackStatus).toEqual('CREATE_COMPLETE');
expect(deployOutput).toContain(`ECS Service '${serviceName}' hotswapped!`);
// The entire string fails locally due to formatting. Making this test less specific
expect(deployOutput).toMatch(/hotswapped!/);
expect(deployOutput).toContain(serviceName);
}),
);

Expand All @@ -2257,7 +2262,7 @@ integTest(
});

// WHEN
await fixture.cdkDeploy('ecs-hotswap', {
const deployOutput = await fixture.cdkDeploy('ecs-hotswap', {
options: ['--hotswap'],
modEnv: {
DYNAMIC_ECS_PROPERTY_VALUE: 'new value',
Expand All @@ -2283,14 +2288,15 @@ integTest(
}),
);
expect(describeServicesResponse.services?.[0].deployments).toHaveLength(1); // only one deployment present
expect(deployOutput).toMatch(/hotswapped!/);
}),
);

integTest(
'hotswap deployment for ecs service detects failed deployment and errors',
withExtendedTimeoutFixture(async (fixture) => {
// GIVEN
await fixture.cdkDeploy('ecs-hotswap');
await fixture.cdkDeploy('ecs-hotswap', { verbose: true });

// WHEN
const deployOutput = await fixture.cdkDeploy('ecs-hotswap', {
Expand All @@ -2299,10 +2305,11 @@ integTest(
USE_INVALID_ECS_HOTSWAP_IMAGE: 'true',
},
allowErrExit: true,
verbose: true,
});

const expectedSubstring = 'Resource is not in the state deploymentCompleted';

// THEN
const expectedSubstring = 'Resource is not in the expected state due to waiter status: TIMEOUT';
expect(deployOutput).toContain(expectedSubstring);
expect(deployOutput).not.toContain('hotswapped!');
}),
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -616,7 +616,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "d77fcb7e4497ea7e1720518ba452504bdbe1a6a6de3a766745440129c8397e9e.zip"
"S3Key": "03b88dabbf9959e20788fc81687646e0ed6a152c994cfe69b0cd04169b3412b1.zip"
},
"Handler": "index.onEventHandler",
"Role": {
Expand Down Expand Up @@ -735,7 +735,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "d77fcb7e4497ea7e1720518ba452504bdbe1a6a6de3a766745440129c8397e9e.zip"
"S3Key": "03b88dabbf9959e20788fc81687646e0ed6a152c994cfe69b0cd04169b3412b1.zip"
},
"Handler": "index.isCompleteHandler",
"Role": {
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ async function tryGetClusterSnapshotStatus(identifier: string): Promise<string |
});
return data.DBClusterSnapshots?.[0].Status;
} catch (err: any) {
if (err.code === 'DBClusterSnapshotNotFoundFault') {
if (err.name === 'DBClusterSnapshotNotFoundFault') {
return undefined;
}
throw err;
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading