Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(integ): capture exit codes of parallel tests #371

Merged
merged 1 commit into from
Mar 30, 2021

Conversation

jericht
Copy link
Contributor

@jericht jericht commented Mar 30, 2021

Notes

Fixed the parallel integration test code to capture the exit code of each integration test and have the script finish with exit code 0 if all tests succeeded, otherwise its exit code is 1.

Testing

  • To verify the exit code logic works correctly, I commented out any calls that deployed/destroyed CF stacks, and replaced the calls to component_e2e.sh to a script runCode.sh that returns a random exit code:
#!/bin/bash

sleep $((RANDOM % 3))
exit_code=$((RANDOM % 2))
exit $exit_code

The replaced code looked like:

# ( (../common/scripts/bash/component_e2e.sh "$COMPONENT_NAME"; exit_code=$?; echo $exit_code > "$INTEG_TEMP_DIR/${COMPONENT_NAME}_exitcode"; exit $exit_code) \
# || ../common/scripts/bash/component_e2e.sh "$COMPONENT_NAME" --destroy-only) &
( (~/tmp/runCode.sh; exit_code=$?; echo $exit_code > "$INTEG_TEMP_DIR/${COMPONENT_NAME}_exitcode"; exit $exit_code) \
|| echo "Failed") &

I ran the integ tests with RUN_TESTS_IN_PARALLEL=true and verified the script captured all exit codes and exited with the appropriate code.
Success Case Output

[jericht@dev-dsk-jericht-2a-f8c9ce54 integ]$ yarn run e2e
yarn run v1.22.10
$ ./scripts/bash/rfdk-integ-e2e.sh
Loading config...
Setting test variables...
Starting RFDK-integ end-to-end tests
Test app deadline_03_workerFleetHttp finished with exit code 0
Test app deadline_04_workerFleetHttps finished with exit code 0
Test app deadline_01_repository finished with exit code 0
Test app deadline_02_renderQueue finished with exit code 0
Complete!
Cleaning up folders...
Exiting...
Done in 3.86s.
[jericht@dev-dsk-jericht-2a-f8c9ce54 integ]$ echo $?
0

Failure Case Output

[jericht@dev-dsk-jericht-2a-f8c9ce54 integ]$ yarn run e2e
yarn run v1.22.10
$ ./scripts/bash/rfdk-integ-e2e.sh
Loading config...
Setting test variables...
Starting RFDK-integ end-to-end tests
Failed
Failed
Test app deadline_01_repository finished with exit code 1
Test app deadline_02_renderQueue finished with exit code 0
Test app deadline_03_workerFleetHttp finished with exit code 0
Test app deadline_04_workerFleetHttps finished with exit code 1
Complete!
Cleaning up folders...
Exiting...
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[jericht@dev-dsk-jericht-2a-f8c9ce54 integ]$ echo $?
1
  • Ran the script as-is in parallel mode and ensured it still works as expected

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@jusiskin jusiskin added the contribution/core This is a PR that came from AWS. label Mar 30, 2021
@ddneilson ddneilson merged commit 276f76f into aws:mainline Mar 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants