Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding logic to print failures and retry if there is an cloud-init error #28598

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions enos/modules/install_packages/scripts/synchronize-repos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -99,11 +99,44 @@ synchronize_repos() {
esac
}

# Function to check cloud-init status and retry on failure
# Before we start to modify repositories and install packages we'll wait for cloud-init to finish
# so it doesn't race with any of our package installations.
# We run as sudo becase Amazon Linux 2 throws Python 2.7 errors when running `cloud-init status` as
# We run as sudo because Amazon Linux 2 throws Python 2.7 errors when running `cloud-init status` as
# non-root user (known bug).
sudo cloud-init status --wait
check_cloud_init() {
local max_retries=1
local retry_count=0
local exit_code

while [[ $retry_count -lt $max_retries ]]; do
if sudo cloud-init status --wait; then
echo "Cloud-init completed successfully"
return 0
else
exit_code=$?
case $exit_code in
1)
echo "cloud-init did not complete successfully. Exit code: $exit_code" 1>&2
;;
2)
echo "Cloud-init completed successfully, but with errors. Exit code: $exit_code" 1>&2
exit_code=0
;;
esac
echo "There were errors when executing cloud-init. Here are the logs for the failure:"
cat /var/log/cloud-init-* | grep "Failed"
retry_count=$((retry_count + 1))
fi
done
return $exit_code
}

# Checking cloud-init
check_cloud_init
if [ $? -eq 1 ]; then
exit 1
fi

begin_time=$(date +%s)
end_time=$((begin_time + TIMEOUT_SECONDS))
Expand Down
Loading