Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update readme with more details on Amazon Linux repo_upgrade race issue #909

Merged
merged 1 commit into from
Nov 12, 2023

Conversation

sparrowt
Copy link
Contributor

@sparrowt sparrowt commented Nov 8, 2023

This adds more explanatory text, and also a specific example of how to set repo_upgrade: none via #cloud-config in user data.

I spent a long time trying to modify the cloud-init config within the AMI before realising I could just set the 'User data' on the Jenkins Clouds config for the AMI, so hopefully this will help someone else too.

Also list other symptoms of the same problem when Docker is being used which results in EOF error messages - it took me quite a while to realise that this was due to Amazon Linux security patching so again hopefully this reference will help others facing the same issue.

Testing done

This is a readme only change - however I have tested that this snippet of 'User Data' in the Jenkins Clouds AMI configuration successfully results in cloud-init not running the yum upgrade command that was previously being run. I now see this step being essentially a no-op in /var/log/cloud-init.log

Nov 08 16:03:49 cloud-init[3626]: handlers.py[DEBUG]: start: modules-config/config-package-update-upgrade-install: running config-package-update-upgrade-install with frequency once-per-instance
Nov 08 16:03:49 cloud-init[3626]: util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-INSTANCEID/sem/config_package_update_upgrade_install - wb: [644] 20 bytes
Nov 08 16:03:49 cloud-init[3626]: helpers.py[DEBUG]: Running config-package-update-upgrade-install using lock (<FileLock using file '/var/lib/cloud/instances/i-INSTANCEID/sem/config_package_update_upgrade_install'>)
Nov 08 16:03:49 cloud-init[3626]: handlers.py[DEBUG]: finish: modules-config/config-package-update-upgrade-install: SUCCESS: config-package-update-upgrade-install ran successfully

whereas previously it actually did something (run yum) between "Running" and "finish" e.g. like this:

Nov 08 15:02:55 cloud-init[3600]: handlers.py[DEBUG]: start: modules-config/config-package-update-upgrade-install: running config-package-update-upgrade-install with frequency once-per-instance
Nov 08 15:02:55 cloud-init[3600]: util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-INSTANCEID/sem/config_package_update_upgrade_install - wb: [644] 20 bytes
Nov 08 15:02:55 cloud-init[3600]: helpers.py[DEBUG]: Running config-package-update-upgrade-install using lock (<FileLock using file '/var/lib/cloud/instances/i-INSTANCEID/sem/config_package_update_upgrade_install'>)
Nov 08 15:02:55 cloud-init[3600]: amazon.py[DEBUG]: Upgrade level: security
Nov 08 15:02:55 cloud-init[3600]: rhel.py[DEBUG]: Using YUM for package management
Nov 08 15:02:55 cloud-init[3600]: util.py[DEBUG]: Running command ['yum', '-t', '-y', '--exclude=kernel', '--exclude=nvidia*', '--exclude=cuda*', '--security', '--sec-severity=critical', '--sec-severity=important', 'upgrade'] with allowed return codes [0] (shell=False, capture=False)
Nov 08 15:03:18 cloud-init[3600]: handlers.py[DEBUG]: finish: modules-config/config-package-update-upgrade-install: SUCCESS: config-package-update-upgrade-install ran successfully

Submitter checklist

Edit tasklist title
Beta Give feedback Tasklist Submitter checklist, more options

Delete tasklist

Delete tasklist block?
Are you sure? All relationships in this tasklist will be removed.
  1. Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    Options
  2. Ensure that the pull request title represents the desired changelog entry
    Options
  3. Please describe what you did
    Options
  4. N/A (Link to relevant issues in GitHub or Jira)
    Options
  5. N/A (Link to relevant pull requests, esp. upstream and downstream changes)
    Options
  6. N/A (Ensure you have provided tests - that demonstrates feature works or fixes the issue)
    Options
Loading

This adds more explanatory text, and also a specific example of how to set `repo_upgrade: none`.

I spent a long time trying to modify the cloud-init config within the AMI before realising I could just set the 'User data' on the Jenkins Clouds config for the AMI, so hopefully this will help someone else too.

Also list other symptoms of the same problem when Docker is being used - it took a while to realise that this was due to Amazon Linux security patching so again hopefully this reference will help others facing the same issue.
@res0nance res0nance added the documentation Documentation update label Nov 12, 2023
@res0nance res0nance merged commit f3d852e into jenkinsci:master Nov 12, 2023
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Documentation update
Projects
None yet
2 participants