Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise an exception when AMI search is blank. #617

Merged
merged 1 commit into from
Jun 2, 2021

Conversation

bazzargh
Copy link
Contributor

Changes the behaviour when the AMI ID and search are blank from logging a warning to raising an exception. Motivated by experiencing what happens when it just warns:

We use the configuration-as-code plugin; when interpolating secrets, if you have a typo in the secret name, it inserts the empty string:

https://github.com/jenkinsci/configuration-as-code-plugin/blob/e7501727c4704aaa1e95bdc246a1454dc5b20440/plugin/src/main/java/io/jenkins/plugins/casc/SecretSourceResolver.java#L124

Unfortunately we had a typo in the name of an AMI used by a rarely-run job, which used a slightly different AMI from all the other jobs on our server. So, when we deployed, we missed the warning message:

Configuration import: Found unresolved variable '/plugins/amazonEC2/special-ami'. Will default to empty string

... then a day later, the job that used this special ami ran. Because the code warned rather than throwing an exeption, it attempted to search for an AMI with no search parameters, then picked one of the results to launch. As luck would have it, there was also a typo in our IAM rule that prevented launching public AMIs; eventually, after two days of retries, the job manage to launch one of the random AMIs: a bitcoin miner 🤦. The node was shut down seconds later when jenkins failed to connect to it, and our alarms noticed the miner, leading us to investigate where it had originated.

Hence this PR...I really don't think anyone wants to use Jenkins as a random AMI launcher :)

  • [ x ] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your master branch!
  • [ x ] Ensure that the pull request title represents the desired changelog entry
  • [ x ] Please describe what you did
  • [ x ] Link to relevant issues in GitHub or Jira
  • [ x ] Link to relevant pull requests, esp. upstream and downstream changes
  • [ x ] Ensure you have provided tests - that demonstrates feature works or fixes the issue

(there is no issue raised to link to)

We use the configuration-as-code plugin; when interpolating secrets, if
you have a typo in the secret name, it inserts the empty string:

https://github.com/jenkinsci/configuration-as-code-plugin/blob/e7501727c4704aaa1e95bdc246a1454dc5b20440/plugin/src/main/java/io/jenkins/plugins/casc/SecretSourceResolver.java#L124

Unfortunately we had a typo in the name of an AMI used by a rarely-run job,
which used a slightly different AMI from all the other jobs on our server.
So, when we deployed, we missed the warning message:

    Configuration import: Found unresolved variable '/plugins/amazonEC2/special-ami'. Will default to empty string

... then a day later, the job that used this special ami ran. Because the
code warned rather than throwing an exeption, it attempted to search for an
AMI with no search parameters, then picked one of the results to launch.
As luck would have it, there was also a typo in our IAM rule that prevented
launching public AMIs; eventually, after two days of retries, the job managed
to launch one of the random AMIs: a bitcoin miner 🤦. The node
was shut down seconds later when jenkins failed to connect to it, and our
alarms noticed the miner, leading us to investigate where it had originated.

Hence this PR...I really don't think anyone wants to use Jenkins as a random
AMI launcher :)
Copy link
Contributor

@res0nance res0nance left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, does the UI warn of this case?
I checked and it doesn't. Probably should add a UI warning here

@res0nance res0nance merged commit 2b4dc77 into jenkinsci:master Jun 2, 2021
@bazzargh bazzargh deleted the be/blank_ami_search branch June 2, 2021 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants