This repository has been archived by the owner on Sep 12, 2022. It is now read-only.
Remove expensive cloud check on unshelve #686
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
calvinmclean
reviewed
Oct 1, 2018
| hypervisor_stats = admin_driver._connection.ex_detail_hypervisor_node( | ||
| hypervisor_hostname | ||
| ) | ||
| return test_capacity(hypervisor_hostname, instance, hypervisor_stats) |
There was a problem hiding this comment.
I don't think this method (test_capacity) is used anywhere except here and can probably be removed also
There was a problem hiding this comment.
Normally this would be caught by the vulture but currently it is too muted rn to be useful :(
This check queries all the instances in the cloud. It blocks requests to unshelve. If the query takes 30 sec, then the request to unshelve takes at least 30 sec. On IU, the query takes a minute. This is way too long for a single api request. If 24 people tried to unshelve the entire app would be hung. nginx was timing out for unshelving instances on IU because of how long this took, with a Gateway Timeout Error.
cdosborn
force-pushed
the
fix-slow-unshelve
branch
from
cd2cfc6
to
b195fc9
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Problem
Unshelving on IU would return nginx gateway timeout error
Solution
Remove unnecessary cloud check
This check queries all the instances in the cloud. It blocks requests to unshelve. If the query takes 30 sec, then the request to unshelve takes at least 30 sec.
On IU, the query takes a minute. This is way too long for a single api request. If 24 people tried to unshelve the entire app would be hung. nginx was timing out for unshelving instances on IU because of how long this took, with a Gateway Timeout Error.
Checklist before merging Pull Requests