Avoid AWS rate-limits #79
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Which returns secret keys and values, but *not* metadata. Use it for the production-like commands — exec and export — which don't need the metadata. Currently it's just a wrapper around List, which uses the AWS DescribeParameters API and thus has access to all of the metadata. Because that API is intended for CLI (not production) usage, it has very strict rate-limits. Separating-out the production usages with the new ListRaw API will allow us to use the AWS GetParametersByPath method instead, which only returns the keys and values, but has much higher rate-limits.
To avoid AWS throttling limits on DescribeParameters
So that anyone using paths who hasn't granted GetParametersByPath permission can still use chamber they way they used-to. They'll get a warning about the need to add the new permission, which will also allow them to avoid rate-limits.
dfuentes
approved these changes
Mar 16, 2018
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix for: #76
The SSM DescribeParameters API has very aggressive rate-limits because it's only meant for manual cli-usage. This can lead to significant delays or timeout errors when using chamber exec/export simultaneously in several places (eg. when starting a whole lot of processes in production).
Chamber uses the DescribeParameters API to find parameters by prefix (eg. 'myservice.*') when listing parameters. There's no better API for this in the general case, but if using paths, AWS provides a GetParametersByPath API which is designed for production usage and has much higher rate-limits. Unlike DescribeParameters, GetParametersByPath only returns parameter names and values; it does not include any metadata (eg. createdOn timestamps).
This PR separates the use-cases of listing parameter metadata (just for the list command) and listing parameter values (for exec and export commands). It then utilises the GetParametersByPath API for the values-only commands IFF usePaths is true.
Since the GetParametersByPath API requires a new permission, chamber will now gracefully fall-back to the old DescribeParameters implementation, but it will log a deprecation warning about the need for a new permission.
At StileEducation, we've been using a fork with the GetParametersByPath code for a week or so now with no problems. It's totally eradicated the rate-limiting issues which were previously preventing us from deploying chamber.
This doesn't fix the rate-limiting problem for anyone who's not using paths, but there's no good way to do that with the given AWS APIs. The best option would be to call GetParametersByPath on the path
'/', then filter the results locally, but that could be really slow for some use-cases; it's not an obvious win like this change is.The best long-term solution is probably to move to using paths as the preferred convention, in-line with the design of the AWS APIs.