Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix for: #76
The SSM DescribeParameters API has very aggressive rate-limits because it's only meant for manual cli-usage. This can lead to significant delays or timeout errors when using chamber exec/export simultaneously in several places (eg. when starting a whole lot of processes in production).
Chamber uses the DescribeParameters API to find parameters by prefix (eg. 'myservice.*') when listing parameters. There's no better API for this in the general case, but if using paths, AWS provides a GetParametersByPath API which is designed for production usage and has much higher rate-limits. Unlike DescribeParameters, GetParametersByPath only returns parameter names and values; it does not include any metadata (eg. createdOn timestamps).
This PR separates the use-cases of listing parameter metadata (just for the list command) and listing parameter values (for exec and export commands). It then utilises the GetParametersByPath API for the values-only commands IFF usePaths is true.
Since the GetParametersByPath API requires a new permission, chamber will now gracefully fall-back to the old DescribeParameters implementation, but it will log a deprecation warning about the need for a new permission.
At StileEducation, we've been using a fork with the GetParametersByPath code for a week or so now with no problems. It's totally eradicated the rate-limiting issues which were previously preventing us from deploying chamber.
This doesn't fix the rate-limiting problem for anyone who's not using paths, but there's no good way to do that with the given AWS APIs. The best option would be to call GetParametersByPath on the path
'/'
, then filter the results locally, but that could be really slow for some use-cases; it's not an obvious win like this change is.The best long-term solution is probably to move to using paths as the preferred convention, in-line with the design of the AWS APIs.