aws: Add singleflight support to SafeCredentialsProvider #503
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jasdel
approved these changes
Mar 16, 2020
skmcgrail
added a commit
to skmcgrail/aws-sdk-go-v2
that referenced
this pull request
Mar 17, 2020
Breaking Change --- * Update SDK retry behavior * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling. * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires. * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`) * Update SDK error handling * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error. * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`. * Bump SDK minimum supported version from Go 1.12 to Go 1.13 * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package. Services --- * Synced the V2 SDK with latest AWS service API definitions. SDK Features --- * `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488)) * `aws/processcreds`: Adds Support for the Process Credential Provider * Fixes [aws#249](aws#249) * `aws/stscreds`: Adds Support for the Web Identity Credential Provider * Fixes [aws#475](aws#475) * Fixes [aws#338](aws#338) * Adds Support for `credential_source` * Fixes [aws#274](aws#274) * `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487)) * `aws`: Updates SDK retry behavior ([aws#487](aws#487)) * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff. * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier. SDK Enhancements --- * `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503) * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently. * `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces. SDK Bugs --- * `service/s3/s3manager`: Fix resource leaks when the following occurred: * Failed CreateMultipartUpload call * Failed UploadPart call
skmcgrail
added a commit
to skmcgrail/aws-sdk-go-v2
that referenced
this pull request
Mar 17, 2020
Breaking Change --- * Update SDK retry behavior * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling. * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires. * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`) * Update SDK error handling * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error. * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`. * Bump SDK minimum supported version from Go 1.12 to Go 1.13 * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package. Services --- * Synced the V2 SDK with latest AWS service API definitions. SDK Features --- * `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488)) * `aws/processcreds`: Adds Support for the Process Credential Provider * Fixes [aws#249](aws#249) * `aws/stscreds`: Adds Support for the Web Identity Credential Provider * Fixes [aws#475](aws#475) * Fixes [aws#338](aws#338) * Adds Support for `credential_source` * Fixes [aws#274](aws#274) * `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487)) * `aws`: Updates SDK retry behavior ([aws#487](aws#487)) * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff. * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier. SDK Enhancements --- * `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503)) * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently. * `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces. SDK Bugs --- * `service/s3/s3manager`: Fix resource leaks when the following occurred: * Failed CreateMultipartUpload call * Failed UploadPart call
skmcgrail
added a commit
that referenced
this pull request
Mar 17, 2020
Breaking Change --- * Update SDK retry behavior * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling. * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires. * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`) * Update SDK error handling * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error. * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`. * Bump SDK minimum supported version from Go 1.12 to Go 1.13 * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package. Services --- * Synced the V2 SDK with latest AWS service API definitions. SDK Features --- * `aws`: Add Support for additional credential providers and credential configuration chaining ([#488](#488)) * `aws/processcreds`: Adds Support for the Process Credential Provider * Fixes [#249](#249) * `aws/stscreds`: Adds Support for the Web Identity Credential Provider * Fixes [#475](#475) * Fixes [#338](#338) * Adds Support for `credential_source` * Fixes [#274](#274) * `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([#487](#487)) * `aws`: Updates SDK retry behavior ([#487](#487)) * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff. * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier. SDK Enhancements --- * `aws`: Add grouping of concurrent refresh of credentials ([#503](#503)) * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently. * `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces. SDK Bugs --- * `service/s3/s3manager`: Fix resource leaks when the following occurred: * Failed CreateMultipartUpload call * Failed UploadPart call
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ports aws/aws-sdk-go#3127 from the V1 SDK, some of the context refactor work to V2 had to be modified in order to ensure a single context cancellation would not terminate the on-going retrieval of credentials from some resource.