Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3Uploader aws client usage efficiency #2198

Merged
merged 2 commits into from
Apr 8, 2021

Conversation

ssalinas
Copy link
Member

@ssalinas ssalinas commented Apr 8, 2021

Previously we spun up one client per S3Uploader object, since they often have separate creds. However, many are shared, and duplicates become a problem when we start hitting thousands of uploader objects.

Open questions:

  • Should there be some sort of cool down period before I junk a client so we aren't recreating too often?
  • Keying on the string concat of creds feels dirty, but BasicAwsCredentials doesn't implement hashCode, thoughts? It's not logged anywhere, and the BasicAwsCreds are already in memory like that elsewhere obviously, just felt weird to write 🤷

cc @baconmania

@baconmania
Copy link
Contributor

🚢

@ssalinas ssalinas merged commit fac6f07 into master Apr 8, 2021
@ssalinas ssalinas deleted the s3_uploader_client_efficiency branch April 8, 2021 20:06
@ssalinas ssalinas added this to the 1.5.0 milestone May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants