Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_helper.py improvements #442

Merged
merged 2 commits into from
Jun 21, 2024
Merged

test_helper.py improvements #442

merged 2 commits into from
Jun 21, 2024

Conversation

graebm
Copy link
Contributor

@graebm graebm commented Jun 19, 2024

Description of changes:

  • Don't skip work if the bucket already exists, just upload everything every time you run init.
    • BECAUSE: This lets you add new objects to this script without obliterating all your buckets first.
  • Don't try and print exceptions as strings, just let python print out the stacktrace
    • BECAUSE: I was getting MemoryErrors because python couldn't allocate a 2GiB buffer, but MemoryError prints as an empty string, so I had no idea what was going wrong.
  • Always print() before doing an S3 call, instead printing after doing an S3 call
    • BECAUSE: It makes the cause of error, and the cause of long delays, more obvious
  • Remove the error-handling that would delete all your buckets if anything went wrong during init.
    • BECAUSE: If you try and add a new file, and something goes wrong, and the script deletes ALL of the team's buckets in response, all kinds of CI will start failing. If you really want to clean up, just run the script again with clean

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@graebm graebm merged commit cb431ba into main Jun 21, 2024
30 checks passed
@graebm graebm deleted the better-test-helper branch June 21, 2024 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants