Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 transfer does not properly handle running out of disk space #739

Closed
jamesls opened this issue Apr 3, 2014 · 0 comments · Fixed by #745
Closed

S3 transfer does not properly handle running out of disk space #739

jamesls opened this issue Apr 3, 2014 · 0 comments · Fixed by #745
Labels

Comments

@jamesls
Copy link
Member

jamesls commented Apr 3, 2014

If we run out of disk space (including inodes) we keep trying to sync. We also don't throw any kind of exception when this happens. I believe the root cause if from this code snippet:

def save_file(filename, response_data, last_update):
    body = response_data['Body']
    etag = response_data['ETag'][1:-1]
    d = os.path.dirname(filename)
    try:
        if not os.path.exists(d):
            os.makedirs(d)
    except Exception:      # <---- bad
        pass

We should be more selective in this case and only allow errno.EEXIST errors.

@jamesls jamesls added the s3 label Apr 3, 2014
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 4, 2014
The "except Exception" is too general for the makedirs
call.  Instead we only catch the specific case where a directory
already exists (because a thread has already come along and created it).
Any other exception propogates.

Fixes aws#739.
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 4, 2014
The "except Exception" is too general for the makedirs
call.  Instead we only catch the specific case where a directory
already exists (because a thread has already come along and created it).
Any other exception propogates.

Fixes aws#739.
@jamesls jamesls closed this as completed in ad05114 Apr 9, 2014
thoward-godaddy pushed a commit to thoward-godaddy/aws-cli that referenced this issue Feb 12, 2022
Travis expects output from commands and if no ouput is recieved,
Travis kills the build thinking command has stalled. Some of our
integ tests cause this issue in travis: https://travis-ci.org/awslabs/aws-sam-cli/jobs/447830765

This commit adds travis_wait to our integ test command. Details on
this can be found here: https://docs.travis-ci.com/user/common-build-problems/#my-builds-are-timing-out
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant