-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update makedirs function to not supress OSErrors #3404
base: proton_4.11
Are you sure you want to change the base?
Conversation
Looks reasonable, thanks. Will merge. |
Updated a different occurrence of this as well. We're also ignoring errors in the |
The logic seems fine, but is |
From what I could gather from the scandir proposal, that seems to be correct. The latest commit should address that. |
proton
Outdated
os.makedirs(path) | ||
except OSError: | ||
#already exists | ||
except OSError as err: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use FileExistsError instead of OSError
proton
Outdated
#not empty | ||
if not any(os.scandir(d)): | ||
os.rmdir(d) | ||
except OSError as err: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead
except OSError as e:
if e.errno != errno.ENOTEMPTY:
raise
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I could gather checking if the directory is empty before attempting to do the operation is less expensive than letting it throw an exception and catching it. Would we still prefer going with the latter approach?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no measurable performance difference between the two (although the deletion and seeing whether it fails is most likely faster as it results in less syscalls on average) but with the exception handling race conditions are impossible. That's typically recommended way to handle these things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. I hadn't considered race conditions. I'll make the changes in a bit :)
The current implementation of makedirs suppresses every OSError possible and assumes the directory already exists when one is raised. This is problematic as there are many reasons why the error could occur, and suppressing it could make it hard to trace the problem back in the future.
My implementation fixes this by checking whether the path exists before attempting to create it, and logs every other OSError appropriately.
I also took the liberty of changing a stray
os.makedirs
call to ourmakedirs
function.