Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directories are created even when a file is skipped due to being in archive #2461

Closed
KennethSamael opened this issue Apr 1, 2022 · 6 comments

Comments

@KennethSamael
Copy link

This might not be noticeable for all use cases, but it's been causing me annoyance since I discovered it. Seems like the directory in which a downloaded file is to be placed is created before checking whether that file should actually be downloaded. If all the files that would go into that directory are already in archive, you just end up with an empty directory that has no reason to exist, and which will be recreated every time you try to download from the same source.

@Hrxn
Copy link
Contributor

Hrxn commented Apr 2, 2022

That's true..

Although it has not really been an issue for me, personally (obligatory "My Workflow! It's my Workflow!).
Because I copy everything from the download destination into my permanent storage place, and so all those empty directories get simply merged into the already existing directories..

@rautamiekka
Copy link
Contributor

It's especially bad with sites like DeviantArt and Tumblr where you've a strict call limit cuz if the username doesn't exist or otherwise is a problem, running a for loop over the usernames would eat >=1 call for nothing.

@github-account1111
Copy link

This behavior has actually been kinda helpful for me, since if a folder is empty I know the uploader has removed or renamed their account.

@Hrxn
Copy link
Contributor

Hrxn commented Apr 3, 2022

This behavior has actually been kinda helpful for me, since if a folder is empty I know the uploader has removed or renamed their account.

That... should be handled by info/warning/error messages.. and it mostly is, otherwise I'd consider that a bug?

@github-account1111
Copy link

github-account1111 commented Apr 3, 2022

That... should be handled by info/warning/error messages.. and it mostly is, otherwise I'd consider that a bug?

I knew there has to be a better way :D

mikf added a commit that referenced this issue May 4, 2022
Only call os.makedirs() when an open() call
fails with a FileNotFoundError

Might cause errors with some post processors etc,
but for now I was unable to find any.
@AlttiRi
Copy link

AlttiRi commented May 7, 2022

Damn, it was implemented without ability to opt-out.

I used to use even the empty folders to put into them related downloaded data.

Also it was a way to to verify that you have input all URLs correctly — if I input 14 URLs, the expected result — 14 folders, if no, probably, there was some problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants