Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[weasyl] Fix favourites extractor and add support for the readable URL format #6113

Merged
merged 5 commits into from
Sep 25, 2024

Conversation

Deer-Spangle
Copy link
Contributor

When you click favourites on Weasyl, it tends to go to the clean URL format:
https://www.weasyl.com/favorites/furoferre
Rather than the user ID format.

The user ID format still works, but you click "more submissions" to get there:
https://www.weasyl.com/favorites?userid=184616&feature=submit

This change allows the extractor to use the clean URL format, as well as the user ID format. It just uses the first to get the user ID for the second.

I did notice btw that some accounts have character favourites and things, alongside submission favourites, for example this account just has character favourites:
https://www.weasyl.com/favorites/jaggy
and those are listed with feature=char:
https://www.weasyl.com/favorites?userid=12345&feature=char
But the weasyl extractors don't have support for characters anyway. But thought it notable

@Deer-Spangle
Copy link
Contributor Author

The last failing test build, for python 3.5, seems related more to yt-dlp, than to any change on my side here.

Also, I notice that weasyl extractor downloads favourites into files such as:

weasyl/{uploader}/Favorites/{id} {title}.png

Which means that basically ever favourite ends up in a different directory.
But similar other extractors, like the Furaffinity extractor, extract into:

weasyl/{user}/Favorites/{id} {title}.png

Is this a mistake? Can I change that?

…downloaded, rather than in different directories for each submission
@Deer-Spangle
Copy link
Contributor Author

In fact, given the line:

submission["user"] = owner_login

It looks like it's just a bug that it doesn't download to one directory, so I'll just change that too

@Deer-Spangle Deer-Spangle changed the title Add support for the readable URL format of Weasyl favourites page [weasyl] Add support for the readable URL format of Weasyl favourites page Aug 31, 2024
@Deer-Spangle
Copy link
Contributor Author

Oh, I hadn't noticed, but the Weasyl fav downloader where it will go into an infinite loop on the final page rather than finishing cleanly. (And then it sends lots and lots of useless requests to Weasyl, oops.)

This is because the "&nextid=" appears in every page, in the sfw link at the top of the page, rather than finding the actual next button.

I've added a fix for that now also

@Deer-Spangle Deer-Spangle changed the title [weasyl] Add support for the readable URL format of Weasyl favourites page [weasyl] Fix favourites extractor and add support for the readable URL format Sep 22, 2024
@mikf mikf merged commit cbecaec into mikf:master Sep 25, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants