-
Notifications
You must be signed in to change notification settings - Fork 9.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[XVideos] Support playlists, searches and channels #30774
base: master
Are you sure you want to change the base?
Conversation
pls merge |
Before I merge it, please test it further if you can. Only a few URLs were used to develop the new extractor functions. |
it works for channel, except the uploader output, -o "%(uploader)s" created a folder named NA and not the uploader's name |
These metadata fields are in the hydration JSON for, eg, https://www.xvideos.com/video64379435/verification_video:
#30689 is supposed to
I guess this uses those fields. With just this PR, use |
ok then will this PR merged with the other PR? |
Hey guys! Here just to comment that it would be awesome to have channel dl support, as well as uploader/uploader_id as --output option for xvideos. I've been doing all sorts of workarounds with other apps to get xvideos dls correctly, in yt-dlp above options work flawlessly on pornhub, but would be awesome to get them for xvideos too thanks and I hope it gets implemented soon :) |
There is some issue once the error "requested format not available" is thrown during download of "#_tabVideos".
Then the download is interrupted... Is this expected behaviour? The used command line options are:
Also i can confirm the id's are being successfully stored for --download-archive |
Surely your format selection is equivalent to Post the full verbose log including the URL: perhaps use |
I found the issue - the root cause is actually sitting in front of the keyboard. So I can confirm that your PR works like a charm. There was a bug in my wrapper script which took the command line arguments in a variable which was wrongly masked and therefore not correctly passed to the "-f" parameter:
Changing my long argument string with your short version visually helped me a lot to find the problem - it's usually better to have everything more readable. Thanks for that. So my issue was absolutely unrelated to your PR. Sorry for wasting your time - i am looking forward to see your changes soon in the master branch and later in the official binary. |
Tested on
|
da0092c
to
2883baa
Compare
I tested this and for me it didn't seem to work with
It also didn't work with
|
150aa26
to
d4ba4d2
Compare
* check all playlist counts, not just max * also consider any actual playlist in the test case
* add uploader, tag, performer and view_count extraction (closes ytdl-org#30689) * add dis/like_count extraction
…star pages * various -channels/... * profiles * pornstars, models * tabs within the above, with sorting and pagination where applicable * also quickie lists and videos
d4ba4d2
to
629d407
Compare
You may not have installed the working code, which gives this: $ python -m youtube_dl --flat-playlist 'https://www.xvideos.com/models/lizvicious1#_tabVideos'
[XVideosChannel] lizvicious1: Downloading webpage
[download] Downloading playlist: Lizvicious - Pornstar page (videos,all)
[XVideosChannel] lizvicious1/videos: Downloading webpage
[XVideosChannel] lizvicious1/videos (+1): Downloading webpage
[XVideosChannel] lizvicious1/videos (+2): Downloading webpage
[XVideosChannel] lizvicious1/videos (+3): Downloading webpage
[XVideosChannel] playlist Lizvicious - Pornstar page (videos,all): Downloading 109 videos
[download] Downloading video 1 of 109
[download] Downloading video 2 of 109
[download] Downloading video 3 of 109
...
[download] Downloading video 107 of 109
[download] Downloading video 108 of 109
[download] Downloading video 109 of 109
[download] Finished downloading playlist: Lizvicious - Pornstar page (videos,all)
$ See, eg, https://stackoverflow.com/questions/13561618/pip-how-to-install-a-git-pull-request. |
629d407
to
cc1657b
Compare
Interesting, I didn't think I did anything incorrectly. I cloned the repo and checked out the branch and ran make, ran the binary it gave me and it gave me that output, here are the commands I used:
Then I ran it from the bin folder. With that said, your pip install suggestion worked for me, thanks! Here is the command I used: |
After checking out the code, you can run it from the |
Hey, could you please update this code for the new site changes? |
You mean like yt-dlp/yt-dlp#9502 ? |
Yes and no. |
If anyone would like to document the site changes with regard to the various types of pages that the site supports and the PR extractor is meant to support, that would make it a lot quicker to update. |
The most important change is the formatting of the video links. Was: Is now: For some reason (haven't had much time to dive into the code) youtube-dl extracts the video style URL, which is invalid. |
Way I see it, the issue is here: extractor/xvideos.py around line 914 def _extract_videos(self, url, playlist_id, num, page): This is the input of one video's data in the {
"id": 72799231,
"u": "/prof-video-click/model/creamy-spot1/uopdkffea37/big_dildo_for_creamy_pussy_squirt_watch_full_uncensored_video_in_red_subscription_",
"i": "https://cdn77-pic.xvideos-cdn.com/videos/thumbs169/28/6d/6f/286d6f37c99ae3ceee9d25ecbe688a59/286d6f37c99ae3ceee9d25ecbe688a59.27.jpg",
"il": "https://cdn77-pic.xvideos-cdn.com/videos/thumbs169ll/28/6d/6f/286d6f37c99ae3ceee9d25ecbe688a59/286d6f37c99ae3ceee9d25ecbe688a59.27.jpg",
"if": "https://cdn77-pic.xvideos-cdn.com/videos/thumbs169lll/28/6d/6f/286d6f37c99ae3ceee9d25ecbe688a59/286d6f37c99ae3ceee9d25ecbe688a59.27.jpg",
"ip": "https://cdn77-pic.xvideos-cdn.com/videos/thumbs169poster/28/6d/6f/286d6f37c99ae3ceee9d25ecbe688a59/286d6f37c99ae3ceee9d25ecbe688a59.27.jpg",
"c": 10,
"tf": "Big Dildo for Creamy Pussy Squirt (Watch full uncensored video in RED subscription)",
"t": "Big Dildo for Creamy Pussy Squirt (Watch full u...",
"d": "2 min",
"r": "99%",
"n": "247.4k",
"v": false,
"vim": 0,
"vv": 0,
"hm": 1,
"h": 1,
"hp": 1,
"td": 0,
"fk": 0,
"ve": 0,
"ui": 611724437,
"p": "creamyspot",
"pn": "Creamyspot",
"pu": "/creamyspot",
"ch": true,
"pm": false,
"ut": null,
"iu": false
} It seems the extractor is using Hope this makes things easier |
Please follow the guide below
Before submitting a pull request make sure you have:
In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:
What is the purpose of your pull request?
Description of your pull request and other information
The XVideos extractor supported single videos presented via various URL formats. The site also offers various playlist-like pages:
This PR tries to support extracting from those pages. It also pulls in small changes from the yt-dlp extractor.
To do: