Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitter extractor (slightly) broken, for some accounts will loop endlessly. #499

Closed
pxssy opened this issue Dec 1, 2019 · 4 comments
Closed

Comments

@pxssy
Copy link

pxssy commented Dec 1, 2019

I'm very sure its isolated to v1.11.0 and later. probably #448 caused it.

Not sure what the conditions it is to trigger the infinite cycling but https://twitter.com/morimokimori is an example of one that does. https://files.catbox.moe/i0qh3c.png It goes from 118... to 120... then back to 118... to 120... rinse and repeat

I appreciate the new addition, hope this doesn't discourage improvements, its just a oversight and i've noticed this only occurs to quite a small proportion of users.

@mikf
Copy link
Owner

mikf commented Dec 3, 2019

The infinite loop for https://twitter.com/morimokimori doesn't happen for me. Could you post a verbose log file for what happens when you try to download from that URL?

$ gallery-dl --verbose --write-log log.txt https://twitter.com/morimokimori

@alice945
Copy link
Contributor

alice945 commented Dec 3, 2019

I was able to reproduce it. It gets stuck at this URL for me
https://twitter.com/i/profiles/show/morimokimori/timeline/tweets?include_available_features=1&include_entities=1&max_position=1188111843300036608&reset_error_state=false&lang=en
For some reason this last page gives
"min_position":null, "has_more_items": true
It doesn't stop because it thinks there is more items but it sets the next position back to the beginning, therefore looping forever.

@mikf
Copy link
Owner

mikf commented Dec 3, 2019

And I'm getting {"min_position":"996018711524917248","has_more_items":false, from that URL
Weird, but at least it's easy to fix.

mikf added a commit that referenced this issue Dec 3, 2019
(Hopefully this doesn't break anything else)
@mikf
Copy link
Owner

mikf commented Dec 13, 2019

@pxssy Is this still happening with v1.12.0 or did 07dafad fix it for you?

@mikf mikf closed this as completed Dec 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants