Patreon error: Unable to extract bootstrap data (gallery-dl 1.26.9)

[GittyGharial]

On May 9th running gallery-dl 1.26.8, I was able to scrape some files from Patreon. Today, May 21st, I was not able to and got an error message: Unable to extract bootstrap data. I searched the message and found two tickets from December stating the issue had been solved in the latest gallery-dl update, so I updated to 1.26.9. However, I get the same error anyway. I have confirmed the version of gallery-dl being called is 1.26.9 with gallery-dl --version

My cookies.txt, test_config.json, and command syntax are the exact same as they were before. I even tested with an older post that had worked on May 9th and it failed with the same error today. I do not know how to fix this at this point and would appreciate any advice you can give. Thank you.

[Tobi823]

@jcurtis56 I think it is better to create a new ticket instead of reviving an older, already closed tikcet.

I can confirm the problem. I also suspect that Patreon changed how the data is embedded into the website.

It is quite late in Germany but I can share the HTML file from https://www.patreon.com/posts/who-really-won-104412696

I reformatted the JSON to make it more readable and to remove private information more easily:
website_reformatted.txt

I hope that my JSON file helps :)

[mikf]

Should be fixed in bc25636

[BoostHellSix66]

I downloaded the latest dev build 1.27.0-dev, and I seem to be getting another error [patreon][warning] Cloudflare challenge

Here's the verbose output:
[gallery-dl][debug] Version 1.27.0-dev: - Executable
[gallery-dl][debug] Python 3.12.3 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.32.2 - urllib3 2.2.1
[gallery-dl][debug] Configuration Files ['*:\\***\\***\\gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/posts/profile-setting-104162348'
[patreon][debug] Using PatreonPostExtractor for 'https://www.patreon.com/posts/profile-setting-104162348'
[patreon][debug] TLS 1.2 disabled.
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /posts/profile-setting-104162348 HTTP/1.1" 403 None
[patreon][warning] Cloudflare challenge
[patreon][error] HttpError: '403 Forbidden' for 'https://www.patreon.com/posts/profile-setting-104162348'

any further help is greatly appreciated, thanks.

[jcurtis56]

Latest dev build works for me. No issues with Cloudflare. Thanks so much!

[GittyGharial]

Thank you!

[suzumiyabi]

Looking forward to the release.

[conkerts]

I'll jump in here, too.
I had this issue obviously on 1.26.9, too, but it still persists for me, even with the latest nightly Version 1.27.0-dev,
i.e. https://github.com/gdl-org/builds/releases/tag/2024.05.28 and previous ones don't work either.
Maybe they changed again something ?

[Tobi823]

The version from 22.05.2024 still works for me. Maybe they are rolling out the changes gradually. Can you double check that your are using the correct cookies and that these cookies are not expired?

[conkerts]

Alright. It works now indeed.

I ofc compared the cookies last time already, and the
patreon_device_id and session_id
were matching, but I still moved over the new file and was using it ...

Some other ones changed, but I didn't think those should be an issue, esp. as I used the new cookies txt anyway.

Now I tried again the latest version form 28. again and got it to work.
It was unrelated to this problem, as I used the ps1 script with all the paths for cookies, and settings etc, but the script was using a global old gallery-dl I put somewhere in my path, not the new nightly I downloaded in the local folder.
So my bad, sry for bothering 😅

I think next time I wont overwrite file in the path, as it makes indeed more sense to have the nightly with a separate file name, to keep them apart and avoid confusion.

[mikf]

@BoostHellSix66
The Cloudflare issue was most likely caused by requests 2.32.2.
This shouldn't be an issue with the latest executables anymore (330162f).

[GittyGharial]

Unfortunately I just tried it using the gallery-dl 1.27.0 macports release, and I got this error:

[patreon][error] HttpError: SSLError: HTTPSConnectionPool(host='www.patreon.com', port=443): Max retries exceeded with url: /posts/{post-name}-{number} (Caused by SSLError(SSLCertVerificationError(5, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')))

The command I ran was
gallery-dl --cookies "./cookies-patreon-com.txt" --config=./test_config.json {url}, which has worked before.

[GittyGharial]

Yep, the macports release is behind. I installed the latest dev version with pip and ran that one (~/Library/Python/3.11/bin/gallery-dl) and it worked just fine.

[siubb]

Hi guys, does version 1.27.1 work?

.\gallery-dl --version
1.27.1

.\gallery-dl --write-metadata --cookies "www.patreon.com_cookies.txt"

[patreon][warning] Cloudflare challenge
[patreon][error] HttpError: '403 Forbidden' for '...

[GittyGharial]

@siubb I think I ran into that issue too. The fix was to update to 1.27.2