Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exhentai SSL issue #4891

Closed
Saurutobi opened this issue Dec 6, 2023 · 17 comments
Closed

exhentai SSL issue #4891

Saurutobi opened this issue Dec 6, 2023 · 17 comments
Labels

Comments

@Saurutobi
Copy link

using version 1.26.3 and previous versions, I've been getting this error on a bunch of galleries not just this one for about a month.

gallerydl.exe https://exhentai.org/s/af341fd42a/2744493-68
[exhentai][info] Logging in as ***me***
[exhentai][warning] Enabled Multi-Page Viewer is not supported
[downloader.http][warning] HTTPSConnectionPool(host='xxcwzlb.qjvlwvmsmdol.hath.network', port=29977): Max retries exceeded with url: /h/af341fd42af05d174b2fa160d878b025dfe8101b-2483476-3840-2160-jpg/keystamp=1701903600-74034d1865;fileindex=138046011;xres=org/066.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)'))) (1/5)
[downloader.http][warning] HTTPSConnectionPool(host='xxcwzlb.qjvlwvmsmdol.hath.network', port=29977): Max retries exceeded with url: /h/af341fd42af05d174b2fa160d878b025dfe8101b-2483476-3840-2160-jpg/keystamp=1701903600-74034d1865;fileindex=138046011;xres=org/066.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)'))) (2/5)
[downloader.http][warning] HTTPSConnectionPool(host='xxcwzlb.qjvlwvmsmdol.hath.network', port=29977): Max retries exceeded with url: /h/af341fd42af05d174b2fa160d878b025dfe8101b-2483476-3840-2160-jpg/keystamp=1701903600-74034d1865;fileindex=138046011;xres=org/066.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)'))) (3/5)
[downloader.http][warning] HTTPSConnectionPool(host='xxcwzlb.qjvlwvmsmdol.hath.network', port=29977): Max retries exceeded with url: /h/af341fd42af05d174b2fa160d878b025dfe8101b-2483476-3840-2160-jpg/keystamp=1701903600-74034d1865;fileindex=138046011;xres=org/066.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)'))) (4/5)
[downloader.http][warning] HTTPSConnectionPool(host='xxcwzlb.qjvlwvmsmdol.hath.network', port=29977): Max retries exceeded with url: /h/af341fd42af05d174b2fa160d878b025dfe8101b-2483476-3840-2160-jpg/keystamp=1701903600-74034d1865;fileindex=138046011;xres=org/066.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)'))) (5/5)
[download][info] Trying fallback URL #1

and that's where I cut it off.

@Saurutobi
Copy link
Author

went back and tried 1.25.8 with verbose:

[gallery-dl][debug] Version 1.25.8 - Executable
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.19045
[gallery-dl][debug] requests 2.31.0 - urllib3 1.26.16
[gallery-dl][debug] Configuration Files ['%USERPROFILE%\\gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://exhentai.org/s/af341fd42a/2744493-68'
[exhentai][debug] Using ExhentaiGalleryExtractor for 'https://exhentai.org/s/af341fd42a/2744493-68'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): exhentai.org:443
[urllib3.connectionpool][debug] https://exhentai.org:443 "GET /s/af341fd42a/2744493-68 HTTP/1.1" 200 None
[exhentai][debug] Sleeping 5.00 seconds (request)
[urllib3.connectionpool][debug] https://exhentai.org:443 "GET /g/2744493/7ff983d799/ HTTP/1.1" 200 None
[exhentai][warning] Enabled Multi-Page Viewer is not supported
# .\gallery-dl\exhentai\2744493 Au Naturel [Pegasus Smith] - 17 - english\2744493_0068_af341fd42a_066.jpg
[exhentai][debug] Sleeping 5.00 seconds (request)
[urllib3.connectionpool][debug] https://exhentai.org:443 "POST /api.php HTTP/1.1" 200 None
[exhentai][error] An unexpected error occurred: KeyError - 'i7'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[exhentai][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 96, in run
  File "gallery_dl\extractor\exhentai.pyc", line 212, in items
  File "gallery_dl\extractor\exhentai.pyc", line 343, in images_from_api
KeyError: 'i7'

@mikf
Copy link
Owner

mikf commented Dec 7, 2023

SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired

Probably just a H@H node with expired certs.
You could use --no-check-certificate to ignore these errors.

@Saurutobi
Copy link
Author

Saurutobi commented Dec 7, 2023

Using --no-check-certificate yields this error:

[exhentai][error] An unexpected error occurred: ValueError - Cannot set verify_mode to CERT_NONE when check_hostname is enabled.. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .

I didn't see a check_hostname disable option in the options.md

@mikf
Copy link
Owner

mikf commented Dec 23, 2023

Using --no-check-certificate yields this error:

[exhentai][error] An unexpected error occurred: ValueError - Cannot set verify_mode to CERT_NONE when check_hostname is enabled.. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .

This should be fixed in urllib3 >= 2.0.4

@aleste81
Copy link

aleste81 commented Jan 3, 2024

I have the same problem.
You say is it fixed in urllib3, but I don't know/use urllib3.
I only use gallery-dl.exe from your release section, under Windows 7.
No Python, no Unix, no nothing.

@aleste81
Copy link

aleste81 commented Jan 3, 2024

ok it seems I have Python after all. The latest version of Python for Windows 7 is 3.8.10
I tried
C:\Users\user\AppData\Local\Programs\Python\Python38\Scripts>pip install -U --force-reinstall requests
But after that command urllib3 is still version 2.1.0

@aleste81
Copy link

aleste81 commented Jan 3, 2024

pip was v21, I upgraded it to v23 with this
python.exe -m pip install --upgrade pip
But after that command urllib3 is still version 2.1.0

@aleste81
Copy link

aleste81 commented Jan 4, 2024

I tried on Windows 10, still the certificat error :

[gallery-dl][debug] Version 1.26.5 - Executable
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.22631
[gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://exhentai.org/g/xxx/xxx/'
[exhentai][debug] Using ExhentaiGalleryExtractor for 'https://exhentai.org/g/xxx/xxx/'
[exhentai][info] Logging in as xxx
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): forums.e-hentai.org:443
[exhentai][error] An unexpected error occurred: ValueError - Cannot set verify_mode to CERT_NONE when check_hostname is enabled.. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[exhentai][debug]
Traceback (most recent call last):
File "gallery_dl\job.pyc", line 127, in run
File "gallery_dl\extractor\exhentai.pyc", line 167, in items
File "gallery_dl\extractor\exhentai.pyc", line 68, in login
File "gallery_dl\cache.pyc", line 115, in call
File "gallery_dl\extractor\exhentai.pyc", line 97, in _login_impl
File "gallery_dl\extractor\exhentai.pyc", line 52, in request
File "gallery_dl\extractor\common.pyc", line 162, in request
File "requests\sessions.pyc", line 589, in request
File "requests\sessions.pyc", line 703, in send
File "requests\adapters.pyc", line 486, in send
File "urllib3\connectionpool.pyc", line 790, in urlopen
File "urllib3\connectionpool.pyc", line 467, in _make_request
File "urllib3\connectionpool.pyc", line 1096, in _validate_conn
File "urllib3\connection.pyc", line 642, in connect
File "urllib3\connection.pyc", line 744, in _ssl_wrap_socket_and_match_hostname
File "ssl.pyc", line 720, in verify_mode
ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is enabled.

@Saurutobi
Copy link
Author

v2.1.0 is the current released one https://pypi.org/project/urllib3/

@mikf
Copy link
Owner

mikf commented Jan 4, 2024

Could you try urllib3 before 2.0?
(pip install -U "urllib3<2" or use a gallery-dl exectutable <= 1.26.2)

Are you using some "special" network settings?
I still can't reproduce this error, even on Windows 7.

@aleste81
Copy link

aleste81 commented Jan 4, 2024

In Windows 10, I removed the parameter
--no-check-certificate
and it works (same network)(same Python,pip,urllib3 version

So I uninstalled/reinstalled Python 3.8.10, and here is the result :

with Windows 11 version 23H2
IT WORKS

with Windows 7 version 6.1 build 7601
IT FAILS
SSL: CERTIFICATE_VERIFY_FAILED

same LAN, same internet, same URL...

@aleste81
Copy link

aleste81 commented Jan 4, 2024

hmmm I finally understood that my current installation of Python has no effect on gallery-dl.
I uninstalled it, and tried a few older versions of gallery-dl

They all fail on Windows 7 AND Windows 11, with error
An unexpected error occurred: KeyError - 'i7'

@Saurutobi
Copy link
Author

did v1.25.8 as well on a random exhentai url. both had the same issue.
I'm also on win10, don't have anything else to test with. I am running H@H and have the ports forwarded, as well as the InsurgencySandstorm localserver ports(27102, 27131)

@Unknowndraigon
Copy link

Unknowndraigon commented Jan 15, 2024

I think I also have this problem? I've been looking around but no matter what I do I end up with this:

[gallery-dl][debug] Version 1.25.8
[gallery-dl][debug] Python 3.10.1 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://e-hentai.org/g/2791954/42da5db332/'
[exhentai][debug] Using ExhentaiGalleryExtractor for 'https://e-hentai.org/g/2791954/42da5db332/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): e-hentai.org:443
[urllib3.connectionpool][debug] https://e-hentai.org:443 "GET /g/2791954/42da5db332/ HTTP/1.1" 200 10153
[exhentai][debug] Sleeping 5.00 seconds (request)
[urllib3.connectionpool][debug] https://e-hentai.org:443 "GET /s/598cd0247b/2791954-1/2791954-1 HTTP/1.1" 200 1557
# E:\gallery-dl\exhentai\2791954 [DrShanks24] Booby Quest ...1_598cd0247b_01_664695_boobyquestng_boobyquest_ch1_pg1.jpg
[exhentai][debug] Sleeping 5.00 seconds (request)
[urllib3.connectionpool][debug] https://e-hentai.org:443 "POST /api.php HTTP/1.1" 200 934
[exhentai][error] An unexpected error occurred: KeyError - 'i7'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[exhentai][debug]
Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\gallery_dl\job.py", line 96, in run
    for msg in extractor:
  File "C:\Python310\lib\site-packages\gallery_dl\extractor\exhentai.py", line 212, in items
    for url, image in images:
  File "C:\Python310\lib\site-packages\gallery_dl\extractor\exhentai.py", line 343, in images_from_api
    origurl, pos = text.extract(page["i7"], '<a href="', '"')
KeyError: 'i7'

@mikf
Copy link
Owner

mikf commented Jan 15, 2024

@Unknowndraigon

[gallery-dl][debug] Version 1.25.8

Your problem is an outdated version. The i7 error got fixed in v1.26.2 (#4730, 68e72a8)

@RedCowOwO
Copy link

RedCowOwO commented Jan 17, 2024

Hi there I'm back. Yeah I have now encountered the same issue as well but, I had help to fix the i7 problem I was previously encountering. Below is the process before I run into the error that keeps repeating for each possible image.

C:\Users\redcow>gallery-dl``_ --verbose https://e-hentai.org/g/379599/f865aec9a4/ [gallery-dl][debug] Version 1.26.3-dev [gallery-dl][debug] Python 3.11.1 - Windows-10-10.0.19045-SP0 [gallery-dl][debug] requests 2.31.0 - urllib3 2.1.0 [gallery-dl][debug] Configuration Files ['%USERPROFILE%\\gallery-dl.conf'] [gallery-dl][debug] Starting DownloadJob for 'https://e-hentai.org/g/379599/f865aec9a4/' [exhentai][debug] Using ExhentaiGalleryExtractor for 'https://e-hentai.org/g/379599/f865aec9a4/' [urllib3.connectionpool][debug] Starting new HTTPS connection (1): e-hentai.org:443 [urllib3.connectionpool][debug] https://e-hentai.org:443 "GET /g/379599/f865aec9a4/ HTTP/1.1" 200 8002 [exhentai][debug] Sleeping 5.00 seconds (request) [urllib3.connectionpool][debug] https://e-hentai.org:443 "GET /s/03221950db/379599-1/379599-1 HTTP/1.1" 200 1516 [exhentai][debug] Using download archive 'skip' [urllib3.connectionpool][debug] Starting new HTTPS connection (1): jkkhcgy.bpnilduvcmvp.hath.network:443 [downloader.http][warning] HTTPSConnectionPool(host='jkkhcgy.bpnilduvcmvp.hath.network', port=443): Max retries exceeded with url: /h/03221950dbbf59caed09d672ac841d3ad3c85fab-557818-1083-1600-jpg/keystamp=1705466100-9d3c264f3b;fileindex=10822242;xres=org/01.jpg (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:992)'))) (1/5)

@Unknowndraigon
Copy link

@Unknowndraigon

[gallery-dl][debug] Version 1.25.8

Your problem is an outdated version. The i7 error got fixed in v1.26.2 (#4730, 68e72a8)

Was able to update and it works now, thanks!

@mikf mikf added bug and removed cant-reproduce labels May 10, 2024
mikf added a commit that referenced this issue May 10, 2024
e.g. when 'browser' is set to a non-empty value and gallery-dl creates
its own SSLContext instance instead of using requests' and urllib3's
defaults.

urllib3 disables this option for its default contexts,
since it does this check on its own.

Fixes "ValueError: Cannot set verify_mode to CERT_NONE when
check_hostname is enabled" when using --no-check-certificate.

(#3614, #4891, #5576)
@mikf mikf closed this as completed May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants