Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

一个while True requests运行的爬虫程序,会报错:HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /aa (Caused by SSLError(OSError(24, 'Too many open files'))) #6766

Closed
wucyAAA opened this issue Jul 11, 2024 · 1 comment
Labels
actions/autoclose-qa Used for automation to auto-close an issue Question/Not a bug

Comments

@wucyAAA
Copy link

wucyAAA commented Jul 11, 2024

with requests.session() as session:
while True:
try:
task = get_task()
if task is None:
time.sleep(1)
continue
response = session.get(url, headers=headers, proxies=task['ip'],timeout=6)
parse_detail(response .text)
except Exception as e:
report_data(task['url'], [], task['tid'],False,str(e))
logger.error(e)
time.sleep(1)
大概就是这样的结构,只有request操作,是什么地方没写好会造成这样的错误呢?能给点建议吗?
小白第一次提问,可能说的不够清晰,抱歉抱歉

@sigmavirus24 sigmavirus24 added Question/Not a bug actions/autoclose-qa Used for automation to auto-close an issue labels Jul 11, 2024
Copy link

As described in the template, we won't be able to answer questions on this issue tracker. Please use Stack Overflow

@github-actions github-actions bot locked as off-topic and limited conversation to collaborators Jul 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
actions/autoclose-qa Used for automation to auto-close an issue Question/Not a bug
Projects
None yet
Development

No branches or pull requests

2 participants