Add a graceful retry logic for intermittent network failure when downloading packages

[abn]

In certain cases, it is possible that the network might not be reliable and be prone to outages. In these cases, it would be great if poetry's artifact download logic would recover gracefully. Ideally, with both an exponential back-off as well as a retry attempts.

A sample output when this happens.

  • Installing html5lib (1.1)

ConnectionError

('Connection aborted.', error(54, 'Connection reset by peer'))

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 183, in _execute_operation
    result = self._do_execute_operation(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 257, in _do_execute_operation
    result = getattr(self, '_execute_{}'.format(method))(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 394, in _execute_install
    return self._install(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 420, in _install
    archive = self._download(operation)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 574, in _download
    return self._download_link(operation, link)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 583, in _download_link
    archive = self._download_archive(operation, link)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/executor.py", line 609, in _download_archive
    response = self._authenticator.request('get', link.url, stream=True)
  File "/usr/local/lib/python2.7/site-packages/poetry/installation/authenticator.py", line 66, in request
    resp = session.send(prepared_request, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)


  • Installing httpretty (0.9.7)

ERROR: InvocationError for command /usr/local/bin/poetry install -vv (exited with code 1)

The relevant logic could be implemented here.

poetry/poetry/installation/authenticator.py

Lines 37 to 70 in cae154c

	def request(self, method, url, **kwargs): # type: (str, str, Any) -> Response
	from requests import Request # noqa
	from requests.auth import HTTPBasicAuth
	request = Request(method, url)
	username, password = self._get_credentials_for_url(url)
	if username is not None and password is not None:
	request = HTTPBasicAuth(username, password)(request)
	session = self.session
	prepared_request = session.prepare_request(request)
	proxies = kwargs.get("proxies", {})
	stream = kwargs.get("stream")
	verify = kwargs.get("verify")
	cert = kwargs.get("cert")
	settings = session.merge_environment_settings(
	prepared_request.url, proxies, stream, verify, cert
	)
	# Send the request.
	send_kwargs = {
	"timeout": kwargs.get("timeout"),
	"allow_redirects": kwargs.get("allow_redirects", True),
	}
	send_kwargs.update(settings)
	resp = session.send(prepared_request, **send_kwargs)
	resp.raise_for_status()
	return resp

[danmur]

Strangely I was about to file an issue for this and saw this near the top. Same problem at my work, the corp proxies are a touch flaky and will occasionally fail a connection, with a long set of dependencies to resolve you're almost assured to get a network failure which derails the whole process.

I have a fix that I've been using for forever now, using urllib3.Retry. I'm still testing it against poetry 1.10.0 so I haven't created a PR yet but it's worked solidly from 0.12.something up to 1.9.0:

https://github.com/python-poetry/poetry/compare/master...danmur:retry-bad-status-line?expand=1

If I don't get any critical feedback in the next few days, and it looks like it's working ok, I'll make a PR for it.

[danmur]

Woops, didn't realise 1.0.10 was already out! Anyhow, it works fine against master.

[abn]

@danmur while the urllib3 retry will work for http errors, i am not sure if that approach will work for this particular issue as this is raising a socket error. The retry might be a good improvement, it will not resolve this particular issue I reckon.

[danmur]

Hmm yeah that makes sense. The error I was getting was a bad status line, at the http level. Thanks, I'll do some testing on the weekend and confirm, but yes will probably need more.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.