Cannot stream large content from version 0.18.0

[khanetor]

Starting from version 0.18.0, my code for downloading large data file via stream does not work.

The code to download is taken straight from this example, except that the GET method is replaced with POST.

import tempfile
import httpx

with tempfile.NamedTemporaryFile() as download_file:
    url = "https://speed.hetzner.de/100MB.bin"
    with httpx.stream("GET", url, timeout=None) as response:
        for chunk in response.iter_bytes():
            download_file.write(chunk)

This exact example works without problem, but when I start to fetch data around 500MB, I start to see timeout error. Notice that I also specify timeout=None.

The error is as follow:

Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpx/_transports/default.py", line 61, in map_httpcore_exceptions
    yield
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpx/_transports/default.py", line 106, in __iter__
    for part in self._httpcore_stream:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_sync/connection_pool.py", line 62, in __iter__
    for chunk in self.stream:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_bytestreams.py", line 56, in __iter__
    for chunk in self._iterator:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_sync/http11.py", line 152, in _receive_response_data
    event = self._receive_event(timeout)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_sync/http11.py", line 169, in _receive_event
    data = self.socket.read(self.READ_NUM_BYTES, timeout)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_backends/sync.py", line 61, in read
    return self.sock.recv(n)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
    raise to_exc(exc) from None
httpcore.ReadTimeout: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "forecast.py", line 112, in <module>
    app()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/zoined/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "forecast.py", line 57, in main
    fetch(customer=customer, customer_token=customer_token,
  File "/Users/kha/Projects/Zoined/singularity/app/utilities/zapi.py", line 80, in fetch
    for chunk in response.iter_bytes():
  File "/opt/homebrew/Caskroom/miniforge/bas

This error is consistent both on Linux and Mac in my case.

[tomchristie]

Thanks for raising this. I confirmed by downloading a large file, and then disconnecting wi-fi midway through the download.
And, as you say, the request then raised a timeout error after 5 seconds. (Despite the timeout=None parameter.)

I've tracked this down to a bug introduced in this pull request #1577 and have now resolved in #1613. A new release should be incoming fairly shortly.

[khanetor]

Thank you!