Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too much data for declared Content-Length when passing string with non-ascii characters via files parameter #1482

Closed
joshuahlang opened this issue Feb 24, 2021 · 1 comment · Fixed by #1537
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@joshuahlang
Copy link

Checklist

  • [x ] The bug is reproducible against the latest release and/or master.
  • [ x] There are no similar issues or pull requests to fix it yet.

Describe the bug

h11._util.LocalProtocolError: Too much data for declared Content-Length is raised when passing string with non-ascii characters via files parameter.

To reproduce

import httpx

response = httpx.post(
    "https://httpbin.org/post",
    files={
        'upload-file': ('example.txt', '\u00E9', 'text/plain; charset=utf-8')
    }
)
response.raise_for_status()
print(response.read())

-->

Expected behavior

An exception not be raised and the content length properly computed

Actual behavior

An exception is raised.

Debugging material

Traceback (most recent call last):
  File "/***redacted***/httpx_bug.py", line 5, in <module>
    response = httpx.post(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_api.py", line 296, in post
    return request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_api.py", line 93, in request
    return client.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 733, in request
    return self.send(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 767, in send
    response = self._send_handling_auth(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 805, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 837, in _send_handling_redirects
    response = self._send_single_request(request, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 861, in _send_single_request
    (status_code, headers, stream, ext) = transport.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 218, in request
    response = connection.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 106, in request
    return self.connection.request(method, url, headers, stream, ext)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 66, in request
    self._send_request_body(stream, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 112, in _send_request_body
    self._send_event(event, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 123, in _send_event
    bytes_to_send = self.h11_state.send(event)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_connection.py", line 501, in send_with_data_passthrough
    writer(event, data_list.append)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_writers.py", line 58, in __call__
    self.send_data(event.data, write)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_writers.py", line 78, in send_data
    raise LocalProtocolError("Too much data for declared Content-Length")
h11._util.LocalProtocolError: Too much data for declared Content-Length

-->

Environment

  • OS: macOS
  • Python version: Python 3.9.1
  • HTTPX version: 0.16.1
  • Async environment: n/a
  • HTTP proxy: no
  • Custom certificates: no

Additional context

n/a

@j178
Copy link
Member

j178 commented Feb 24, 2021

Looks like we calculate Content-Length using Unicode length:

if isinstance(self.file, (str, bytes)):
return len(headers) + len(self.file)

but sends with bytes:

httpx/httpx/_multipart.py

Lines 122 to 123 in 0f280af

if isinstance(self.file, (str, bytes)):
yield to_bytes(self.file)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
2 participants