Streaming multipart/form-data parser

streaming_form_data provides a Python parser for parsing multipart/form-data input chunks (the encoding used when submitting data over HTTP through HTML forms).

Testimonials

this speeds up file uploads to my Flask app by more than factor 10

Thanks a lot for your fix with streaming-form-data. I can finally upload gigabyte sized files at good speed and without memory filling up!

huge thanks to @siddhantgoel with his "streaming-form-data" that saves me from the slow file reads I get with @FastAPI!

Installation

$ pip install streaming-form-data

In case you prefer cloning the Github repository and installing manually, please note that main is the development branch, so stable is what you should be working with.

Usage

>>> from streaming_form_data import StreamingFormDataParser
>>> from streaming_form_data.targets import FileTarget, NullTarget, GCSTarget, S3Target, ValueTarget
>>>
>>> headers = {"Content-Type": "multipart/form-data; boundary=boundary"}
>>>
>>> parser = StreamingFormDataParser(headers=headers)
>>>
>>> parser.register("name", ValueTarget())
>>> parser.register("file-local", FileTarget("/path/to/file.txt"))
>>> parser.register("file-s3", S3Target("s3://bucket/path/to/key"))
>>> parser.register("file-gcs", GCSTarget("gs://bucket/path/to/key"))
>>> parser.register("discard-me", NullTarget())
>>>
>>> for chunk in request.body:
...     parser.data_received(chunk)
...
>>>

Documentation

Up-to-date documentation is available on Read the Docs.

Development

Please make sure you have Python 3.9+, poetry, and just installed.

Since this package includes a C extension, please make sure you have a working C compiler available. On Debian-based distros this usually means installing the build-essentials package.

1. Git clone the repository: git clone https://github.com/siddhantgoel/streaming-form-data

2. Install the packages required for development: poetry install

3. That's basically it. You should now be able to run the test suite: just test

Note that if you make any changes to Cython files (.pyx, .pxd, .pxi), you'll need to re-compile (just compile) and re-install streaming_form_data before you can test your changes.