HTTP streaming response support for Azure functions [In Public Preview Now]

[nitinl]

Is there support for real-time streaming HttpResponse from Azure function?

This question has been asked before, and the solution looks exactly like what is needed. However, func.HttpResponse object has no attribute 'write'.

[YunchuWang]

@nitinl thanks for following up! we are releasing a new worker version which supports http streaming req/response soon. stay tuned!

[rgn15996]

Any updates on ETA? Everyone wants to return nice streamed responses from LLMs!

[zhoxing-ms]

+1

[RoyceAroc]

+1

[DJSaunders1997]

+1

[eavanvalkenburg]

+1

[bastienjalbert]

Any news about current implementation status ?

[tyler-suard-parker]

@YunchuWang Sorry to bother you about this again, but my company's project is looking to stream response from an Azure functions app. Is that supported yet?

[fabge]

while you might have to wait a while until this is implemented, you could actually just use Azure Web PubSub as intermediary and stream responses this way.
e.g. a streaming response from the server could look like this:

1. user connects to websocket (Azure Web PubSub) using a /connect endpoint
2. user sends a message to /message endpoint
3. azure function receives the message and uses the azure pub sub sdk (e.g. for python) to stream the response via Azure Web PubSub to the user
4. user receives the message via the existing websocket connection

[waynebaby]

I am using .net, and i am confusing if i can use streaming output in .net version or it will ship with python features together?

[bastienjalbert]

It seems to be working with .net function already

[RoyceAroc]

when is the expected release date?

[andredewes]

@YunchuWang Is there any rough estimate when this will be released?

[YunchuWang]

Hi everyone, thank you for your attention. We are actively working on supporting this feature for python function as one of top priorities.
The latest update on timeline is as below:

• Prototyping/Design completed internally
• PRs merged and ready for release.
• Public preview: around Microsoft build event time (May 21-23)

If there is any update on the timeline, i will share it here.

update1: feature is being released and will be in next azure function runtime release today
update2: we are preparing for public preview once released and will be out soon. will keep you updated.
update3: no need to sign up and instructions to enable http streaming will be given in public preview announcements along with samples
update4: it will be released into public preview around Microsoft build event time (May 21-23)
update5:
Hi everyone, thank you for the attention and interest. Great news, the http streaming feature is released in public preview today! Please feel free to check it out following the announcement https://azure.microsoft.com/en-us/updates/public-preview-azure-functions-support-for-http-streams-in-python/ and let us know your experience/feedback/issues :)!

[alan-czysz]

Any updates for the timeline? Really looking forward to using streaming responses!

[habibregask86]

released preview/beta version

[mycaule]

I get this error while running the extension with Python 3.11 on a macOS Intel.
and latest versions of

azure-functions = "^1.20.0"
azurefunctions-extensions-http-fastapi = "^1.0.0b1"

Exception: FunctionLoadError: cannot load the stream_count function: 
'req' binding type "httpTrigger" and dataType "None" in function.json 
do not match the corresponding function parameter's Python type annotation "Request"

What should I put in function.json ?

[jvdalen]

@mycaule I'm having the same issue. After a lot of testing it seems that once you import
from azurefunctions.extensions.http.fastapi import Request, StreamingResponse

you MUST use req: Request and not req: Func.httprequest and replace all with the fastapi versions of the functions.

I can't seem to find a way to mix and match the fastapi and azurefunction request types in one function app, and am not good enough of a coder to understanding how the bindings work and why that's an issue. Love for someone to clarify.

[mycaule]

Thanks, finally I didn't use StreamingResponse from FastAPI, because of my syntax problems, but also lack of support in the main Azure Functions package.

What I did is I implemented a WebSocket server somewhere else in my architecture, not running with App Services, and sent websockets using the python-socketio emit() function. There are sample architectures in the documentation using Azure Web PubSub, but this is essentially the same as running a regular websocket server on App Service, at the difference you have to handle the hosting and scaling yourself.

I would recommend websockets for someone who needs a real time architecture.