Support proxy when sending/receiving events over websocket

[jakekaplan]

Describe the current behavior

Currently, Prefect uses the websockets package to facilitate sending events to the server. A known limitation of this package is that it does not support proxying through HTTP or SOCKS.

While Prefect has always used this package for client-side events, with the introduction of Prefect 3.x, the ability to send and receive events has become more critical to core Prefect functionality. This includes tasks such as recording client-side orchestrated task run data and resolving distributed futures.

Describe the proposed behavior

Users of Prefect who utilize a proxy should be able to access all of Prefect's functionality.

Possible solutions:

• Introduce websockets_proxy (a potential drop-in replacement with proxy support).
• Replace usage of the websockets package with websockets-client.
• Introduce an optional HTTP-only interface for sending and receiving events. This would likely lead to performance degradation, so other options should be exhausted first.

Example Use

No response

Additional context

No response

[desertaxle]

Another possible approach is to use the python-socks package in conjunction with websockets. websockets-proxy is a relatively thin layer around websockets and python-socks, so it looks like something that could be implemented within the prefect client. Also, it seems like websockets-client (along with a couple of other websocket libraries) depends on python-socks for proxy support.

I lean towards using python-socks directly and then falling back to websockets-client if that doesn't work, but I'm happy to hear other approaches!

[jakekaplan]

Prefect communicates with the server (self hosted or cloud) over regular http requests and websockets. As of this PR, Prefect supports connecting over an http proxy for websocket communication as well.

To enable communication over a proxy, set HTTP_PROXY/HTTPS_PROXY and prefect will respect the setting for both http requests and websockets.

Note: Your proxy will likely need explicit configuration updates to allow websocket connections. See here for an example with squid.