-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add support for NO_PROXY for websocket connections #16538
Conversation
CodSpeed Performance ReportMerging #16538 will not alter performanceComparing Summary
|
Adjusted the used proxy to be def parse_proxy_url(url: str) -> Tuple[ProxyType, str, int, Optional[str], Optional[str]]:
parsed = urlparse(url)
scheme = parsed.scheme
if scheme == 'socks5':
proxy_type = ProxyType.SOCKS5
elif scheme == 'socks4':
proxy_type = ProxyType.SOCKS4
elif scheme == 'http':
proxy_type = ProxyType.HTTP
else:
> raise ValueError(f'Invalid scheme component: {scheme}') # pragma: no cover
E ValueError: Invalid scheme component: https |
src/prefect/events/clients.py
Outdated
@@ -84,6 +84,20 @@ def events_out_socket_from_api_url(url: str): | |||
return http_to_ws(url) + "/events/out" | |||
|
|||
|
|||
def _host_matches_no_proxy(host: str) -> bool: | |||
"""Checks whether a given host matches NO_PROXY configuration. | |||
NO_PROXY entries with a leading dot are treated as domain suffixes, while all others are treated as exact matches. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm inclined to use from urllib.request import proxy_bypass
here to avoid having our own implementation if possible.
My reading of NO_PROXY is that it's either an exact match or substring (as opposed to just a suffix), which would conflict with this as written but would be supported by proxy_bypass
.
I'm not super familiar, did you see something that suggested it should work differently?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was looking for precisely this, also not super familiar here.
This looks indeed perfect, swapped it out and working as expected.
Pushed a change to swap out just now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jbw-vtl will need a review from a core maintainer to merge but this LGTM. Thanks for contributing the fix!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you @jbw-vtl !
In #16326 the team added support for HTTP_PROXY & HTTPS_PROXY for the websocket connection used by the agent to submit events (such as task runs).
However in some environments these proxies are necessary to route connection for the flow itself running (i.e. to connect to external in an air gapped environment).
Having the events routed through the proxy might mean they cannot reach the server.
This PR adds basic support for the NO_PROXY variable for the websocket connection, allowing to ignore the proxy for the server connection.
Checklist
<link to issue>
"[ ] If this pull request removes docs files, it includes redirect settings inmint.json
.This stemmed from the discussion around #15153 and closes #16537