Connection Refused when using client behind coporate proxy to connect to AWS Cluster

[jueast08]

So I've been using the client for about a week, and I have to say I love it and what it can do. However, I've noticed that when using the client at work I receive the following errors despite the fact that all kubectl commands work :

2018-06-18 11:20:15,250 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff4dc6f69d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/namespaces/default/status 2018-06-18 11:20:15,254 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff4dc6f6910>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/namespaces/default/status 2018-06-18 11:20:15,258 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff4dc6f6a90>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/namespaces/default/status HTTPSConnectionPool(host='<ip_address>', port=443): Max retries exceeded with url: /api/v1/namespaces/default/status (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff4dc6f6a10>: Failed to establish a new connection: [Errno 111] Connection refused',))

I've tried using the default config.load_kube_config() function. I've also tried to set the proxy using :
conf = client.Configuration() conf.http_proxy_url = <proxy> config.load_kube_config(client_configuration=conf)
The latter command fixes the host as 'localhost' rather than the IP of my AWS cluster

[tomplus]

Can you show us your kubeconfig file ~/.kube/config (with removed certs/passwords/ips if it's neccessary) ? It's interesting that kubectl works and this library try to connect but the connection is refused.

[jueast08]

Weirdly enough, I went into the source code of the ApiClient to see if I could kick start the thing and gain more info about what the problem may be.

I was able to make the client work behind my proxy by doing the following in the init function of the api_client.py module:

def __init__(self, configuration=None, header_name=None, header_value=None, cookie=None): if configuration is None: configuration = Configuration() configuration.proxy = <proxy_url> # <---- I added my proxy url here to see if there would be a change in the result self.configuration = configuration ...

edit : The copy of the code looks pretty bad, but I just added to line 65 of the file api_client.py :
configuration.proxy = <proxy>

Setting the proxy via the specified methods or launching my code in an environment with the correctly proxy variables set had no effect, where as this "fixed" the problem immediately

[jueast08]

Ok. So I ended up doing the following (this time without editing the sources)

config.load_kube_config()
conf = client.Configuration()
conf.proxy = <my_proxy>
client.Configuration.set_default(conf)

Not sure if it was indicated in the docs to do a load_kube_config() and use that to solve the proxy issue, but I could not see anything of the kind

[abushoeb]

Check this #333 (comment)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Omegastick]

Ok. So I ended up doing the following (this time without editing the sources)

config.load_kube_config()
conf = client.Configuration()
conf.proxy = <my_proxy>
client.Configuration.set_default(conf)

Not sure if it was indicated in the docs to do a load_kube_config() and use that to solve the proxy issue, but I could not see anything of the kind

This doesn't work for me. Only changing the ApiClient source as described in the previous comment does.