How to use only http in watch?

[liziwl]

I have some problem using websocket, however using pure http is working well.

Are there any config to set watcher using only http protocol?

[rohanKanojia]

KubernetesClient seems to use http watch if it fails to start websocket initially. What behavior are you observing?

[liziwl]

I use official client works fine. But I use this client got this exception.

io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.

	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:103)
	at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:97)
	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.lambda$run$2(WatchConnectionManager.java:133)
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
	at io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl$BuilderImpl$1.onFailure(OkHttpWebSocketImpl.java:71)
	at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.kt:592)
	at okhttp3.internal.ws.RealWebSocket$connect$1.onFailure(RealWebSocket.kt:202)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:525)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
	Suppressed: java.lang.Throwable: waiting here
		at io.fabric8.kubernetes.client.utils.Utils.waitUntilReady(Utils.java:168)
		at io.fabric8.kubernetes.client.utils.Utils.waitUntilReadyOrFail(Utils.java:179)
		at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.waitUntilReady(WatchConnectionManager.java:96)
		at io.fabric8.kubernetes.client.dsl.base.BaseOperation.watch(BaseOperation.java:572)
		at io.fabric8.kubernetes.client.dsl.base.BaseOperation.watch(BaseOperation.java:83)

Caused by: java.net.SocketTimeoutException: timeout
	at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143)
	at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162)
	at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:335)
	at okio.RealBufferedSource.indexOf(RealBufferedSource.kt:427)
	at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:320)
	at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29)
	at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:178)
	at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)

Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at okio.InputStreamSource.read(JvmOkio.kt:90)
	at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:129)
	... 22 more

[liziwl]

ok, I manually add an interceptor to drop header "Connection", "Upgrade", then I can use http.

[manusa]

We should probably look into automatically falling back in case there is a timeout when upgrading to websocket.

[stale]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[shawkins]

Another major difference with fabric8 vs client-go / kubectl is the use of websocket watches. There is still a yet to be determined issue we seem to be having with websocket watches not closing properly. It would probably be good to have a mechanism for control whether http or websocket watches are used as it may be applicable in general as well.

[manusa]

Given the amount of non-deterministic problems in informers related to watches and their underlying websocket connection. And considering that client-go is hard killing the connection after ~15 minutes + jitter (AFAIR), we might want to reverse the current default. i.e. try to acquire an HTTP watch connection or fallback to websocket, since HTTP connections are overall more predictable and consistent for all of the HTTP client implementations. It would also make it easier to create tests.
I'm not sure that this approach (short-lived HTTP) can bring any disadvantage as compared to using websockets, but I can't think of any ATM.

[shawkins]

And considering that client-go is hard killing the connection after ~15 minutes + jitter (AFAIR)

It's 5-10 minutes.

I'm not sure that this approach (short-lived HTTP) can bring any disadvantage as compared to using websockets, but I can't think of any ATM.

Watching some of the progress upstream it's been surprising the lack of support / progress around websockets.

So my next step here is to look at adding a config property to control what style watches to use. For websockets I'm hoping we'll get some feedback on how frequently the close issue is happening and with which clients - #5189 (comment) - it could be 5-10 minutes for recylcing websockets is too aggressive and/or we may want to tighten up the 1 minute timeout down in the httpclient layer for okhttp, jetty, and jdk (that just came from the okhttp implementation really. the spec language goes something like "the client may disconnect after a reasonable amount of time after sending close).