-
Notifications
You must be signed in to change notification settings - Fork 292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Connecting to api server fails with HTTP/2 handshake error #1112
Comments
looks like server side rejected tls1.2 |
seems windows8 does not support tls1.3, is your kubenav running on same machine? |
@tg123 yes, my kubenav is on the same machine |
My first thought also was the TLS version problem, but as far as I can understand the dumps, SSL handshake completes sucessfully even thoughs it's tls1.2 (I think it's allowed on the api server). If that was not the case, I'd be receiving SSL errors, not HTTP/2, right? |
6443 -> you RST, which means server closed the connection |
Not sure what to test though, but I did open the {
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
"reason": "Forbidden",
"details": {},
"code": 403
} In the wireshark I see that the connection is done via TLS 1.3, and it seems to be keepalived. And I did run it from the same machine, yes. Should I check something else? |
Okay, I did the following from a c# program: HttpClientHandler handler = new HttpClientHandler() {
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
using (HttpClient client = new HttpClient(handler))
{
using (HttpResponseMessage result = await client.GetAsync("https://<apiserver-url>:6443"))
{
Console.WriteLine(result.StatusCode);
Console.WriteLine(await result.Content.ReadAsStringAsync());
}
} and it gives me the same HTTP response {
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
"reason": "Forbidden",
"details": {},
"code": 403
} And in wireshark I see that it's done via TLS 1.2 |
Ok, I am able to reproduce the problem using the following change to the code above: using (HttpRequestMessage req = new HttpRequestMessage(HttpMethod.Get, "https://<apiserver-url>:6443") {
Version = new Version(2, 0)
})
{
using (HttpResponseMessage result = await client.SendAsync(req)) So switching request to Http/2 causes the same problem as I experience with the client library. |
anything special with your api server, the sdk assumes it supports http2 and did not have n option to change it at the moment. |
don't think so. Just a vanilla K8s, bootstrapped via kubeadm. The only thing that comes to mind, is that we've set it up with |
I also have no luck in googling any HTTP/2 issues with kube api server, except for this, but it's from K3s, which we don't use. |
OK, some more info. I tried running my original app from Win 11 machine, and it does connect successfully using the same kubeconf file. So it seems that it's really not a HTTP/2 problem. |
could you please take a look at api server to see why it does not like windows 8.1 |
Yes, but I'm not sure what to look at. Maybe you could give some hints? I think, we have all TLS versions and all ciphers enabled at apiserver. |
https://kubernetes.io/docs/tasks/debug/debug-cluster/#control-plane-nodes LMK if it helps, so we can do something to improve sdk's compatibility |
It seems that there's a specific requirement in rfc7540 that says that in case of HTTP/2 over TLS1.2 there's a list of additionally forbidden ciphers. It looks that what I'm facing is that my client and server negotiate a cypher that is OK in general, but is not allowed for HTTP/2. So when client requests an HTTP/2 connection AFTER SSL handshake, api server rejects it because of inappropriate cipher used for the channel. The part of a problem is that So I'll be checking the --tls-cipher-suites option of api-server, but for now I have no luck finding a proper ciphers subset. Just for a reference, here's a win 8.1 ciphers list. |
good catch |
add some config to bypass force http2 |
Thanks for considering and implementing this feature! Hope to see a package release soon!
Hm, I think HTTP keepalive (not closing TCP connection after request completed) is available as an option for HTTP/1.1, see for example this. So you probably mean the option of sending PING frames, which is HTTP/2 specific?
Thanks for highlighting this! Will see if there's an upgrade possible. |
long discussion about the tcp keep alive is not easy to setup, thus, the sdk prefers https now |
close as disable http2 option added |
Just checked the new release, and it works! Finally! Thanks again! |
Describe the bug
I'm getting the following exception while connecting to my K8s cluster:
The code:
Whole stacktrace is:
Kubernetes C# SDK Client Version
9.0.38
Server Kubernetes Version
1.25.4
Dotnet Runtime Version
net6
Expected behavior
I expected to be able to connect to the api server
KubeConfig
my kubeconf file looks like this:
Where do you run your app with Kubernetes SDK (please complete the following information):
Additional context
I doubt that there's a problem with my kubeconf file, because both Lens and KubeNav have no problems connecting to the API server using the same file.
Here are screenshots from wireshark:
My app with the problem:
![image](https://user-images.githubusercontent.com/2208971/205495910-d74d9642-94a7-4bbe-8ff0-66fd1ad31dc0.png)
Kubenav with no problem:
![image](https://user-images.githubusercontent.com/2208971/205496063-c868a596-b0a4-47e8-a502-eb57a3c5f48c.png)
(black rects mask my machine, green ones mask api server)
The text was updated successfully, but these errors were encountered: