Cannot use kubelet argument --hostname-override

[siredmar]

/kind bug

What steps did you take and what happened:
While setting up a cluster i need the nodes to be part of a custom wireguard VPN (netbird). In order to let them communicate with other nodes that are also connected to the VPN i need the hostnames to be resolvable by the VPN's DNS with the domain netbird.cloud.
So i'm trying to set a FQDN name to the node. I'm added this command to the preKubeadmCommands:

echo "KUBELET_EXTRA_ARGS=--hostname-override=$(hostname).netbird.cloud" > /etc/default/kubelet

However the kubelet won't start up correctly. These are the errors i get from kubelet

certificate_manager.go:488] kubernetes.io/kube-apiserver-client-kubelet: certificate request was not signed: cannot watch on the certificate signing request: certificate signing request is denied, reason: CSRValidationFailed, message: Validation by cluster-api-provider-hetzner failed: unexpected subject actual=pkix.Name{Country:[]string(nil), Organization:[]string{"system:nodes"}, OrganizationalUnit:[]string(nil), Locality:[]string(nil), Province:[]string(nil), StreetAddress:[]string(nil), PostalCode:[]string(nil), SerialNumber:"", CommonName:"system:node:dog-control-plane-zmgqq.netbird.cloud", Names:[]pkix.AttributeTypeAndValue{pkix.AttributeTypeAndValue{Type:asn1.ObjectIdentifier{2, 5, 4, 10}, Value:"system:nodes"}, pkix.AttributeTypeAndValue{Type:asn1.ObjectIdentifier{2, 5, 4, 3}, Value:"system:node:dog-control-plane-zmgqq.netbird.cloud"}}, ExtraNames:[]pkix.AttributeTypeAndValue(nil)}, expected=pkix.Name{Country:[]string(nil), Organization:[]string{"system:nodes"}, OrganizationalUnit:[]string(nil), Locality:[]string(nil), Province:[]string(nil), StreetAddress:[]string(nil), PostalCode:[]string(nil), SerialNumber:"", CommonName:"system:node:dog-control-plane-zmgqq", Names:[]pkix.AttributeTypeAndValue{pkix.AttributeTypeAndValue{Type:asn1.ObjectIdentifier{2, 5, 4, 10}, Value:"system:nodes"}, pkix.AttributeTypeAndValue{Type:asn1.ObjectIdentifier{2, 5, 4, 3}, Value:"system:node:dog-control-plane-zmgqq"}}, ExtraNames:[]pkix.AttributeTypeAndValue(nil)}

and

kubelet_node_status.go:93] "Unable to register node with API server" err="nodes \"dog-control-plane-zmgqq.netbird.cloud\" is forbidden: node \"dog-control-plane-zmgqq\" is not allowed to modify node \"dog-control-plane-zmgqq.netbird.cloud\"" node="dog-control-plane-zmgqq.netbird.cloud"

What did you expect to happen:
I'd expect that i could change the node name by using the --hostname-override argument.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• cluster-api-provider-hetzner version: v1.0.0-beta.22
• Kubernetes version: (use kubectl version): 1.22.17
• OS (e.g. from /etc/os-release): Ubuntu 22.04

[batistein]

This is currently not supported.
There are security mechanism in place which prevent overriding the hostname on the node (e.g csr controller)
I'm also not sure if this is supported by cluster-api as the hostname is also used in different other aspects of the cluster lifecycle.

[siredmar]

@batistein so you say that i don't have any chance of naming the nodes with a FQDN like .mydomain.com?

[apricote]

There is also going to be issues with the cloud-controller-manager, which relies on the assumption that hostname == name of server in API to set the Node.Spec.ProviderID.

---

I am not sure which naming requirements Machine objects follow or how easily this can be configured in CAPI (with random suffixes and such). I saw that 1.5.2 release has support for customizing the naming of created objects in ClusterClass: kubernetes-sigs/cluster-api#9340