-
Notifications
You must be signed in to change notification settings - Fork 460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
short-form dns query *nslookup kubernetes.default* not working #109
Comments
@kubernetes/sig-apps-misc @kubernetes/sig-network-misc This seems expected. @jwfang Have you set up the proper permissions for these services/accounts to communicate? |
@cmluciano thanks you. i made the following changes for kube-dns pod:
is there other things i need to do ? what confused me is that the short form svcName.Namespace doesn't work. i can't even query the short form in the same namespace, but nslookup kubernetes.default is in the official docs for checking if DNS is working correctly. my remaining questions is:
here is my /etc/resolv.conf for busybox. i expect kubernetes.default to will pick up the second search,
|
RBAC doesn't affect DNS queries at all... the kube-dns RBAC role was made in the kube 1.6 timeframe, and works with the version of kube-dns that was current at that time ( |
I would still check the kube-dns logs... it seems like the old image is likely making additional API calls that your updates to the role did not catch |
@liggitt thanks for help. i didn't find anything suspicious from my kube-dns logs. my kube-dns/kubedns logs look like this:
and here is logs from kube-dns/dnsmasq:
the cluster.local.svc.cluster.local seems wierd to me, but not sure. |
Can you post the YAML for your kube-dns pod to a gist? |
sure, @bowei ,here is my yaml from kubectl edit deployment kubedns -n kube-system: it's generated from: https://github.com/kubernetes-incubator/kubespray/blob/774c4d0d6fe2b5449432192ee2cde9c07ff1e897/roles/kubernetes-apps/ansible/templates/kubedns-deploy.yml. currently i am not using RBAC, but with my certs changes at kubernetes-sigs/kubespray#1351. kubenetes.default still doesn't resolved. i will try using a vanilla kubespray install and test, maybe it's due to how k8s is configured in kubespray. |
did a vanilla install and changed kube-dns/dnsmasq to the following:
still can't resolve short form svc1.ns1, only FQDN works. |
i guess i figure the cause of the problem somehow, when doing a kubedns.kube-system query from default namespace, got the following from dnsmasq log:
but doing a kubedns got me this:
seems my local busybox is not using the search config when it got dots in query. my busybox /etc/resolv.conf looks like this:
so it's really wierd that it didn't use the search domain for kubedns.kube-system. |
i am closing this. the short form works for alpine and tutum/dnsutils. |
in case someone encounter the same problem, i write my finding here.
kube-dns behaviour:
NOTE: short-form query such as kubernetes.default is unknown to kubedns
EDIT: from https://github.com/skynetservices/skydns/blob/f694f5637b31e2b9c9871fb396773d5d18b9309e/server/exchange.go#L29, it's not doing NSRotate. when no NSRotate, it always use the first name server first, and it only retry on connection error, for application error it just directly forward upstream error code.
so the short-form query works like this:
client behaviour:
for 5, different client seem to have differnt behaviour regards to differnt error from 3.
for busybox, it seem to be only append search for NXDOMAIN, and not append search for REFUSED;
and for alpine and tutum/dnsutils, it will append search for both NXDOMAIN and REFUSED.
my installation it's a bit unusually, although they have idential /etc/resolv.conf, the first name server behaviour differently on different node: some are not recusive and will return REFUSED, and the other
will return NXDOMAIN.
so i got this weird behaviour:
when kube-dns is on NXDOMAIN node, busybox nslookup testing works;
when kube-dns is on REFUSED node, busybox nslookup testing fails.
and alpine/tutum/dnsutils always works regards which node kube-dns is on.
so, when deploy kube-dns, you should ensure the first nameserver in your host /etc/resolv.conf works as expected.
=============== BELOW are original question ===============================
i enabled RBAC for my on-premise k8s cluster, but found cross-name space DNS query different from non-RBAC.
i didn't find any document for this behaviour, so this issue.
for no-RBAC:
svc1.ns1
;svc1.ns1
with namespace.but in RBAC, i have to use the FQDN:
svc1.ns1
from ns2;svc1.ns1
with namespace;from a busybox in default namespace, i got the following output:
here is my container info:
The text was updated successfully, but these errors were encountered: