Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allows users to have more control on DNS #9270

Merged
merged 1 commit into from
Sep 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/dns-stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,23 @@ ndots value to be used in ``/etc/resolv.conf``
It is important to note that multiple search domains combined with high ``ndots``
values lead to poor performance of DNS stack, so please choose it wisely.

## dns_timeout

timeout value to be used in ``/etc/resolv.conf``

## dns_attempts

attempts value to be used in ``/etc/resolv.conf``

### searchdomains

Custom search domains to be added in addition to the cluster search domains (``default.svc.{{ dns_domain }}, svc.{{ dns_domain }}``).

Most Linux systems limit the total number of search domains to 6 and the total length of all search domains
to 256 characters. Depending on the length of ``dns_domain``, you're limited to less than the total limit.

`remove_default_searchdomains: true` will remove the default cluster search domains.

Please note that ``resolvconf_mode: docker_dns`` will automatically add your systems search domains as
additional search domains. Please take this into the accounts for the limits.

Expand Down Expand Up @@ -270,7 +280,8 @@ nodelocaldns_secondary_skew_seconds: 5

* the ``searchdomains`` have a limitation of a 6 names and 256 chars
length. Due to default ``svc, default.svc`` subdomains, the actual
limits are a 4 names and 239 chars respectively.
limits are a 4 names and 239 chars respectively. If `remove_default_searchdomains: true`
added you are back to 6 names.

* the ``nameservers`` have a limitation of a 3 servers, although there
is a way to mitigate that with the ``upstream_dns_servers``,
Expand Down
2 changes: 2 additions & 0 deletions docs/vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Some variables of note include:
* *kube_proxy_mode* - Changes k8s proxy mode to iptables mode
* *kube_version* - Specify a given Kubernetes version
* *searchdomains* - Array of DNS domains to search when looking up hostnames
* *remove_default_searchdomains* - Boolean that removes the default searchdomain
* *nameservers* - Array of nameservers to use for DNS lookup
* *preinstall_selinux_state* - Set selinux state, permitted values are permissive, enforcing and disabled.

Expand Down Expand Up @@ -166,6 +167,7 @@ variables to match your requirements.
addition to Kubespray deployed DNS
* *nameservers* - Array of DNS servers configured for use by hosts
* *searchdomains* - Array of up to 4 search domains
* *remove_default_searchdomains* - Boolean. If enabled, `searchdomains` variable can hold 6 search domains.
* *dns_etchosts* - Content of hosts file for coredns and nodelocaldns

For more information, see [DNS
Expand Down
8 changes: 8 additions & 0 deletions inventory/sample/group_vars/k8s_cluster/k8s-cluster.yml
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,14 @@ kube_encrypt_secret_data: false
cluster_name: cluster.local
# Subdomains of DNS domain to be resolved via /etc/resolv.conf for hostnet pods
ndots: 2
# dns_timeout: 2
# dns_attempts: 2
# Custom search domains to be added in addition to the default cluster search domains
# searchdomains:
# - svc.{{ cluster_name }}
# - default.svc.{{ cluster_name }}
# Remove default cluster search domains (``default.svc.{{ dns_domain }}, svc.{{ dns_domain }}``).
# remove_default_searchdomains: false
# Can be coredns, coredns_dual, manual or none
dns_mode: coredns
# Set manual server if using a custom cluster DNS server
Expand Down
13 changes: 9 additions & 4 deletions roles/kubernetes/preinstall/tasks/0040-set_facts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,17 @@
changed_when: false
check_mode: no

- name: set default dns if remove_default_searchdomains is false
set_fact:
default_searchdomains: ["default.svc.{{ dns_domain }}", "svc.{{ dns_domain }}"]
when: not remove_default_searchdomains|default()|bool or (remove_default_searchdomains|default()|bool and searchdomains|default([])|length==0)

- name: set dns facts
set_fact:
resolvconf: >-
{%- if resolvconf.rc == 0 and resolvconfd_path.stat.isdir is defined and resolvconfd_path.stat.isdir -%}true{%- else -%}false{%- endif -%}
bogus_domains: |-
{% for d in [ 'default.svc.' + dns_domain, 'svc.' + dns_domain ] + searchdomains|default([]) -%}
{% for d in default_searchdomains|default([]) + searchdomains|default([]) -%}
{{ dns_domain }}.{{ d }}./{{ d }}.{{ d }}./com.{{ d }}./
{%- endfor %}
cloud_resolver: "{{ ['169.254.169.254'] if cloud_provider is defined and cloud_provider == 'gce' else
Expand Down Expand Up @@ -169,11 +174,11 @@
- name: generate search domains to resolvconf
set_fact:
searchentries:
search {{ ([ 'default.svc.' + dns_domain, 'svc.' + dns_domain ] + searchdomains|default([])) | join(' ') }}
search {{ (default_searchdomains|default([]) + searchdomains|default([])) | join(' ') }}
domainentry:
domain {{ dns_domain }}
supersede_search:
supersede domain-search "{{ ([ 'default.svc.' + dns_domain, 'svc.' + dns_domain ] + searchdomains|default([])) | join('", "') }}";
supersede domain-search "{{ (default_searchdomains|default([]) + searchdomains|default([])) | join('", "') }}";
supersede_domain:
supersede domain-name "{{ dns_domain }}";

Expand All @@ -196,7 +201,7 @@
- name: generate nameservers for resolvconf, including cluster DNS
set_fact:
nameserverentries: |-
{{ ( ( [nodelocaldns_ip] if enable_nodelocaldns else []) + coredns_server|d([]) + nameservers|d([]) + cloud_resolver|d([]) + configured_nameservers|d([])) | unique | join(',') }}
{{ (([nodelocaldns_ip] if enable_nodelocaldns else []) + (coredns_server|d([]) if not enable_nodelocaldns else []) + nameservers|d([]) + cloud_resolver|d([]) + configured_nameservers|d([])) | unique | join(',') }}
supersede_nameserver:
supersede domain-name-servers {{ ( coredns_server|d([]) + nameservers|d([]) + cloud_resolver|d([])) | unique | join(', ') }};
when: not dns_early or dns_late
Expand Down
4 changes: 1 addition & 3 deletions roles/kubernetes/preinstall/tasks/0060-resolvconf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,7 @@
{% for item in nameserverentries.split(',') %}
nameserver {{ item }}
{% endfor %}
options ndots:{{ ndots }}
options timeout:2
options attempts:2
options ndots:{{ ndots }} timeout:{{ dns_timeout|default('2') }} attempts:{{ dns_attempts|default('2') }}
state: present
insertbefore: BOF
create: yes
Expand Down
9 changes: 7 additions & 2 deletions roles/kubernetes/preinstall/tasks/0063-networkmanager-dns.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,17 @@
backup: yes
notify: Preinstall | update resolvconf for networkmanager

- name: set default dns if remove_default_searchdomains is false
set_fact:
default_searchdomains: ["default.svc.{{ dns_domain }}", "svc.{{ dns_domain }}"]
when: not remove_default_searchdomains|default()|bool or (remove_default_searchdomains|default()|bool and searchdomains|default([])|length==0)

- name: NetworkManager | Add DNS search to NM configuration
ini_file:
path: /etc/NetworkManager/conf.d/dns.conf
section: global-dns
option: searches
value: "{{ ([ 'default.svc.' + dns_domain, 'svc.' + dns_domain ] + searchdomains|default([])) | join(',') }}"
value: "{{ (default_searchdomains|default([]) + searchdomains|default([])) | join(',') }}"
mode: '0600'
backup: yes
notify: Preinstall | update resolvconf for networkmanager
Expand All @@ -24,7 +29,7 @@
path: /etc/NetworkManager/conf.d/dns.conf
section: global-dns
option: options
value: "ndots:{{ ndots }};timeout:2;attempts:2;"
value: "ndots:{{ ndots }};timeout:{{ dns_timeout|default('2') }};attempts:{{ dns_attempts|default('2') }};"
mode: '0600'
backup: yes
notify: Preinstall | update resolvconf for networkmanager
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
if [ $reason = "BOUND" ]; then
if [ -n "$new_domain_search" -o -n "$new_domain_name_servers" ]; then
RESOLV_CONF=$(cat /etc/resolv.conf | sed -r '/^options (timeout|attempts|ndots).*$/d')
OPTIONS="options timeout:2\noptions attempts:2\noptions ndots:{{ ndots }}"
OPTIONS="options timeout:{{ dns_timeout|default('2') }} attempts:{{ dns_attempts|default('2') }} ndots:{{ ndots }}"

printf "%b\n" "$RESOLV_CONF\n$OPTIONS" > /etc/resolv.conf
fi
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
zdnsupdate_config() {
if [ -n "$new_domain_search" -o -n "$new_domain_name_servers" ]; then
RESOLV_CONF=$(cat /etc/resolv.conf | sed -r '/^options (timeout|attempts|ndots).*$/d')
OPTIONS="options timeout:2\noptions attempts:2\noptions ndots:{{ ndots }}"
OPTIONS="options timeout:{{ dns_timeout|default('2') }} attempts:{{ dns_attempts|default('2') }} ndots:{{ ndots }}"

echo -e "$RESOLV_CONF\n$OPTIONS" > /etc/resolv.conf
fi
Expand Down
4 changes: 4 additions & 0 deletions roles/kubernetes/preinstall/templates/resolved.conf.j2
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@
DNS={{ ([nodelocaldns_ip] if enable_nodelocaldns else coredns_server )| list | join(' ') }}
{% endif %}
FallbackDNS={{ ( upstream_dns_servers|d([]) + nameservers|d([]) + cloud_resolver|d([])) | unique | join(' ') }}
{% if remove_default_searchdomains is sameas false or (remove_default_searchdomains is sameas true and searchdomains|default([])|length==0)%}
Domains={{ ([ 'default.svc.' + dns_domain, 'svc.' + dns_domain ] + searchdomains|default([])) | join(' ') }}
{% else %}
Domains={{ searchdomains|default([]) | join(' ') }}
{% endif %}
#LLMNR=no
#MulticastDNS=no
DNSSEC=no
Expand Down