Coredns breaks with ansible > 2.10.0

[cristicalin]

Environment:

• Cloud provider or hardware configuration:
  VMWare VM

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):

Linux 4.18.0-301.1.el8.x86_64 x86_64
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"

• Version of Ansible (ansible --version):

ansible 2.10.9
  config file = /root/kubespray/ansible.cfg
  configured module search path = ['/root/kubespray/library']
  ansible python module location = /root/venv/lib/python3.6/site-packages/ansible
  executable location = /root/venv/bin/ansible
  python version = 3.6.8 (default, Mar 19 2021, 05:13:41) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]

• Version of Python (python --version):

Python 3.6.8

Kubespray version (commit) (git rev-parse --short HEAD):

6e764936

Network plugin used:
Calico 3.18.2

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

localhost

Command used to invoke ansible:

ANSIBLE_ENABLE_TASK_DEBUGGER=True ansible-playbook -i inventory.ini cluster.yml -vvvvv

Output of ansible run:

Monday 10 May 2021  03:03:03 -0400 (0:00:00.300)       0:02:21.635 ************                                                                                                                                    
Using module file /root/kubespray/library/kube.py                                                                                                                                                                  
Pipelining is enabled.                                                                                                                                                                                             
<centos-stream.kaveman.intra> ESTABLISH LOCAL CONNECTION FOR USER: root                                                                                                                                            
<centos-stream.kaveman.intra> EXEC /bin/sh -c 'ALL_PROXY='"'"''"'"' FTP_PROXY='"'"''"'"' HTTPS_PROXY='"'"''"'"' HTTP_PROXY='"'"''"'"' NO_PROXY='"'"''"'"' all_proxy='"'"''"'"' ftp_proxy='"'"''"'"' http_proxy='"'$
''"'"' https_proxy='"'"''"'"' no_proxy='"'"''"'"' /usr/libexec/platform-python && sleep 0'                                                                                   
                                                                                                                                                                   
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm Kube-DNS service] *******************************************************************************************************************************$
task path: /root/kubespray/roles/kubernetes-apps/ansible/tasks/cleanup_dns.yml:23                                                                    
ok: [centos-stream.kaveman.intra] => {                                                                
    "changed": false,                                                                                                                                              
    "invocation": {                                                                                                                   
        "module_args": {                                                                                                                                           
            "all": false,                                                                 
            "filename": null,                                                                                                                                                                                     
            "force": false,                                                                                                                                                                                       
            "kubectl": "/usr/local/bin/kubectl",                                                                                      
            "label": null,                                                                                                               
            "log_level": 0,                                                                           
            "name": "kube-dns",                                                                                     
            "namespace": "kube-system",                                                                                                                                                                           
            "recursive": false,                                                                                                                                         
            "resource": "svc",                                                                                                                     
            "server": null,                                                                                                                                                
            "state": "absent",                                                                                                                                                                                     
            "wait": false                                                                 
        }                                                                                               
    },                                                                                                                                                                                                             
    "msg": "success: service \"kube-dns\" deleted"                               
}                                                                                                                                                                                                                 
ERROR! 'item' is undefined                                    

Anything else do we need to know:

It seems ansible 2.10 and above breaks with constructs like the sample below which is used in the coredns role:

- hosts: localhost
  tasks:
    - name: abcd
      action: "{{ item.module }}"
      args:
        var: ansible_distribution
      loop:
        - { module: debug }

From what I can tell ansible 2.10 no longer allows jinja templating for actions, in order to move to newer ansible versions we would need to update the logic in: https://github.com/kubernetes-sigs/kubespray/blob/master/roles/kubernetes-apps/ansible/tasks/coredns.yml#L2-L25

[floryut]

Thanks for the complete report, we could split this in 2 actions (template/copy) without templating the "action" part

[floryut]

Maybe something like that

- name: Kubernetes Apps | Lay Down CoreDNS templates
  template:
    src: "{{ item }}.j2{% endif %}"
    dest: "{{ kube_config_dir }}/{{ item }}"
  loop:
    - coredns-config.yml
    - coredns-deployment.yml
    - coredns-svc.yml
    - dns-autoscaler.yml
  register: coredns_manifests
  vars:
    clusterIP: "{{ skydns_server }}"
  when:
    - dns_mode in ['coredns', 'coredns_dual']
    - inventory_hostname == groups['kube_control_plane'][0]
  tags:
    - coredns

- name: Kubernetes Apps | Lay Down CoreDNS files
  copy:
    src: "{{ item }}"
    dest: "{{ kube_config_dir }}/{{ item }}"
  loop:
    - coredns-sa.yml
    - coredns-clusterrole.yml
    - coredns-clusterrolebinding.yml
    - dns-autoscaler-sa.yml
    - dns-autoscaler-clusterrole.yml
    - dns-autoscaler-clusterrolebinding.yml
  register: coredns_files
  vars:
    clusterIP: "{{ skydns_server }}"
  when:
    - dns_mode in ['coredns', 'coredns_dual']
    - inventory_hostname == groups['kube_control_plane'][0]
  tags:
    - coredns

(with adding the new coredns_files to roles/kubernetes-apps/ansible/tasks/main.yml:39)

[cristicalin]

@floryut thanks, I used your proposal with a slight variation to preserve context