CI failing on downstream Ansible 2.9 test during inventory tests

[geerlingguy]

SUMMARY

Our CI is failing currently (starting this morning) with the following exception:

The full traceback is:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 147, in run
    res = self._execute()
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 665, in _execute
    result = self._handler.run(task_vars=variables)
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/plugins/action/normal.py", line 46, in run
    result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/plugins/action/__init__.py", line 790, in _execute_module
    self._make_tmp_path()
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/plugins/action/__init__.py", line 358, in _make_tmp_path
    result = self._low_level_execute_command(cmd, sudoable=False)
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/plugins/action/__init__.py", line 1085, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/ansible/plugins/connection/kubectl.py", line 286, in exec_command
    local_cmd, censored_local_cmd = self._build_exec_cmd([self._play_context.executable, '-c', cmd])
ValueError: too many values to unpack (expected 2)
fatal: [inventory-7c8cd99447-rskmg_inventory]: FAILED! => {
    "msg": "Unexpected failure during module execution.",

This exception occurs during this task:

- debug: var=hostvars

Located here: https://github.com/ansible-collections/community.kubernetes/blob/main/molecule/default/converge.yml#L85

ISSUE TYPE

• Bug Report

COMPONENT NAME

molecule tests for downstream Ansible 2.9.

ANSIBLE VERSION

2.9

CONFIGURATION

See GitHub Actions workflow.

OS / ENVIRONMENT

GitHub Actions

[geerlingguy]

Ansible 2.9.14 was released yesterday, and that seems to have been the inflection point—CI wasn't failing until then: https://github.com/ansible/ansible/blob/stable-2.9/changelogs/CHANGELOG-v2.9.rst#id51

(Though I don't see any changes in the changelog that seem pertinent to our usage...)

[geerlingguy]

Added critical priority since this technically blocks any merges due to the fact that all CI builds are failing.

[Akasurde]

Fix for -
Stable-2.9 - ansible/ansible#72130
Stable-2.8 - ansible/ansible#72131

[tima]

This is a summary where I think we are on this issue.

A typo was made in backporting a CVE fix to the kubectl connection plugin in ansible/ansible that broke the k8s inventory plugin in this collection. @Akasurde has submitted PRs to fix the problem in the next releases (pending merge) of 2.8 and 2.9. So that is squared away.

Question: Why is this collection's inventory plugin using the kubectl connection plugin that ships with Ansible rather than the one that ships here in the collection? To me, this CI break shouldn't have been an issue otherwise. Thoughts?

[tima]

This line sets the connection plugin in the hostvars for each pod which is set to just kubectl (no namespacing). I think this is where things are going wrong for us in our CI. Can we force Ansible to use the connection plugin in this collection rather than what's built-in to Ansible? Is it as easy as using the FQCN like 'community.kubernetes.kubectl'?

[geerlingguy]

@tima - I'll attempt that quick fix in a PR in a moment.