Standardize of tests/requirements.yml

[ssbarnea]

Summary

Apparently we currently have 18/19 collections that have a tests/requirements.yml file that is not loaded by any tool and that is not compatible with our official requirements.yml schema.

Current unsupported format looks inspired from old requirements v1 format:

---
integration_tests_dependencies:
 - community.general
 - amazon.aws >= 3.0.0
unit_tests_dependencies: []

In order to improve the development experience we should aim to find an answer to the following questions:

• Can we avoid using two different file formats and schemas for two files with identical names?
• Could we extend the existing root requirements.yml to allow it to specify test requirements? Obviously that this means that ansible-galaxy or hub should just ignore the unknown content instead of choking when encountering it.
• Can we maybe make use of tags to mark dependencies that are used only for testing?
• Do we need more than unit and integration type of dependencies?
• What about roles? Current format does not allow users to define any standalone roles as being required for testing only.

Now is probably the right time to make suggestions and get these right and hopefully future proof.

Additional Information

Possible impact on other tools:

• ansible-galaxy command seems to not be able to allow adding any keys in addition to roles: and collections: on a requirements.yml file. That is big bad because it does not allow us to extend the file for incorporating test requirements via new keys. If we try to add special tags for test requirements, they will install with older versions of ansible-galaxy that do not know to skip these tags. Bit bad, I wonder if core team can help use with some ideas here.
• ansible-hub upload (TBD)
• ansible-builder v3 makes indirect use of official requirements.yml files
• ansible-lint and molecule are easier to fix regardless what we decide

---
roles:
  - name: geerlingguy.mysql
    src: https://github.com/geerlingguy/ansible-role-mysql
    version: 4.2.0
    tags: [test, integration] # <-- ansible-galaxy ignores these

collections:
  - name: https://galaxy.ansible.com/download/community-molecule-0.1.0.tar.gz
    tags: [test] # <-- ansible-galaxy ignores these

[bcoca]

simplest solution is not to alter any formats, just move to sudirs:

test/units/requirements.yml
test/integration/requirements.yml

And use existing supported ansible-galaxy format

[ssbarnea]

@felixfontein I think that the proposal above has very good selling points. We will only have to split the current 18 files into two other files and the problem would be sorted. All files will have the same file format and tools using them might decide to ignore some of them.

[felixfontein]

A Galaxy dependency file per directory sounds like a good idea.

Is there a standard directory for molecule tests in collections BTW?

[ssbarnea]

I do have a PR in progress for changing linter/compat and making it recognize invalid requirements.yml file with a message like:

tests/requirements.yml file is not a valid Ansible requirements file. Only 'roles' and 'collections' keys are allowed at root level. Recognized valid locations are:
requirements.yml
roles/requirements.yml
collections/requirements.yml
tests/requirements.yml
tests/integration/requirements.yml
tests/unit/requirements.yml

We will have to fix those community collections manually to remove the undocumented/unsupported "integration_tests_dependencies" bits, so people will not get confused. I will do PRs for some of them, lucky we don't have so many.

[felixfontein]

I created some more PRs for all collections I have locally checked out.

[mariolenz]

I've had a look at community.vmware and I've seen that there is also a test-requirements.txt in the root folder.

@bcoca @felixfontein @ssbarnea Is this a file we should aim to get rid of, too? There are quite some collections in ansible-collections that have this file.

[ssbarnea]

I agree with @mariolenz but we probably need to do this in gradual steps. For example I want to add an entire documentation page into the linter about where to put requirements, so we can tell people to follow it.

Regarding requirements.txt -- these are very different than ansible ones (.yml), do not confuse them. For know the linter does nothing about them.

BTW, I am quite happy to see that a solution slowly emerges here.

[mariolenz]

@ssbarnea FYI I ran into a problem when I removed test-requirements.txt.

I agree that we should do this in gradual steps. Actually, I don't even know if this file is a problem... just wanted to mention it so you could have a look.

[felixfontein]

The standard places for Python test requirements are tests/unit/requirements.txt for unit tests, and tests/integration/requirements.txt for integration tests. ansible-test has been using these for a long time (at least the former, I've never used the latter so far). test-requirements.txt is something I've never (actively) seen before. That's something that comes from the Ansible Zuul config: https://github.com/ansible/zuul-config/blob/master/roles/ansible-test/defaults/main.yaml#L28

No idea why it was introduced or by whom.

[mariolenz]

test-requirements.txt is something I've never (actively) seen before. That's something that comes from the Ansible Zuul config: https://github.com/ansible/zuul-config/blob/master/roles/ansible-test/defaults/main.yaml#L28

I've tried to remove the file but then Zuul fails with:

ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/home/zuul-worker/.ansible/collections/ansible_collections/community/vmware/test-requirements.txt'

[mariolenz]

@ssbarnea Please close this issue if done, or open a new forum topic and then close the issue with a pointer to the new discussion: Community-topics: Archiving the repo

[felixfontein]

I think this is kind of standardized, so we can close it. The only thing left is to document the result somewhere. Because right now, it is nowhere documented as a standard...

@ssbarnea is this somewhat documented in the ansible-lint docs?

[Andersson007]

as it's still possible to write comments here, let's close it and move this to the forum if needed, thanks everyone!