Unable to install collection by git hash due to issue with dependencies #1182

kennedyjosh · 2022-10-18T17:59:58Z

Summary

Due to a recent bug fix in a version > 5.0.2, my project requires a version >= a specific commit from this repo. Additionally, my project must use a requirements.yml file to install necessary collections, as it is infeasible for other users of my project to use the workaround mentioned here. Due to an issue with dependencies and/or the way this project has versioned itself, this currently does not work.

Issue Type

Bug Report

Component Name

amazon.aws

Ansible Version

$ ansible --version
ansible [core 2.13.5]
  config file = /Users/josh/Code/server/ansible.cfg
  configured module search path = ['/Users/josh/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/josh/Code/server/hapy/venv/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/josh/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/josh/Code/server/hapy/venv/bin/ansible
  python version = 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)]
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /Users/josh/.ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    5.0.2  
ansible.posix 1.4.0  
community.aws 3.5.0  

# /Users/josh/Code/server/hapy/venv/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.4.0  
ansible.netcommon             3.1.1  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.11.1 
arista.eos                    5.0.1  
awx.awx                       21.5.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.3.0  
cisco.aci                     2.2.0  
cisco.asa                     3.1.0  
cisco.dnac                    6.6.0  
cisco.intersight              1.0.19 
cisco.ios                     3.3.1  
cisco.iosxr                   3.3.1  
cisco.ise                     2.5.3  
cisco.meraki                  2.11.0 
cisco.mso                     2.0.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.1.1  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.5.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.5.0  
community.digitalocean        1.21.0 
community.dns                 2.3.2  
community.docker              2.7.1  
community.fortios             1.0.0  
community.general             5.6.0  
community.google              1.0.0  
community.grafana             1.5.2  
community.hashi_vault         3.2.0  
community.hrobot              1.5.2  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.5.1  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.2.0  
community.proxysql            1.4.0  
community.rabbitmq            1.2.2  
community.routeros            2.3.0  
community.sap                 1.0.0  
community.sap_libs            1.3.0  
community.skydive             1.0.0  
community.sops                1.4.0  
community.vmware              2.9.1  
community.windows             1.11.0 
community.zabbix              1.8.0  
containers.podman             1.9.4  
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.2  
dellemc.openmanage            5.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.19.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.7  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.8.2  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.9.0  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.3.0  
inspur.ispim                  1.0.1  
inspur.sm                     2.0.0  
junipernetworks.junos         3.1.0  
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.23.0
netapp.storagegrid            21.11.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.9.1  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.2.3  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.1.0  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.31.0 
theforeman.foreman            3.6.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.1.0  
vyos.vyos                     3.0.1  
wti.remote                    1.0.4

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.18.0
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/josh/Code/server/hapy/venv/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.21.0
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/josh/Code/server/hapy/venv/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
DEFAULT_HOST_LIST(/Users/josh/Code/server/ansible.cfg) = ['/Users/josh/Code/server/ansible/inventory_aws_ec2.yml']
DISPLAY_SKIPPED_HOSTS(/Users/josh/Code/server/ansible.cfg) = False
HOST_KEY_CHECKING(/Users/josh/Code/server/ansible.cfg) = False
INTERPRETER_PYTHON(/Users/josh/Code/server/ansible.cfg) = python

OS / Environment

macOS 12.5

Steps to Reproduce

requirements.yml

collections:
  ## TODO: Update this to a semver when this repo releases something >= 5.0.3
  ## Necessary to set availability zone when creating new instance
  - name: https://github.com/ansible-collections/amazon.aws.git
    type: git
    version: 395d4fda1ec549020d070d4465835166094e4777
  - name: ansible.posix
    version: 1.4.0
  - name: community.aws
    version: 3.5.0

Expected Results

I expect the collection to be installed with no issues

Actual Results

Starting galaxy collection install process
Process install dependency map
Cloning into '/Users/josh/.ansible/tmp/ansible-local-27493pd7ly847/tmppyopqis9/amazon.awsxolu8amj'...
remote: Enumerating objects: 18764, done.
remote: Counting objects: 100% (2539/2539), done.
remote: Compressing objects: 100% (746/746), done.
remote: Total 18764 (delta 1918), reused 2261 (delta 1751), pack-reused 16225
Receiving objects: 100% (18764/18764), 15.83 MiB | 19.61 MiB/s, done.
Resolving deltas: 100% (13915/13915), done.
Note: switching to '395d4fda1ec549020d070d4465835166094e4777'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 395d4fda ec2_instance: fix to handle create instance in specified AZ (#1150)
ERROR! Failed to resolve the requested dependencies map. Could not satisfy the following requirements:
* amazon.aws:>=3.4.0 (dependency of community.aws:3.5.0)
* amazon.aws:6.0.0-dev0 (dependency of "virtual collection Git repo")
make: *** [setup-dev-machine] Error 1

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

kennedyjosh · 2022-10-18T18:01:46Z

I'm making this issue here because I believe the issue has to do with the lack of semantic versioning currently in place by using 6.0.0-dev0 as the version. If it is a more of an issue with ansible-galaxy I can make the issue in the core ansible repo instead.

gravesm · 2022-10-19T15:48:00Z

The best we can offer from the collection side of things is to use ansible-galaxy collection install -r requirements.yml --pre, which should successfully install. Other than that, you would need to file an issue on the ansible/ansible repo as the dependency resolution is all handled there.

tremble · 2022-10-19T16:40:23Z

Just to add: 6.0.0-dev0 is a valid semver.

See https://semver.org/ for more information.

A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-].

It's reported as a prerelease (dev0), because we've already split stable-5 (5.x) from main, so that we can land planned breaking changes for 6.0.0 as early as possible. What you're seeing is actually an Ansible bug ansible/ansible#79112 and something we will only change on our side once main is branched into stable-x for release.

kennedyjosh · 2022-10-19T21:33:19Z

The best we can offer from the collection side of things is to use ansible-galaxy collection install -r requirements.yml --pre, which should successfully install

ansible-galaxy doesn't seem to recognize the --pre tag when I try this @gravesm

Just to add: 6.0.0-dev0 is a valid semver.

See https://semver.org/ for more information.

Thank you – I learn something new every day!

gravesm · 2022-10-20T18:21:52Z

The help for ansible-galaxy on ansible 2.13 shows the --pre option. If you are finding it's not working you will need to file an issue in the main ansible repo.

…sible-collections#1182) Tagging fragment - Move simplest cases over to the docs fragment. Depends-On: ansible-collections#844 SUMMARY Migrate simplest cases over to the new docs fragment and add resource_tags as an alias to tags. ISSUE TYPE Docs Pull Request Feature Pull Request COMPONENT NAME changelogs/fragments/1182-tagging.yml plugins/modules/aws_glue_job.py plugins/modules/aws_msk_cluster.py plugins/modules/aws_secret.py plugins/modules/aws_step_functions_state_machine.py plugins/modules/dynamodb_table.py plugins/modules/ec2_eip.py plugins/modules/ec2_transit_gateway_vpc_attachment.py plugins/modules/ec2_vpc_peer.py plugins/modules/elb_application_lb.py plugins/modules/elb_network_lb.py plugins/modules/iam_role.py plugins/modules/iam_user.py plugins/modules/networkfirewall.py plugins/modules/networkfirewall_policy.py plugins/modules/networkfirewall_rule_group.py plugins/modules/rds_cluster.py plugins/modules/rds_instance.py plugins/modules/rds_instance_snapshot.py plugins/modules/rds_option_group.py plugins/modules/rds_subnet_group.py plugins/modules/redshift.py ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@b11ffae

…sible-collections#1200) Tagging fragment - rds_instance_snapshot - Move over to the docs fragment. SUMMARY The rds_instance_snapshot tests are slow enough to cause test timeouts when combined with too many other changed, split this off. ISSUE TYPE Feature Pull Request COMPONENT NAME rds_instance_snapshot ADDITIONAL INFORMATION The change itself was approved on ansible-collections#1182 This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@9ca5540

…sible-collections#1182) Tagging fragment - Move simplest cases over to the docs fragment. Depends-On: ansible-collections#844 SUMMARY Migrate simplest cases over to the new docs fragment and add resource_tags as an alias to tags. ISSUE TYPE Docs Pull Request Feature Pull Request COMPONENT NAME changelogs/fragments/1182-tagging.yml plugins/modules/aws_glue_job.py plugins/modules/aws_msk_cluster.py plugins/modules/aws_secret.py plugins/modules/aws_step_functions_state_machine.py plugins/modules/dynamodb_table.py plugins/modules/ec2_eip.py plugins/modules/ec2_transit_gateway_vpc_attachment.py plugins/modules/ec2_vpc_peer.py plugins/modules/elb_application_lb.py plugins/modules/elb_network_lb.py plugins/modules/iam_role.py plugins/modules/iam_user.py plugins/modules/networkfirewall.py plugins/modules/networkfirewall_policy.py plugins/modules/networkfirewall_rule_group.py plugins/modules/rds_cluster.py plugins/modules/rds_instance.py plugins/modules/rds_instance_snapshot.py plugins/modules/rds_option_group.py plugins/modules/rds_subnet_group.py plugins/modules/redshift.py ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@b11ffae

…sible-collections#1182) Tagging fragment - Move simplest cases over to the docs fragment. Depends-On: ansible-collections#844 SUMMARY Migrate simplest cases over to the new docs fragment and add resource_tags as an alias to tags. ISSUE TYPE Docs Pull Request Feature Pull Request COMPONENT NAME changelogs/fragments/1182-tagging.yml plugins/modules/aws_glue_job.py plugins/modules/aws_msk_cluster.py plugins/modules/aws_secret.py plugins/modules/aws_step_functions_state_machine.py plugins/modules/dynamodb_table.py plugins/modules/ec2_eip.py plugins/modules/ec2_transit_gateway_vpc_attachment.py plugins/modules/ec2_vpc_peer.py plugins/modules/elb_application_lb.py plugins/modules/elb_network_lb.py plugins/modules/iam_role.py plugins/modules/iam_user.py plugins/modules/networkfirewall.py plugins/modules/networkfirewall_policy.py plugins/modules/networkfirewall_rule_group.py plugins/modules/rds_cluster.py plugins/modules/rds_instance.py plugins/modules/rds_instance_snapshot.py plugins/modules/rds_option_group.py plugins/modules/rds_subnet_group.py plugins/modules/redshift.py ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None>

…sible-collections#1199) Tagging fragment - dynamodb_table - Move over to the docs fragment. SUMMARY The dynamodb tests are slow enough to cause test timeouts when combined with too many other changed, split this off. ISSUE TYPE Feature Pull Request COMPONENT NAME dynamodb_table ADDITIONAL INFORMATION The change itself was approved on ansible-collections#1182

…sible-collections#1200) Tagging fragment - rds_instance_snapshot - Move over to the docs fragment. SUMMARY The rds_instance_snapshot tests are slow enough to cause test timeouts when combined with too many other changed, split this off. ISSUE TYPE Feature Pull Request COMPONENT NAME rds_instance_snapshot ADDITIONAL INFORMATION The change itself was approved on ansible-collections#1182

…sible-collections#1198) Tagging fragment - networkfirewall - Move over to the docs fragment SUMMARY The networkfirewall tests are slow enough to be flakey, split this off. ISSUE TYPE Feature Pull Request COMPONENT NAME networkfirewall ADDITIONAL INFORMATION The change itself was approved on ansible-collections#1182