Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community.aws.route53_info query:record_sets - type and max_items ignored on all 2.10 versions #529

Closed
tamirhad opened this issue Apr 6, 2021 · 8 comments · Fixed by #813
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3

Comments

@tamirhad
Copy link
Contributor

tamirhad commented Apr 6, 2021

SUMMARY

on every 2.10.x ansible-base version, when using community.aws.route53_info module with query:record_sets:

ISSUE TYPE
  • Bug Report
COMPONENT NAME

community.aws.route53_info
plugins/modules/route53_info.py

ANSIBLE VERSION

tested with every version under 2.10
but for this exmaple:

ansible 2.10.7
  config file = None
  configured module search path = ['/Users/tamir/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/tamir/opt/anaconda3/envs/django/lib/python3.9/site-packages/ansible
  executable location = /Users/tamir/opt/anaconda3/envs/django/bin/ansible
  python version = 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:20) [Clang 11.0.1 ]
CONFIGURATION
ansible-config dump --only-changed produce no output
OS / ENVIRONMENT
  • brand new environment on macos, ubuntu18
STEPS TO REPRODUCE
- name: Get alias
   route53_facts:
     query: record_sets
     hosted_zone_id: "{{host_zone_id}}"
     start_record_name: "{{ dns.records[0].record }}"
     type: "A"
     max_items: 1
   register: record_sets
EXPECTED RESULTS

single A record due to max_items:1 with type: "A"

ACTUAL RESULTS

all records with every type under the host_zone.
cant paste the actual results due to privacy, but its easy to reproduce.


@ansibullbot
Copy link

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot ansibullbot added bug This issue/PR relates to a bug needs_triage python3 labels Apr 6, 2021
@tamirhad
Copy link
Contributor Author

tamirhad commented Apr 6, 2021

!component =plugins/modules/route53_info.py

@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot
Copy link

@ansibullbot ansibullbot added module module plugins plugin (any type) labels Apr 6, 2021
@felixfontein
Copy link
Contributor

I don't think this has any relation to #523:

  1. it's a different module (route53 vs. route53_info);
  2. the route53_info module has not really changed since it was added to this collection.

I checked the version of the module included in Ansible 2.9, it also ignored type. It didn't ignore max_items though. It does ignore max_items in ansible/ansible's pre-ansible-base branch, so that bug has been introduced between stable-2.9's branching and creation of this collection.

Looking at the commit history, the most likely candidate is ansible/ansible@6075536#diff-23a0c9250633162d50c3f06442b7a552a5ae0659a24dd01a328c0e165e473616.

dseeley pushed a commit to dseeley/clusterverse that referenced this issue Apr 17, 2021
+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523).
+ In any case, state=get on the route53 module is not likely to remain supported indefinitely.
+ Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.
dseeley added a commit to dseeley/clusterverse that referenced this issue Apr 17, 2021
+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523).
+ In any case, state=get on the route53 module is not likely to remain supported indefinitely.
+ Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.
+ Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.
dseeley-sky added a commit to sky-uk/clusterverse that referenced this issue Jun 1, 2021
+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523).
+ In any case, state=get on the route53 module is not likely to remain supported indefinitely.
+ Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.
+ Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.
dseeley pushed a commit to dseeley/clusterverse that referenced this issue Jun 28, 2021
…sions.

+ Using route53 info in clean/tasks/dns.yml is too slow with large recordsets, (hopefully only until ansible-collections/community.aws#529 is fixed).
+ It is only supported between Ansible 2.9.6 and 2.10.6 and after Ansible 4; assertions added to pre-flight check.
dseeley added a commit to dseeley/clusterverse that referenced this issue Jun 28, 2021
…rsions. (#3)

+ Using `route53_zone`/ `route53_info`/ `route53` in clean/tasks/dns.yml is too slow with large recordsets, (hopefully only until ansible-collections/community.aws#529 is fixed).
+ Old mechanism (`route53(state=get)`) is only supported between Ansible 2.9.6 and 2.10.6 and after Ansible 4; assertions added to pre-flight check.
dseeley-sky added a commit to sky-uk/clusterverse that referenced this issue Jun 30, 2021
+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523).
+ In any case, state=get on the route53 module is not likely to remain supported indefinitely.
+ Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.
+ Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.
antoineserrano pushed a commit to sky-uk/clusterverse that referenced this issue Jul 1, 2021
…delete (#92)

* Use route53_info to get record sets for delete

+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523).
+ In any case, state=get on the route53 module is not likely to remain supported indefinitely.
+ Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.
+ Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.

* Fix for private zone

* Fix non-private DNS. Add testsuite support for scaling up/down

* Break dependence upon 2.9.6 <= Ansible <= 2.10.6.  Route53 deletion enhancement.

* Remove  '--format json' from 'ansible-galaxy collection list' assertion, as 'format option' is not present in ansible-galaxy 2.10.x
@haukebruno
Copy link

I can confirm that this issues also/still exists in Ansible 2.10.9 + community.aws 1.5.0

@t-readyroc
Copy link

Also in Ansible 2.10.12/community.aws 1.5.0. I also get a warning:

 - The value "1" (type int) was converted to "'1'" (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

@ansibullbot
Copy link

ansible-zuul bot pushed a commit that referenced this issue Nov 30, 2021
Fix route53_info max_items / type being ignored

SUMMARY
Currently if max_items is set on the route53_info module then it is ignored meaning all items are returned.
type is also ignored due to an incorrect if statement
It looks like it was a regression introduced here:
ansible/ansible@6075536#diff-23a0c9250633162d50c3f06442b7a552a5ae0659a24dd01a328c0e165e473616
The tests have been updated to reflect a check on max_items and type
Fixes: #529
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53_info
ADDITIONAL INFORMATION
The problem with max_items being ignored is resolved by adding PaginationConfig and adding MaxItems to that instead.
The problem with type being ignored is resolved by fixing an if statement.
Boto3 docs: https://boto3.amazonaws.com/v1/documentation/api/1.18.7/reference/services/route53.html#Route53.Paginator.ListResourceRecordSets

Reviewed-by: Mark Chappell <None>
Reviewed-by: Markus Bergholz <git@osuv.de>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>
dseeley pushed a commit to dseeley/clusterverse that referenced this issue Mar 18, 2022
…o get current records

+ route53_info module doesn't honour the max_items or type fields (ansible-collections/community.aws#529), so if you have lots of records, it can take a very long time to return.  This is likely fixed in ansible>=6.0.0)
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this issue May 25, 2022
…meter if no name and no filter provided (ansible-collections#529)

ec2_instance When searching for instances fall back to tag(Name) parameter if no name and no filter provided

SUMMARY
Reported in ansible-collections#526
When the name parameter isn't set but the Name tag is set the Name tag is ignored when searching for existing instances.
The documentation states:
- By default, instances are filtered for counting by their "Name" tag, base AMI, state (running, by default), and
  subnet ID.

As such I think we should consider this a bug.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
ec2_instance
ADDITIONAL INFORMATION
fixes: ansible-collections#526

Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>
abikouo pushed a commit to abikouo/community.aws that referenced this issue Oct 24, 2023
)

Fix route53_info max_items / type being ignored

SUMMARY
Currently if max_items is set on the route53_info module then it is ignored meaning all items are returned.
type is also ignored due to an incorrect if statement
It looks like it was a regression introduced here:
ansible/ansible@6075536#diff-23a0c9250633162d50c3f06442b7a552a5ae0659a24dd01a328c0e165e473616
The tests have been updated to reflect a check on max_items and type
Fixes: ansible-collections#529
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53_info
ADDITIONAL INFORMATION
The problem with max_items being ignored is resolved by adding PaginationConfig and adding MaxItems to that instead.
The problem with type being ignored is resolved by fixing an if statement.
Boto3 docs: https://boto3.amazonaws.com/v1/documentation/api/1.18.7/reference/services/route53.html#Route53.Paginator.ListResourceRecordSets

Reviewed-by: Mark Chappell <None>
Reviewed-by: Markus Bergholz <git@osuv.de>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections@e323f83
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue/PR relates to a bug has_pr module module plugins plugin (any type) python3
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants