dynamodb_table: AnsibleAwsModule, billing_mode, point_in_time_recovery #68320

anryko · 2020-03-19T00:07:59Z

SUMMARY

Fix AnsibleAwsModule migration stale PR (#45402). Add billing_mode and point_in_time_recovery. Small bug fixes. Integration tests fixes.

Fixes #45402
Fixes #56923

ISSUE TYPE

Bugfix Pull Request
Feature Pull Request

COMPONENT NAME

dynamodb_table

ADDITIONAL INFORMATION

    - name: enable on-demand billing and backups
      dynamodb_table:
        name: table_name
        hash_key_name: myhashkey
        billing_mode: PAY_PER_REQUEST
        point_in_time_recovery: yes

ansibot · 2020-03-19T00:12:07Z

cc @jillr @loia @s-hertel @tremble
click here for bot help

ansibot · 2020-03-19T00:12:08Z

@anryko, just so you are aware we have a dedicated Working Group for aws.
You can find other people interested in this in #ansible-aws on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

ansibot · 2020-03-19T00:29:51Z

The test ansible-test sanity --test validate-modules [explain] failed with 10 errors:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: doc-default-does-not-match-spec: Argument 'billing_mode' in argument_spec defines default as (None) but documentation defines default as ('PROVISIONED')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: option-incorrect-version-added: version_added for new option (sse_enabled) should be '2.10'. Currently StrictVersion ('2.8')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: option-incorrect-version-added: version_added for new option (sse_kms_master_key_id) should be '2.10'. Currently StrictVersion ('2.8')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: option-incorrect-version-added: version_added for new option (sse_type) should be '2.10'. Currently StrictVersion ('2.8')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: option-incorrect-version-added: version_added for new option (stream_enabled) should be '2.10'. Currently StrictVersion ('2.8')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: option-incorrect-version-added: version_added for new option (stream_view_type) should be '2.10'. Currently StrictVersion ('2.8')
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: parameter-type-not-in-doc: Argument 'sse_kms_master_key_id' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: parameter-type-not-in-doc: Argument 'sse_type' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: parameter-type-not-in-doc: Argument 'stream_view_type' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: required_if-key-in-requirements: required_if contains its key sse_type in requirements

The test ansible-test sanity --test pylint [explain] failed with 3 errors:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:248:4: ansible-bad-module-import: Import external package or ansible.module_utils not ansible.utils.vars
lib/ansible/modules/cloud/amazon/dynamodb_table.py:295:31: ansible-format-automatic-specification: Format string contains automatic field numbering specification
lib/ansible/modules/cloud/amazon/dynamodb_table.py:305:8: ansible-format-automatic-specification: Format string contains automatic field numbering specification

The test ansible-test sanity --test compile --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:330:11: SyntaxError: for key, value in merge_hash(tags, table_tags).items()

The test ansible-test sanity --test future-import-boilerplate [explain] failed with 1 error:

test/units/modules/cloud/amazon/test_dynamodb_table.py:0:0: missing: from __future__ import (absolute_import, division, print_function)

The test ansible-test sanity --test import --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:330:11: traceback: SyntaxError: invalid syntax

The test ansible-test sanity --test metaclass-boilerplate [explain] failed with 1 error:

test/units/modules/cloud/amazon/test_dynamodb_table.py:0:0: missing: __metaclass__ = type

click here for bot help

ansibot · 2020-03-23T14:05:00Z

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: doc-default-does-not-match-spec: Argument 'billing_mode' in argument_spec defines default as (None) but documentation defines default as ('PROVISIONED')

The test ansible-test sanity --test future-import-boilerplate [explain] failed with 1 error:

test/units/modules/cloud/amazon/test_dynamodb_table.py:0:0: missing: from __future__ import (absolute_import, division, print_function)

The test ansible-test sanity --test metaclass-boilerplate [explain] failed with 1 error:

test/units/modules/cloud/amazon/test_dynamodb_table.py:0:0: missing: __metaclass__ = type

click here for bot help

Thanks to Dave Hewy for providing PR for sse encryption and stream functionality. Cleaned up the code to improve the readability. Added unit-test for delete index.

ansibot · 2020-03-23T17:39:53Z

The test ansible-test sanity --test ansible-doc [explain] failed with the error:

Output on stderr from ansible-doc is considered an error.

Command "ansible-doc -t module dynamodb_table" returned exit status 0.
>>> Standard Error
[WARNING]: module dynamodb_table not found in:
/dev/null:/root/ansible/lib/ansible/modules

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python3.6 /root/ansible/test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make singlehtmldocs' failed with status code: 2
--> Standard Output
PYTHONPATH=../../lib ../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
PYTHONPATH=../../lib ../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
mkdir -p rst/cli
PYTHONPATH=../../lib ../../hacking/build-ansible.py generate-man --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
PYTHONPATH=../../lib ../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ./keyword_desc.yml
PYTHONPATH=../../lib ../../hacking/build-ansible.py document-plugins -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/modules/ 
Evaluating module files...
Makefile:99: recipe for target 'modules' failed
--> Standard Error
Traceback (most recent call last):
  File "../../hacking/build-ansible.py", line 92, in <module>
    main()
  File "../../hacking/build-ansible.py", line 81, in main
    retval = command.main(args)
  File "/root/ansible/hacking/build_library/build_ansible/command_plugins/plugin_formatter.py", line 747, in main
    plugin_info, categories = get_plugin_info(args.module_dir, limit_to=args.limit_to, verbose=(args.verbosity > 0))
  File "/root/ansible/hacking/build_library/build_ansible/command_plugins/plugin_formatter.py", line 225, in get_plugin_info
    doc, examples, returndocs, metadata = plugin_docs.get_docstring(module_path, fragment_loader, verbose=verbose)
  File "/root/ansible/lib/ansible/utils/plugin_docs.py", line 128, in get_docstring
    add_fragments(data['doc'], filename, fragment_loader=fragment_loader)
  File "/root/ansible/lib/ansible/utils/plugin_docs.py", line 116, in add_fragments
    raise AnsibleError('unknown doc_fragment(s) in file {0}: {1}'.format(filename, to_native(', '.join(unknown_fragments))))
ansible.errors.AnsibleError: unknown doc_fragment(s) in file ../../lib/ansible/modules/cloud/amazon/dynamodb_table.py: aws, ec2
make: *** [modules] Error 1

The test ansible-test sanity --test package-data [explain] failed with 2 errors:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: File not installed
lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: File was not added to sdist

The test ansible-test sanity --test import --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: traceback: ImportWarning: Not importing directory '/root/ansible/lib/ansible/modules/cloud': missing __init__.py

The test ansible-test sanity --test import --python 2.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:0:0: traceback: ImportWarning: Not importing directory '/root/ansible/lib/ansible/modules/cloud': missing __init__.py

The test ansible-test sanity --test import --python 3.5 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:282:0: traceback: NameError: name 'AWSRetry' is not defined

The test ansible-test sanity --test import --python 3.6 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:282:0: traceback: NameError: name 'AWSRetry' is not defined

The test ansible-test sanity --test import --python 3.7 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:282:0: traceback: NameError: name 'AWSRetry' is not defined

The test ansible-test sanity --test import --python 3.8 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:282:0: traceback: NameError: name 'AWSRetry' is not defined

The test ansible-test sanity --test import --python 3.9 [explain] failed with 1 error:

lib/ansible/modules/cloud/amazon/dynamodb_table.py:282:0: traceback: NameError: name 'AWSRetry' is not defined

The test ansible-test sanity --test integration-aliases [explain] failed with 1 error:

test/integration/targets/dynamodb_table/aliases:0:0: missing alias `shippable/cloud/group1` or `unsupported`

click here for bot help

marknet15 · 2020-05-01T16:18:03Z

Hi @anryko I'm not sure if you are still currently working on this PR but I have created the following based on your changes, if you wouldn't mind taking a look at ? it provides some fixes on top of your changes based on usage of your updated module internally.
anryko#1

tremble · 2020-05-04T07:17:52Z

Hi @marknet15 thanks for taking the time to try and move this forward.

Unfortunately the AWS modules have been moved from the main Ansible repository. There are now two repos which include AWS modules

https://github.com/ansible-collections/amazon.aws which includes the 'supported' modules
https://github.com/ansible-collections/community.aws which includes the 'community' modules (and will be the initial home for most future modules)

dynamodb_table can now be found in the 'community' repo: https://github.com/ansible-collections/community.aws/blob/master/plugins/modules/dynamodb_table.py

Additionally the IAM policies used for testing have been moved into https://github.com/mattclay/aws-terminator/tree/master/hacking (the repo will probably get renamed at some point)

If you migrate the PR over to the new repo feel free to CC me and I'll have a first pass at reviewing it.

marknet15 · 2020-05-04T08:34:25Z

Hi @tremble, no problem at all and thanks for your response and extra information, much appreciated! I will look at getting the ball rolling and re-create the PR as requested.

Akasurde · 2020-08-20T08:13:07Z

Closing in favor of ansible-collections/community.aws#65

Akasurde · 2020-08-20T08:13:31Z

@anryko Thanks for the contribution.

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed core_review In order to be merged, this PR must follow the core review workflow. labels Mar 23, 2020

anryko force-pushed the dynamodb_boto3_billing branch from 72ce688 to 8491340 Compare March 23, 2020 16:26

ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Mar 23, 2020

sramak1396c and others added 9 commits March 23, 2020 17:49

Fixed integration test and pylint

c7a4b8f

Fixed lint issues

ebb0bd8

Add dynamodb_table integration test

539e767

Fix integration test

56f6d31

Added unit tests for dynamodb_table

72cc8c7

Add sse and stream functionality to dynamodb_table module

3b2ea74

Thanks to Dave Hewy for providing PR for sse encryption and stream functionality. Cleaned up the code to improve the readability. Added unit-test for delete index.

dynamodb_table: add biling_mode, point_in_time_recovery, purge_tags

9f3d62b

dynamodb_table: integration tests

6f96fb9

dynamodb_table: ci fixes

ba80f94

anryko force-pushed the dynamodb_boto3_billing branch from 8491340 to ba80f94 Compare March 23, 2020 16:51

ansibot removed the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Mar 23, 2020

bcoca removed the needs_triage Needs a first human triage before being processed. label Mar 26, 2020

ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Mar 26, 2020

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Apr 3, 2020

ansibot added collection Related to Ansible Collections work collection:community.aws labels Apr 24, 2020

marknet15 mentioned this pull request May 1, 2020

dynamodb_table: billing_mode, sse, gsi fixes anryko/ansible#1

Closed

Akasurde closed this Aug 20, 2020

ansible locked and limited conversation to collaborators Sep 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dynamodb_table: AnsibleAwsModule, billing_mode, point_in_time_recovery #68320

dynamodb_table: AnsibleAwsModule, billing_mode, point_in_time_recovery #68320

anryko commented Mar 19, 2020

ansibot commented Mar 19, 2020

ansibot commented Mar 19, 2020

ansibot commented Mar 19, 2020

ansibot commented Mar 23, 2020

ansibot commented Mar 23, 2020

marknet15 commented May 1, 2020

tremble commented May 4, 2020 •

edited

Loading

marknet15 commented May 4, 2020

Akasurde commented Aug 20, 2020

Akasurde commented Aug 20, 2020

dynamodb_table: AnsibleAwsModule, billing_mode, point_in_time_recovery #68320

dynamodb_table: AnsibleAwsModule, billing_mode, point_in_time_recovery #68320

Conversation

anryko commented Mar 19, 2020

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

ansibot commented Mar 19, 2020

ansibot commented Mar 19, 2020

ansibot commented Mar 19, 2020

ansibot commented Mar 23, 2020

ansibot commented Mar 23, 2020

marknet15 commented May 1, 2020

tremble commented May 4, 2020 • edited Loading

marknet15 commented May 4, 2020

Akasurde commented Aug 20, 2020

Akasurde commented Aug 20, 2020

tremble commented May 4, 2020 •

edited

Loading