Release v1.0.3a #123
Merged
Release v1.0.3a #123
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix a race condition where python3 needs to cast list to JSON tree_to_list(json_data=json.dumps(e), myList=myList)
Fix a race condition with list where list keyword is mandatory in py3 and not in py2.7
* Issue #11 - cv_container refactor to intend based Refactor cv_container to be intend based and support a containers topology provided by user. Module compare topology to cv_facts content, create missing containers and remove containers not in topology and with no children nor devices. Current data model can be extended later to support other cv_* modules Short play example: ------------------- - name: Test cv_container hosts: cvp connection: local gather_facts: no vars: verbose: False containers: - name: Fabric parent_container: Tenant - name: Spines parent_container: Fabric - name: Leaves parent_container: Fabric tasks: - name: "Gather CVP facts {{inventory_hostname}}" cv_facts: host: '{{ansible_host}}' username: '{{cvp_username}}' password: '{{cvp_password}}' protocol: https register: cvp_facts - name: "Build Container topology on {{inventory_hostname}}" cv_container: host: '{{ansible_host}}' username: '{{cvp_username}}' password: '{{cvp_password}}' topology: '{{containers}}' cvp_facts: '{{cvp_facts.ansible_facts}}' * Issue #11 - Update module documentation Update documentation related to cv_container in README and playbook.container.demo.yaml * Issue #14 - cv_container with empty toplogy Implement a fix to support empty topology In this scenario, all empty containers are removed from CVP topology. Updates: - execute creation process only if topology not empty - update deletion process to protect lookup for iterable topology * Enhance playbook about cv_container Create a playbook to create / test / cleanup container's topology Supported Version: ------------------ - CVP 2018.2.4 - CVP 2018.2.5 Supported Scenario: ------------------- - Create first level container - Create sub-level container - Parse facts for container - Delete complex containers structure Tags: ----- - To buld topology: --tags build - To check if container has been created: --tags check - To cleanup: --tags cleanup * Issue #13 - fix cv_configlet wrong error message Implement protection to loolup container and device name targeted by configlet. * Issue #18 - Update new cv_facts Update result to use same structure as previous version ``` result = dict(changed=False, ansible_facts={}) module.client = connect(module) result['ansible_facts'] = cv_facts(module) ``` * Issue #18 - Import code from Hugh repository Code import: ------------ Import module_utils Import library/cv_facts Import library/cv_configlet_v2 Repository update: ------------------ Update ansible.cfg to point to correct module_utils Update inventory to be more consistent (foster / grant / dev / lab) Update playbook for cv_facts Create playbook for cv_configlet_v2 * update inventory * Update requirements for v2 * Issue #19 - Support EOS offline devices Implement protection when getting configuration device: - If device is not online --> reply Device unreachable - If CVP does not send device configuration --> reply Configuration not found "devices": [ { "complianceCode": "", "complianceIndication": "", "config": "Device unreachable", "deviceSpecificConfiglets": [], "fqdn": "veos-3", "imageBundle": "", "key": "50:bb:46:4c:8c:f5", "name": "veos-3", "parentContainerKey": "undefined_container", "parentContainerName": "Undefined", "version": "4.23.0F" } ] Implemented for both cvp2018 and cvp2019 Project Issue: #18 * Issue #18 - Implement cv_configlet playbook Implement a basic playbook to test cv_configlet against lab. Coverage: - Create static configlet using a list of configlet configured with content To be done later: - Complete idempotency approach (Add/delete/update) - Use of empty list of configlet * Issue #18 - Update configlet playbook Implement a basic configlet playbook demo * Issue #18 - Update cv_configlet Rename cvp_filter to configlet_filter Add password protection * Issue #18 - Module documentation Build module documentation. Can be seen with: ansible-doc cv_facts ansible-doc cv_configlet * Issue #18 - Update documentation * Issue23 refactor cv_container (#26) * Issue #17 - package CVP client to module_utils * configure default port for CVP server * Display task execution duration * Issue #23 - cv_container update for cv_facts After issue #18, cf_containers had to be updated to use new outputs from cv_facts module. - Supported CVP version: Foster/2018.2.5 - Enhancement: use module_utils instead of cvprac Ship a demo playbook to use as vaidator for this module. * Issue #23 - Add functions to build container tree. Add 2 functions to build a tree of container and to list in the correct order all containers to create. See docstring for documentation. * Issue #23 - Support new data model for creation Phase 1 of new data structure: - Add support for new data structure for container creation - Add reference to treelib to build topology structure - Update demo playbook CVP Version supported: - 2018.2.5 WARNINGS: - Deletion model is de-activated for Phase 1 * Issue #23 - Support new data model for creation Phase 2: Support new data structure for deletion ------------------------------------------------ - Add support for new data structure for container deletion - Code cleanup Phase 1: Support new data structure for creation ------------------------------------------------ - Add support for new data structure for container creation - Add reference to treelib to build topology structure - Update demo playbook CVP Version supported: ---------------------- - 2018.2.5 Execution time (Tree of 3 containers): -------------------------------------- Build Container topology on cvp_foster -- 7.92s Cleanup Container topology on cvp_foster -- 8.90s * Update git ignore * Issue #23 - cv_container move devices Implement move devices to container based on topology description. Supported CVP: 2018.2.x * Issue #23 - enhance device move - Cover more use cases about device move / container deletion - Build module output ``` { "msg": { "changed": true, "creation_result": { "containers_created": "4" }, "deletion_result": { "containers_deleted": "1" }, "failed": false, "moved_result": { "devices_moved": 3, "list": [ "veos01", "veos02", "veos03" ] } } } ``` Test results: ------------- 1. Build topology: - Create 4 different containers in a tree - Move 3 devices in 2 containers - Create tasks - Delete one container => Execution time: 39sec 2. Cleanup topology: - Create one container - Move 3 devices to one container - Delete a tree of 4 containers => Execution time: 45sec * Issue #17 - Change ref in module_utils/legacy_cvp * Issue #23 - Implement taskIds in module output Register module output to cv_container var to be accessible by other modules. Return list of created tasks on CVP side: { "msg": { "changed": true, "creation_result": { "containers_created": "4" }, "cv_container": {}, "deletion_result": { "containers_deleted": "1" }, "failed": false, "moved_result": { "devices_moved": 3, "list": [ "veos01", "veos02", "veos03" ] }, "taskIds": [ [ "76", "77", "78" ] ] } } * Issue #23 - Fix output * Issue #23 refactor cv container (#33) * Issue #17 - package CVP client to module_utils * configure default port for CVP server * Display task execution duration * Issue #23 - cv_container update for cv_facts After issue #18, cf_containers had to be updated to use new outputs from cv_facts module. - Supported CVP version: Foster/2018.2.5 - Enhancement: use module_utils instead of cvprac Ship a demo playbook to use as vaidator for this module. * Issue #23 - Add functions to build container tree. Add 2 functions to build a tree of container and to list in the correct order all containers to create. See docstring for documentation. * Issue #23 - Support new data model for creation Phase 1 of new data structure: - Add support for new data structure for container creation - Add reference to treelib to build topology structure - Update demo playbook CVP Version supported: - 2018.2.5 WARNINGS: - Deletion model is de-activated for Phase 1 * Issue #23 - Support new data model for creation Phase 2: Support new data structure for deletion ------------------------------------------------ - Add support for new data structure for container deletion - Code cleanup Phase 1: Support new data structure for creation ------------------------------------------------ - Add support for new data structure for container creation - Add reference to treelib to build topology structure - Update demo playbook CVP Version supported: ---------------------- - 2018.2.5 Execution time (Tree of 3 containers): -------------------------------------- Build Container topology on cvp_foster -- 7.92s Cleanup Container topology on cvp_foster -- 8.90s * Update git ignore * Issue #23 - cv_container move devices Implement move devices to container based on topology description. Supported CVP: 2018.2.x * Issue #23 - enhance device move - Cover more use cases about device move / container deletion - Build module output ``` { "msg": { "changed": true, "creation_result": { "containers_created": "4" }, "deletion_result": { "containers_deleted": "1" }, "failed": false, "moved_result": { "devices_moved": 3, "list": [ "veos01", "veos02", "veos03" ] } } } ``` Test results: ------------- 1. Build topology: - Create 4 different containers in a tree - Move 3 devices in 2 containers - Create tasks - Delete one container => Execution time: 39sec 2. Cleanup topology: - Create one container - Move 3 devices to one container - Delete a tree of 4 containers => Execution time: 45sec * Issue #17 - Change ref in module_utils/legacy_cvp * Issue #23 - Implement taskIds in module output Register module output to cv_container var to be accessible by other modules. Return list of created tasks on CVP side: { "msg": { "changed": true, "creation_result": { "containers_created": "4" }, "cv_container": {}, "deletion_result": { "containers_deleted": "1" }, "failed": false, "moved_result": { "devices_moved": 3, "list": [ "veos01", "veos02", "veos03" ] }, "taskIds": [ [ "76", "77", "78" ] ] } } * Issue #23 - Fix output * Issue #23 - Return complete information of tasks * Issue #23 - rename tasksIds to tasks (#37) * added cv_task (#36) * Fixing bug in cv_task * Issue #23 - Implement configlet attachment to container (#38) * Issue #23 - rename tasksIds to tasks * Issue #23 - Implement configlet attachement Enable capability to attach a configlet to a container, This configlet must have been created previsouly using cv_configlet. Do not support configlet removal at T0 * Issue #23 - Fix configlet attachement logic (#40) * Update cv-modules from Hugh's repo (#39) * Update cv-modules from Hugh's repo Backport changes made in https://github.com/Hugh-Adams/CVP_Ansible_Modules/commit/d7b23ef9edfd7d44e449609f18cfbc27b36f55c5 Only add new changes. Add cvpinfo part of facts: "cvp_info": { "appVersion": "Foster_Build_03", "version": "2018.2.5" }, modified: ../library/cv_device.py modified: ../library/cv_facts.py * Issue #17 - Remove cvprac reference Use code within module_utils instead of cvprac * Update facts structure Update structure to fit with structure expected by other modules It also register ansible_facts within ansible during the run. * Change target for lab * Issue #24 - Update cv_device Update content from Hugh's repository to make it idempotent Input example: ``` - name: Test cv_device hosts: cvp connection: local gather_facts: no vars: configlet_list: cv_device_test01: "alias a{{ 999 | random }} show version" cv_device_test02: "alias a{{ 999 | random }} show version" # Device inventory for provision activity: bind configlet devices_inventory: veos01: name: veos01 configlets: - cv_device_test01 - SYS_TelemetryBuilderV2_172.23.0.2_1 - veos01-basic-configuration - SYS_TelemetryBuilderV2 ``` Demo playbook part of the commit * Issue #42 - Update cv_api2019.py (#43) Add missing functions from @Hugh-Adams - update_configlets_on_device - update_imageBundle_on_device appears - reset_device appears - provison_device - move_device_to_container * Add Grant support for cv_container (#44) * Fix import path * Issue #23 - Update python lib for cv_container Move python code from module_utils/cvp_legacy to lib supporting 2018.2 and 2019.1 Testing report: -------------- Grant server: Thursday 17 October 2019 18:35:18 +0200 (0:00:00.037) 0:02:31.945 ****** =============================================================================== Cleanup Container topology on cvp_grant ----------- 68.74s Gather CVP facts from cvp_grant ------------------- 53.17s Gather CVP facts from cvp_grant ------------------- 29.88s Display cv_container outputs ---------------------- 0.08s Check if Leaves container has been removed -------- 0.04s Foster server: Thursday 17 October 2019 18:41:38 +0200 (0:00:00.029) 0:02:05.724 ****** =============================================================================== Cleanup Container topology on cvp_grant ----------- 69.03s Gather CVP facts from cvp_grant -------------------- 30.39s Gather CVP facts from cvp_grant -------------------- 26.16s Display cv_container outputs ----------------------- 0.07s Check if Leaves container has been removed --------- 0.03s (Poor link connectivity during testing) * Create Ansible Collection structure (#45) * Issue #17 - Remove cvprack requirement * Issue #4 - Create collection structure * Issue #4 - Update Python path With new collection implementation, module_utils path has to be updated to use `ansible_collections.<namespace>.<collection>.plugins.module_utils` This change implies, collection has to be built/deployed for validation and not just executed as a module like in ansible2.6 * update gitignore * Rename playbook * Issue #4 - Create metadata * Copy License to collection * Create example folder to use with doc * cleanup * Ignor volatile collections folder * Update ansible version to 2.9 * Fix collection key name * Issue #4 - insert collections ref to playbook * Issue #4 - Change module_utils ref * Update makefile * Repository cleanup * Cleanup import method * First take on connection plugin (#48) * First take on connection plugin * adding more files * Grant Release documentation (#51) * Update repository documentation * Update doc after issue #29 * Update doc * Fix missing doc (#52) * Update repository documentation * Update doc after issue #29 * Update doc * Grant release documentation (#53) * Update repository documentation * Update doc after issue #29 * Update doc * Update module doc * Update module doc * Create custom issue template (#54) Provides templates for: - New Feature request - New bug report * Fix dead link * Update demo inventory * Fix python ref and requirements * Update to version 1.0.0 * Create Dockerfile for testing (#57) * Issue #56 - Dockerfile for testing Create a basic Docker file to build image and allow testing Base image: python:2.7-alpine3.6 Image size: 455M Collection is installed during boot All examples folder is included under /playbooks Build image: docker build -t ansible-cvp:latest . Use image: docker run --rm -it ansible-cvp:latest * Update docker image * Issue #56 - Create docker images for py2.7 & py3 * Update Dockerfile as per review * Issue #56 - Remove unecessary cmd * WIP - Update code to match ansible-test (#65) * Issue #64 - PEP8 & Pylint for cv_task Fix code to be PEP8 and PYLINT compliant * Issue #64 - shebang & validate-modules for cv_task Update cv_task to be compliant with ansible-test for following tests: - shebang - validate-modules - yamllint * Remove unused module * Issue #64 - Code Sanity for cv_facts Update module to be compliant with ansible-test with - pylint - pep8 - yamllint - validate-modules ansible-test sanity cv_task --docker --skip-test import * Issue #64 - Code Sanity for cv_device Update module to be compliant with ansible-test with - pylint - pep8 - yamllint - validate-modules ansible-test sanity cv_device --docker --skip-test import * Move import section to top * ssue #64 - Code Sanity for cv_container Update module to be compliant with ansible-test with - pylint - pep8 - yamllint Still active: - validate-modules related to treelib dependency ansible-test sanity cv_container --docker --skip-test import * Issue #64 - Code Sanity for cv_configlet Update module to be compliant with ansible-test with - pylint - pep8 - yamllint - validate-modules Still active: - validate-modules related to fuzzylib dependency ansible-test sanity cv_configlet --docker --skip-test import * Update gitignore * Issue #64 - Code Linting for ansible-test Global code linting to be compliant with ansible-test - Change import of external lib with try/except - Update module_utils path in import - PEP8 update for module and module_utils - YAML Linting * Fix issue #61 As PR #65 creates a large update, this fix is manually backport to this branch instead of a specifc issue branch * Update ansible version * Continuous Integration Implementation (#66) * Create a pre-commit script for py linting To use that script just run following command at root of the repository $ ln -s -f ../../.github/pre-commit .git/hooks/pre-commit Script uses pylint for code linting as it is tool uses by ansible-test Pylint configuration is similar to configuration uses in ansible-test To install pylint, run: $ pip install pylint * Issue #34 - Create Makefile for repo Makefile to manage: - Code Linting - Ansible-test sanity validation - Docker Build (need to be updated after issue #56) - Build and install collection * Implement github CI for code sanity First iteration of CI for code sanity validation on push in issue* and feature* branch Will be extended later for PR and other branchs * Update Issue templates * Rename Action in workflow * Issue #34 - Makefile update Create a self-documented Makefile * Create default pylint configuration Pylint configuration to use with make linting * Update requirements * Update Makefile with more scenrio * Implement CI with GA for code sanity * Implement mode to manage topology in cv_container (#68) * Issue #58 - Implement mode phase 1 Implement mode option with following options: - merge - override Demo playbook has been updated to reflect modes: - tags=provision --> mode=merge - tags=cleanup --> mode=override * Issue #58 - Implement delete mode Implement delete mode in cv_container Delete topology provided within `topology` and do not interact with other containers. * Fix #58 - Update code after grant-release rebase * Implement #67 - Use ansible_collections in repo (#73) Add ansible_collections to allow live testing during development: - Update ansible.cfg in examples to point to this collection - Update ansible.cfg in ansible_collections/arista/cvp/tests to point to this collection - Update Makefile for ansible-test use cases - Update docker file * Deactivate Github Actions (#74) * Deactivate CI for Github Actions * Update repository documentation (#70) * Update repository documentation * Symlink collection README to repo README * Update documentation * Update documentation * Apache License * Update README * Update README.md * Update collection path * Fix #75 - Configure multiple dev in undefined (#76) Implement checks to save topology only when we configure last new_device of the list. * Fix #79 Tasks not caught by cv_task from cv_device (#80) Change cv_device to use real JSON fields and make it compatible with cv_task module * Update Documentation as discussed in PR#77 (#81) * Fix typo in module_utils * Update Inventory ref * Update collection version to 1.0.1 * Remove old path * Fix #82 - Deduplicate taskIds (#84) Collect all taskIds for any action, then, deduplicate and get information from CVP to allow cv_task to execute them This situation could happen when cv_device and cv_configlet are called with no tasks execution in between if both modules generate changes on same devices. Align output with other module with result['data'] instead of result['cv_container'] (Old name remains as deprecated) * Makefile for testing (#85) * Create Makefile for dev testing * Remove old playbooks * Update testing playbooks * Add documentation * Pre-release update
As CvpClient requires a name's key in device, cv-facts now implement this key based on hostname key.
In 2019.1.1 during deletion, unknown containerID error appears during deletion. Excluding default CVP containers from is_container_empty solved issue.
Validate Python3 support
Use a single entry point to build docker image with Py3 by default. Image tag will be based on git branch and master will use latest git tag. A Make command is also available to start a container and run shell into this container. (No mountpoint supported)
Add additional test to ensure output is not updating if CV returns a list with no entry.
libssl1.0 is deprecated and do not permit the installation of world
Update Dockerfile-3 for upgrade of libssl
upgrade libssl1.0 to libssl1.1 because ot was deprecated and pip do not want to install world
Implement a fucntion to get container's name acting as root container. If not found, returns Tenant as it is default CVP value.
Identify root container from CV facts and use it dynamically instead of relying on Tenant which can be changed - Implement a function to identify root container - Build list of builtin containers at startup - Remove tenant references - Implement baseline for debug outputs
Allow custom root container in cv_container
cv_facts : use extend() instead of append() to avoid creating a list of list. Fixes #118
upgrade of libssl
Implement wait_task_ids in update process to avoid duplicate task IDs
networkop
approved these changes
Jan 28, 2020
Comment on lines
+309
to
+310
| debug : bool, optional | ||
| Activate debug logging, by default False |
There was a problem hiding this comment.
i think there's an easier way to debugging. something like https://github.com/ansible/ansible/blob/5b3526535c9e2f9e117e73d8fbea24b6f9c849ef/lib/ansible/modules/monitoring/logicmonitor.py#L519
which works like this :
https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/basic.py#L1861
it looks cleaner at least than passing this variable around every function
There was a problem hiding this comment.
Thanks for these pointers. Looks like definitely a better approach and will make it part of the next version for all collection's modules.
There was a problem hiding this comment.
Track with issue #124
titom73
added
status: testing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Supported CloudVision version:
2018.2.52019.1.02019.1.12019.1.2Enhancement:
Fixed issues:
cv_containerdid not return correct list of attached configlets (Issue In cv_container configlet_attached is updated even if not used #108)save_topologyfromcv_container(Remove save_topology option (#115) #116)