Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add site rules support (fixes broken request rules) #45

Merged
merged 3 commits into from
Jul 17, 2020

Conversation

fwpt
Copy link

@fwpt fwpt commented Jul 7, 2020

Site rules have replaced request rules and signal rules in the SigSci API.
Request rules calls where broken in the Site Management Tool: filter_data in util.py reported missing key error because there is no 'action' in the JSON response data from site rules endpoint. Also 'signal' is optional (caused error too) and 'type' was missing (essential for restore).

Description

Implemented support for site rules which should replace the current request and signal rules. Request and signal rules are still supported for clone and deploy commands when these are identified in the source data. If request rules in the backup command are still required for new backups, then suggest to add support for exclude categories in the backups method as indicated in backup.py, line 101.

The following properties of site rules differ from request rules:

  • can have multiple actions in 'actions' property which replaced the 'action' property;
  • has a 'type' (request (former request rule) or signal exclusion (former signal rule));
  • optionally, has a 'signal' (in case of signal exclusion).

Please note that I have not updated Readme to include SITE_RULES category for exclude/include parameters. Suggest to add a message that request rules and signal rules are deprecated now.

Motivation and Context

It fixes the following errors in the backup command:

Traceback (most recent call last):
  File "env/bin/sigsci_site_manager", line 11, in <module>
    sys.exit(main())
  File "env/lib/python3.6/site-packages/sigsci_site_manager/site_manager.py", line 417, in main
    args.func(args)
  File "env/lib/python3.6/site-packages/sigsci_site_manager/site_manager.py", line 61, in do_backup
    backup(api, args.site_name, args.file_name)
  File "env/lib/python3.6/site-packages/sigsci_site_manager/backup.py", line 119, in backup
    data = backups(api, site_name)
  File "env/lib/python3.6/site-packages/sigsci_site_manager/backup.py", line 96, in backups
    data['request_rules'] = get_request_rules(api)
  File "env/lib/python3.6/site-packages/sigsci_site_manager/backup.py", line 22, in get_request_rules
    return filter_data(data['data'], keys)
  File "env/lib/python3.6/site-packages/sigsci_site_manager/util.py", line 9, in filter_data
    r = {k: item[k] for k in keys}
  File "env/lib/python3.6/site-packages/sigsci_site_manager/util.py", line 9, in <dictcomp>
    r = {k: item[k] for k in keys}
KeyError: 'signal'

and also a similar error for the action key:
KeyError: 'action'

This might be depending on the SigSciApi version. The latest version redirects SigSciApi.add_request_rules to SigSciApi.add_site_rules which might have introduced this error.

How Has This Been Tested?

  • Backup: By invoking the backup command to validate that the error is gone and that the site requests are properly stored in JSON format in the designated output file. sigsci_site_manager backup --name site1 --out site-policy.json

  • Deploy (implicitly tests Clone): By invoking the deploy command with the output file from previous test: sigsci_site_manager deploy --name site2 --file site-policy.json --dry-run --include SITE_RULES and sigsci_site_manager deploy --name site2 --file site-policy.json --include RULE_LISTS,SIGNAL_RULES,CUSTOM_SIGNALS,SITE_RULES and verify results via Console and by running backup command for site2 to compare site rules against site rules from site1 backup. Tested rules include both rule types (request and signal exclusion).

  • Merge: Created a new site 'test2', invoked: sigsci_site_manager merge --src test --dest test2 --include SITE_RULES and did the same comparison as in Deploy test.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows conforms to PEP8.
  • My change requires a change to the documentation (README.md).
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@fwpt
Copy link
Author

fwpt commented Jul 7, 2020

Please note the test_build_category_list test failed because consts.py is updated to include SITE_RULES. Updated the test to reflect new consts.

@pizen pizen requested review from pizen and joeyleake July 7, 2020 13:45
@pizen pizen merged commit 964f320 into turnerlabs:master Jul 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants