MagpieAdapter request/response hooks

[fmigneault]

Summary

The main purpose is presented here: https://github.com/Ouranosinc/Magpie/pull/517/files#diff-1c805585a74c7c16218b333063aed89c528c90d8ec5aa76257a1e248eda0a10fR139-R155
Using this config, pointing at file https://github.com/Ouranosinc/Magpie/blob/request-hooks/tests/hooks/request_hooks.py, MagpieAdapter would add the X-WPS-Output-Context header in the request sent during job creation toward Weaver using the requesting user ID, effectively producing the WPS-outputs in its user directory.

Changes

• Add JSON schema validation of loaded Service configuration (providers.cfg).
• Add optional hooks section under each Service definition of the providers.cfg or combined configuration
  file that allows pre/post request/response processing operations using plugin Python scripts.
• Store the validated Service configuration in magpie.services settings for later access to hooks definitions
  by the MagpieAdapter.
• Rename the webhooks section stored in settings to magpie.webhooks to avoid possible name clashes.

References

• Depends on add request/response hooks to adapter utility bird-house/twitcher#114 to call the hooks.
• Addresses DAC-145, DAC-168, DAC-467

WIP

• Add specific tests for hooks
• Merge update pylint and apply new lint fixes #516 (this PR is based on it) to reduce diff of actual changes by this PR

[codecov-commenter]

Codecov Report

Merging #517 (c235cad) into master (27e060a) will increase coverage by 0.17%.
The diff coverage is 77.82%.

@@            Coverage Diff             @@
##           master     #517      +/-   ##
==========================================
+ Coverage   80.27%   80.44%   +0.17%     
==========================================
  Files          71       72       +1     
  Lines        9615     9749     +134     
  Branches     1323     1348      +25     
==========================================
+ Hits         7718     7843     +125     
+ Misses       1597     1595       -2     
- Partials      300      311      +11     

Impacted Files	Coverage Δ
magpie/api/login/__init__.py	100.00% <ø> (ø)
magpie/api/management/group/group_views.py	100.00% <ø> (ø)
magpie/api/management/resource/resource_formats.py	98.24% <ø> (ø)
magpie/api/management/user/user_views.py	98.67% <ø> (ø)
magpie/api/requests.py	95.41% <ø> (ø)
magpie/register.py	48.79% <38.09%> (+0.53%)	⬆️
magpie/models.py	89.13% <60.00%> (ø)
magpie/api/login/login.py	57.67% <66.66%> (ø)
magpie/ui/utils.py	67.39% <66.66%> (ø)
magpie/utils.py	70.16% <69.23%> (+1.72%)	⬆️
... and 19 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2857cb...c235cad. Read the comment docs.

[ChaamC]

Looks good, some minor comments to check, but that's it!

[fmigneault]

Blocked by release pending bird-house/twitcher#114 (comment)
CI test suite still only sees v0.6.2, so adapter-hook test operations are not executed.

[dbyrns]

Everything looks good in that implementation! Very versatile.
As discussed, to be 100% flexible, a new hook should be included in twitcher to literally allow to override the _send_request method which is not really generic.
If you're worried about messing with the current implementation you could literally keep it as is in the default twitcher implementation, but allow MagpieAdapter to define it's own.