Auto user permissions creation over its deployed process #247

fmigneault · 2022-06-06T21:42:08Z

When a process is deployed on Weaver, the user that deployed it (provided they had permission to do so), will automatically receive the necessary permissions to obtain access to its description and execution of jobs with it.

The process will be visible only by the deploying user (and admins) until it gets shared/published with other users/groups with the appropriate permissions applied in Magpie.

Changes

Non-breaking changes

Adds Magpie response hook that will create necessary permissions for user onto its deployed Weaver process.

Breaking changes

n/a

Note: only commits after e661ef2 can be used to see only the diff of this PR omitting other dependencies.

Related Issue / Discussion

Resolves DAC-206 - user private process deployment

Additional Information

Depends on magpie/twitcher/weaver requests hooks for WPS outputs in user workspace #244
Depends on add magpie/twticher response hook for filtering weaver processes #245
Depends on WeaverClient/CLI with auth handler and other CLI adjustments crim-ca/weaver#442
(only for using the CLI below, not impacting the stack itself)

Results / Demonstration

Using the Weaver CLI in combination of https://github.com/Ouranosinc/requests-magpie/, following command is sent and received:

weaver deploy \
  -u http://localhost:8000/ows/proxy/weaver \
  -p test-report \
  --cwl https://raw.githubusercontent.com/crim-ca/weaver/master/docs/examples/docker-python-script-report.cwl \
  --auth-handler requests_magpie.MagpieAuth \
  --auth-url http://localhost:2001 \
  --auth-username authtest \
  --auth-password authtestauthtest

[2022-06-06 16:21:35] WARNING  [weaver.utils] No settings container provided by [weaver.utils.wrapped], request options might not be applied as expected.
[2022-06-06 16:21:35] WARNING  [weaver.utils] No settings container provided by [weaver.utils.wrapped], request options might not be applied as expected.
[2022-06-06 16:21:35] WARNING  [weaver.utils] No settings container provided by [weaver.utils.wrapped], request options might not be applied as expected.
Resolved '/tmp/tmp5qbm9jep/package' to 'file:///tmp/tmp5qbm9jep/package'
{
  "description": "Process successfully deployed.",
  "processSummary": {
    "id": "test-report",
    "mutable": true,
    "keywords": [
      "application"
    ],
    "metadata": [],
    "jobControlOptions": [
      "async-execute",
      "sync-execute"
    ],
    "outputTransmission": [
      "reference",
      "value"
    ],
    "processDescriptionURL": "http://localhost:4002/processes/test-report",
    "executeEndpoint": "http://localhost:4002/processes/test-report/jobs"
  },
  "deploymentDone": true
}

note:
http://localhost:8000/ows/proxy/weaver is a Twitcher instance running with configured Magpie on http://localhost:2001 and Weaver defined as a service behind them.

Before execution, the following is shown in Magpie. These definitions are similar to current birdhouse stack when Weaver is enabled to be registered with minimal permissions. Deploy permission (write) is added to the test user using match modifier (contrary to recursive) such that they can POST (i.e.: deploy) on that specific endpoint, but cannot execute all underlying process/jobs.

After the execution of the CLI command, the following is obtained. This grants access to the user for execution and un-deployment of the new process, but still cannot modify/execute other processes they do not "own".

Logs display the valid creation:

2022-06-06 16:44:19,543 INFO  [TWITCHER|magpie.adapter.magpieowssecurity:245][ThreadPoolExecutor-0_0] User <User: name=authtest id=17089> resolved with allowed 'write' access to [/ows/proxy/weaver/processes]
2022-06-06 16:44:19,544 INFO  [TWITCHER:57][ThreadPoolExecutor-0_0] Using adapter: '<class 'magpie.adapter.MagpieAdapter'>'
2022-06-06 16:44:19,546 DEBUG [TWITCHER:79][ThreadPoolExecutor-0_0] url = http://localhost:4002/processes
2022-06-06 16:44:19,712 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Using settings from local thread.
2022-06-06 16:44:19,712 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant alternate search: magpie.providers_hooks_path
2022-06-06 16:44:19,712 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant found in definitions with: MAGPIE_PROVIDERS_HOOKS_PATH
2022-06-06 16:44:19,715 DEBUG [TWITCHER|magpie.adapter:316][ThreadPoolExecutor-0_0] Hook matched response (POST /processes) [/home/francis/dev/daccs/birdhouse-deploy/birdhouse/components/weaver/config/magpie/weaver_hooks.py:allow_user_deployed_processes]
2022-06-06 16:44:19,716 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant alternate search: magpie.admin_group
2022-06-06 16:44:19,716 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant found in definitions with: MAGPIE_ADMIN_GROUP
[2022-06-06 16:44:26 -0400] [719671] [CRITICAL] WORKER TIMEOUT (pid:719703)
[2022-06-06 16:44:26 -0400] [719703] [INFO] Worker exiting (pid: 719703)
Exiting 1 (-v to see traceback)
2022-06-06 16:44:42,619 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Using settings from local thread.
2022-06-06 16:44:42,620 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant alternate search: magpie.anonymous_user
2022-06-06 16:44:42,620 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant found in definitions with: MAGPIE_ANONYMOUS_USER
2022-06-06 16:44:42,621 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Using settings from local thread.
2022-06-06 16:44:43,588 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Using settings from local thread.
2022-06-06 16:44:43,588 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant alternate search: magpie.anonymous_user
2022-06-06 16:44:43,589 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Constant found in definitions with: MAGPIE_ANONYMOUS_USER
2022-06-06 16:44:43,589 DEBUG [magpie.utils:163][ThreadPoolExecutor-0_0] Using settings from local thread.
2022-06-06 16:45:02,248 INFO  [birdhouse-weaver-hooks:257][ThreadPoolExecutor-0_0] Successful creation of permissions for user [authtest] to access deployed process [test-report] in Weaver.

… workspace directory (relates to Ouranosinc/Magpie#517 and bird-house/twitcher#114)

…llowed permission to read them

…ter configs (within Twitcher) using directory loading strategy

fmigneault · 2022-06-06T21:46:16Z

@ahandan @francisPLT @huard
This PR might be of interest to you, it shows a use case of weaver CLI used in combination with Magpie authentication on a single command, with a simple CWL reference to deploy a new process.

crim-jenkins-bot · 2022-06-06T22:31:40Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/919/
Result : success

BIRDHOUSE_DEPLOY_BRANCH : weaver-publish-process
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/764/

NOTEBOOK TEST RESULTS

    
[2022-06-06T21:55:21.343Z] ============================= test session starts ==============================
[2022-06-06T21:55:21.343Z] platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
[2022-06-06T21:55:21.343Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2022-06-06T21:55:21.343Z] plugins: anyio-3.5.0, dash-2.3.1, nbval-0.9.6, tornasync-0.6.0.post2
[2022-06-06T21:55:21.343Z] collected 233 items
[2022-06-06T21:55:21.343Z] 
[2022-06-06T21:55:31.372Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2022-06-06T21:55:58.210Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2022-06-06T21:56:07.569Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 10%]
[2022-06-06T21:56:15.886Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 13%]
[2022-06-06T21:56:18.204Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 18%]
[2022-06-06T21:58:26.327Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ..........          [ 22%]
[2022-06-06T21:58:39.607Z] pavics-sdi-master/docs/source/notebooks/eccc_geoapi.ipynb .....          [ 24%]
[2022-06-06T21:58:54.500Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .                 [ 24%]
[2022-06-06T21:59:16.398Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 27%]
[2022-06-06T21:59:16.398Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 27%]
[2022-06-06T21:59:25.041Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 30%]
[2022-06-06T21:59:31.884Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 33%]
[2022-06-06T22:03:10.472Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 39%]
[2022-06-06T22:03:58.875Z] ..............                                                           [ 45%]
[2022-06-06T22:04:07.267Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 47%]
[2022-06-06T22:04:09.516Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 50%]
[2022-06-06T22:04:32.899Z] .................                                                        [ 57%]
[2022-06-06T22:04:39.583Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 60%]
[2022-06-06T22:04:51.639Z] finch-master/docs/source/notebooks/dap_subset.ipynb ..........           [ 64%]
[2022-06-06T22:05:01.655Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 66%]
[2022-06-06T22:05:50.993Z] finch-master/docs/source/notebooks/subset.ipynb .....................    [ 75%]
[2022-06-06T22:05:51.259Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 76%]
[2022-06-06T22:05:55.985Z] ......                                                                   [ 78%]
[2022-06-06T22:06:28.135Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 79%]
[2022-06-06T22:06:47.506Z] .............                                                            [ 84%]
[2022-06-06T22:06:55.627Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 85%]
[2022-06-06T22:07:47.530Z] ....s.                                                                   [ 87%]
[2022-06-06T22:07:59.767Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2022-06-06T22:08:03.041Z] ...                                                                      [ 89%]
[2022-06-06T22:08:15.267Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2022-06-06T22:08:53.833Z] ......                                                                   [ 92%]
[2022-06-06T22:09:03.838Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2022-06-06T22:12:13.582Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2022-06-06T22:12:13.582Z] 
[2022-06-06T22:12:13.582Z] ================= 232 passed, 1 skipped in 1011.18s (0:16:51) ==================

crim-jenkins-bot · 2022-06-07T18:44:33Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/923/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : weaver-publish-process
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-4.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/767/

NOTEBOOK TEST RESULTS

    
[2022-06-07T17:59:50.305Z] ============================= test session starts ==============================
[2022-06-07T17:59:50.305Z] platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
[2022-06-07T17:59:50.305Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2022-06-07T17:59:50.305Z] plugins: anyio-3.5.0, dash-2.3.1, nbval-0.9.6, tornasync-0.6.0.post2
[2022-06-07T17:59:50.305Z] collected 244 items
[2022-06-07T17:59:50.305Z] 
[2022-06-07T17:59:58.974Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2022-06-07T18:00:36.353Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2022-06-07T18:00:46.700Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [  9%]
[2022-06-07T18:00:57.441Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 13%]
[2022-06-07T18:00:59.841Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 17%]
[2022-06-07T18:10:27.901Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ..........          [ 21%]
[2022-06-07T18:10:50.043Z] pavics-sdi-master/docs/source/notebooks/eccc_geoapi.ipynb .....          [ 23%]
[2022-06-07T18:11:04.939Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .                 [ 23%]
[2022-06-07T18:11:31.920Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 26%]
[2022-06-07T18:11:32.181Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 26%]
[2022-06-07T18:11:35.328Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .FFFFFF            [ 29%]
[2022-06-07T18:11:42.165Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 31%]
[2022-06-07T18:15:22.853Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 37%]
[2022-06-07T18:16:14.084Z] ..............                                                           [ 43%]
[2022-06-07T18:16:25.198Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 45%]
[2022-06-07T18:16:27.516Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 48%]
[2022-06-07T18:16:51.706Z] .................                                                        [ 55%]
[2022-06-07T18:16:58.139Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 57%]
[2022-06-07T18:16:59.529Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 57%]
[2022-06-07T18:17:10.687Z] ..F....FF.                                                               [ 61%]
[2022-06-07T18:17:23.078Z] finch-master/docs/source/notebooks/dap_subset.ipynb ..........           [ 65%]
[2022-06-07T18:17:32.181Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 68%]
[2022-06-07T18:18:23.569Z] finch-master/docs/source/notebooks/subset.ipynb .....................    [ 77%]
[2022-06-07T18:18:24.503Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 77%]
[2022-06-07T18:18:27.817Z] ......                                                                   [ 79%]
[2022-06-07T18:18:54.396Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 80%]
[2022-06-07T18:19:09.928Z] .............                                                            [ 85%]
[2022-06-07T18:19:18.051Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2022-06-07T18:21:14.591Z] ....s.                                                                   [ 88%]
[2022-06-07T18:21:29.581Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2022-06-07T18:21:33.475Z] ...                                                                      [ 90%]
[2022-06-07T18:21:45.708Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2022-06-07T18:22:14.635Z] ......                                                                   [ 93%]
[2022-06-07T18:22:20.769Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2022-06-07T18:25:45.903Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2022-06-07T18:25:45.903Z] 
[2022-06-07T18:25:45.903Z] =================================== FAILURES ===================================

birdhouse/components/weaver/config/magpie/weaver_hooks.py

crim-jenkins-bot · 2022-06-07T19:58:38Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/924/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : weaver-publish-process
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-4.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/768/

NOTEBOOK TEST RESULTS

    
[2022-06-07T19:25:11.600Z] ============================= test session starts ==============================
[2022-06-07T19:25:11.600Z] platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
[2022-06-07T19:25:11.600Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2022-06-07T19:25:11.600Z] plugins: anyio-3.5.0, dash-2.3.1, nbval-0.9.6, tornasync-0.6.0.post2
[2022-06-07T19:25:11.600Z] collected 233 items
[2022-06-07T19:25:11.600Z] 
[2022-06-07T19:25:19.992Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2022-06-07T19:25:31.870Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2022-06-07T19:25:39.806Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 10%]
[2022-06-07T19:25:47.616Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ......FF       [ 13%]
[2022-06-07T19:25:49.833Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 18%]
[2022-06-07T19:27:43.519Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ..........          [ 22%]
[2022-06-07T19:27:50.121Z] pavics-sdi-master/docs/source/notebooks/eccc_geoapi.ipynb .....          [ 24%]
[2022-06-07T19:28:05.049Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .                 [ 24%]
[2022-06-07T19:28:26.247Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 27%]
[2022-06-07T19:28:27.639Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 27%]
[2022-06-07T19:28:33.241Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 30%]
[2022-06-07T19:28:39.653Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 33%]
[2022-06-07T19:31:18.150Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 39%]
[2022-06-07T19:32:12.196Z] ..............                                                           [ 45%]
[2022-06-07T19:32:17.378Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 47%]
[2022-06-07T19:32:18.850Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 50%]
[2022-06-07T19:32:42.321Z] .................                                                        [ 57%]
[2022-06-07T19:32:48.744Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 60%]
[2022-06-07T19:32:59.798Z] finch-master/docs/source/notebooks/dap_subset.ipynb ..........           [ 64%]
[2022-06-07T19:33:08.669Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 66%]
[2022-06-07T19:33:55.267Z] finch-master/docs/source/notebooks/subset.ipynb .....................    [ 75%]
[2022-06-07T19:33:56.212Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 76%]
[2022-06-07T19:33:59.592Z] ......                                                                   [ 78%]
[2022-06-07T19:34:21.551Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 79%]
[2022-06-07T19:34:33.608Z] .............                                                            [ 84%]
[2022-06-07T19:34:43.584Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 85%]
[2022-06-07T19:35:25.936Z] ....s.                                                                   [ 87%]
[2022-06-07T19:35:34.066Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2022-06-07T19:35:38.718Z] ...                                                                      [ 89%]
[2022-06-07T19:35:50.964Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2022-06-07T19:36:19.961Z] ......                                                                   [ 92%]
[2022-06-07T19:36:27.386Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2022-06-07T19:39:47.090Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2022-06-07T19:39:47.091Z] 
[2022-06-07T19:39:47.091Z] =================================== FAILURES ===================================

fmigneault · 2022-06-08T21:02:50Z

run tests

fmigneault · 2022-06-08T21:10:50Z

@tlvu
I will bundle #244, #245 and this PR together once I get your approval.
You can focus only on this change: https://github.com/bird-house/birdhouse-deploy/pull/247/files#diff-35f094ca9709a11d6e791bdd0b311be7fd4e57ce56a5c00fa42582f1f0aa10dcR172-R281
The rest you have reviewed it already.

crim-jenkins-bot · 2022-06-08T21:49:28Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/942/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : weaver-publish-process
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-4.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/771/

NOTEBOOK TEST RESULTS

    
[2022-06-08T21:15:42.020Z] ============================= test session starts ==============================
[2022-06-08T21:15:42.020Z] platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
[2022-06-08T21:15:42.020Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2022-06-08T21:15:42.020Z] plugins: anyio-3.5.0, dash-2.3.1, nbval-0.9.6, tornasync-0.6.0.post2
[2022-06-08T21:15:42.020Z] collected 244 items
[2022-06-08T21:15:42.020Z] 
[2022-06-08T21:15:49.852Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2022-06-08T21:16:07.098Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2022-06-08T21:16:14.633Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [  9%]
[2022-06-08T21:16:23.731Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 13%]
[2022-06-08T21:16:25.943Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 17%]
[2022-06-08T21:17:51.144Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ..........          [ 21%]
[2022-06-08T21:18:03.153Z] pavics-sdi-master/docs/source/notebooks/eccc_geoapi.ipynb .....          [ 23%]
[2022-06-08T21:18:18.028Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .                 [ 23%]
[2022-06-08T21:18:39.196Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 26%]
[2022-06-08T21:18:40.578Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 26%]
[2022-06-08T21:18:45.855Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 29%]
[2022-06-08T21:18:52.734Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 31%]
[2022-06-08T21:21:15.971Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 37%]
[2022-06-08T21:22:05.769Z] ..............                                                           [ 43%]
[2022-06-08T21:22:11.526Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 45%]
[2022-06-08T21:22:13.794Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 48%]
[2022-06-08T21:22:36.862Z] .................                                                        [ 55%]
[2022-06-08T21:22:43.017Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 57%]
[2022-06-08T21:22:44.409Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 57%]
[2022-06-08T21:22:55.333Z] .......FF.                                                               [ 61%]
[2022-06-08T21:23:06.852Z] finch-master/docs/source/notebooks/dap_subset.ipynb ..........           [ 65%]
[2022-06-08T21:23:15.683Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 68%]
[2022-06-08T21:24:03.602Z] finch-master/docs/source/notebooks/subset.ipynb .....................    [ 77%]
[2022-06-08T21:24:04.981Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 77%]
[2022-06-08T21:24:08.094Z] ......                                                                   [ 79%]
[2022-06-08T21:24:30.043Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 80%]
[2022-06-08T21:24:42.714Z] .............                                                            [ 85%]
[2022-06-08T21:24:50.830Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2022-06-08T21:25:29.892Z] ....s.                                                                   [ 88%]
[2022-06-08T21:25:38.066Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2022-06-08T21:25:43.206Z] ...                                                                      [ 90%]
[2022-06-08T21:25:53.194Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2022-06-08T21:26:23.230Z] ......                                                                   [ 93%]
[2022-06-08T21:26:31.879Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2022-06-08T21:29:48.696Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2022-06-08T21:29:48.696Z] 
[2022-06-08T21:29:48.696Z] =================================== FAILURES ===================================

tlvu · 2022-06-08T21:55:53Z

@tlvu I will bundle #244, #245 and this PR together once I get your approval. You can focus only on this change: https://github.com/bird-house/birdhouse-deploy/pull/247/files#diff-35f094ca9709a11d6e791bdd0b311be7fd4e57ce56a5c00fa42582f1f0aa10dcR172-R281 The rest you have reviewed it already.

Oh thank you, I was a bit lost with all the PR with many similar code. That change LTGM.

tlvu

LGTM

crim-jenkins-bot · 2022-06-08T22:57:40Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/943/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : weaver-publish-process
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL :

NOTEBOOK TEST RESULTS

    
</code>

fmigneault added 11 commits May 12, 2022 15:21

prepare magpie/twitcher/weaver requests hooks for WPS outputs in user…

f1a97c1

… workspace directory (relates to Ouranosinc/Magpie#517 and bird-house/twitcher#114)

fix invalid paths + remove test code + add more details/case handling

e1c3f73

fix wrong file name for weaver hooks

85e3faa

weaver process filtering implementation

9470caf

adjust permission and node parsing for weaver processes filtered by a…

180ee5e

…llowed permission to read them

fix filter process in case of anonymous (no user authenticated)

2af0d5d

bump magpie 3.26.0 to provide context paramter in response hook

9b9e4e8

adjust weaver request/response hooks volume-mounts to load MagpieAdap…

40d6ff9

…ter configs (within Twitcher) using directory loading strategy

document sample JSON body for weaver processes listing used in hooks

28ad95c

documentation reference link about hooks definition and usage

5c4f9b2

auto-create user perms for deployed weaver process

e661ef2

github-actions bot added the ci/operations Continuous Integration components label Jun 6, 2022

fmigneault requested review from tlvu and matprov June 6, 2022 21:44

fmigneault added 2 commits June 7, 2022 13:58

update changes

1ccc3c1

Merge remote-tracking branch 'origin/master' into weaver-publish-process

de7e592

matprov reviewed Jun 7, 2022

View reviewed changes

birdhouse/components/weaver/config/magpie/weaver_hooks.py Outdated Show resolved Hide resolved

fix typo

52e7d63

fmigneault self-assigned this Jun 7, 2022

matprov approved these changes Jun 8, 2022

View reviewed changes

tlvu approved these changes Jun 8, 2022

View reviewed changes

Bump version: 1.18.13 → 1.19.0

c67c003

github-actions bot added the documentation Improvements or additions to documentation label Jun 8, 2022

fmigneault merged commit 9bfa486 into master Jun 8, 2022

fmigneault deleted the weaver-publish-process branch June 8, 2022 22:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto user permissions creation over its deployed process #247

Auto user permissions creation over its deployed process #247

fmigneault commented Jun 6, 2022 •

edited

Loading

fmigneault commented Jun 6, 2022 •

edited

Loading

crim-jenkins-bot commented Jun 6, 2022

crim-jenkins-bot commented Jun 7, 2022

crim-jenkins-bot commented Jun 7, 2022

fmigneault commented Jun 8, 2022

fmigneault commented Jun 8, 2022

crim-jenkins-bot commented Jun 8, 2022

tlvu commented Jun 8, 2022

tlvu left a comment

crim-jenkins-bot commented Jun 8, 2022

Auto user permissions creation over its deployed process #247

Auto user permissions creation over its deployed process #247

Conversation

fmigneault commented Jun 6, 2022 • edited Loading

Changes

Related Issue / Discussion

Additional Information

Results / Demonstration

fmigneault commented Jun 6, 2022 • edited Loading

crim-jenkins-bot commented Jun 6, 2022

E2E Test Results

DACCS-iac Pipeline Results

PAVICS-e2e-workflow-tests Pipeline Results

crim-jenkins-bot commented Jun 7, 2022

E2E Test Results

DACCS-iac Pipeline Results

PAVICS-e2e-workflow-tests Pipeline Results

crim-jenkins-bot commented Jun 7, 2022

E2E Test Results

DACCS-iac Pipeline Results

PAVICS-e2e-workflow-tests Pipeline Results

fmigneault commented Jun 8, 2022

fmigneault commented Jun 8, 2022

crim-jenkins-bot commented Jun 8, 2022

E2E Test Results

DACCS-iac Pipeline Results

PAVICS-e2e-workflow-tests Pipeline Results

tlvu commented Jun 8, 2022

tlvu left a comment

Choose a reason for hiding this comment

crim-jenkins-bot commented Jun 8, 2022

E2E Test Results

DACCS-iac Pipeline Results

PAVICS-e2e-workflow-tests Pipeline Results

fmigneault commented Jun 6, 2022 •

edited

Loading

fmigneault commented Jun 6, 2022 •

edited

Loading