Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Autodeploy: optionally fix file permissions #343

Merged
merged 5 commits into from
Jun 16, 2023
Merged

Autodeploy: optionally fix file permissions #343

merged 5 commits into from
Jun 16, 2023

Conversation

mishaschwartz
Copy link
Collaborator

@mishaschwartz mishaschwartz commented Jun 14, 2023 •
edited by tlvu
Loading

Overview

The autodeploy mechanism creates new files owned by root. If this is not desired then users have to manually update the file ownership after each autodeployment. This adds an option to change the ownership of all files to a specific user after each autodeployment.

For example, if the code in this repo is currently owned by a user named birduser with uid 1002, then by setting export AUTODEPLOY_CODE_OWNERSHIP="1002:1002" in env.local, all files and folders in this repo will continue to be owned by birduser after each autodeployment.

Changes

Non-breaking changes

adds an optional environment variable to env.local

Breaking changes

None

Related Issue / Discussion

Additional Information

@mishaschwartz mishaschwartz requested a review from tlvu June 14, 2023 16:22
@github-actions github-actions bot added ci/deployment Related to deployment utilities and scripts documentation Improvements or additions to documentation labels Jun 14, 2023
tlvu
tlvu approved these changes Jun 14, 2023
View reviewed changes
Copy link
Collaborator

@tlvu tlvu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks !

@tlvu
Copy link
Collaborator

tlvu commented Jun 14, 2023

Humm, funny this permission problem is the root cause since birdhouse/deployment/fix-write-perm was meant to specifically fix this kind of permission problem. Or maybe I did not fully understand the usage of setfacl. Thanks for catching this.

@tlvu
Copy link
Collaborator

tlvu commented Jun 14, 2023

I think this fixes #342

@crim-jenkins-bot
Copy link
Collaborator

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1646/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : autoupdate-permission-fix
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-20.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1185/

NOTEBOOK TEST RESULTS 
    
[2023-06-14T17:00:51.532Z] ============================= test session starts ==============================
[2023-06-14T17:00:51.532Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-06-14T17:00:51.532Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-06-14T17:00:51.532Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-06-14T17:00:51.532Z] collected 245 items
[2023-06-14T17:00:51.532Z] 
[2023-06-14T17:01:00.533Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2023-06-14T17:01:09.123Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2023-06-14T17:01:19.447Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [  9%]
[2023-06-14T17:01:27.771Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 13%]
[2023-06-14T17:01:31.072Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 17%]
[2023-06-14T17:09:01.484Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 22%]
[2023-06-14T17:09:01.484Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 22%]
[2023-06-14T17:09:08.577Z] ...............                                                          [ 28%]
[2023-06-14T17:09:18.525Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 30%]
[2023-06-14T17:09:25.874Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .....F            [ 33%]
[2023-06-14T17:09:41.827Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 35%]
[2023-06-14T17:09:43.468Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 35%]
[2023-06-14T17:09:48.222Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 38%]
[2023-06-14T17:09:52.744Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 40%]
[2023-06-14T17:13:24.809Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 46%]
[2023-06-14T17:14:41.801Z] .............                                                            [ 52%]
[2023-06-14T17:14:44.494Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 53%]
[2023-06-14T17:14:46.711Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 57%]
[2023-06-14T17:15:03.616Z] .................                                                        [ 64%]
[2023-06-14T17:15:09.476Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 66%]
[2023-06-14T17:15:11.391Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 66%]
[2023-06-14T17:15:20.224Z] .FF......                                                                [ 70%]
[2023-06-14T17:15:31.470Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 74%]
[2023-06-14T17:15:40.695Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 77%]
[2023-06-14T17:15:42.078Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 77%]
[2023-06-14T17:15:45.137Z] ......                                                                   [ 80%]
[2023-06-14T17:15:53.263Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 80%]
[2023-06-14T17:16:09.900Z] .............                                                            [ 85%]
[2023-06-14T17:16:19.900Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-06-14T17:16:56.787Z] ....s.                                                                   [ 88%]
[2023-06-14T17:17:04.917Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2023-06-14T17:17:20.445Z] ...                                                                      [ 90%]
[2023-06-14T17:17:32.672Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-06-14T17:17:57.162Z] ......                                                                   [ 93%]
[2023-06-14T17:17:59.043Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-06-14T17:21:03.318Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2023-06-14T17:21:03.318Z] 
[2023-06-14T17:21:03.318Z] =================================== FAILURES ===================================

@mishaschwartz
Copy link
Collaborator Author

I think this fixes #342

I just double checked and unfortunately it does not fix that issue

@github-actions github-actions bot added the ci/operations Continuous Integration components label Jun 16, 2023
@mishaschwartz mishaschwartz merged commit 5f22cb7 into master Jun 16, 2023
@mishaschwartz mishaschwartz deleted the autoupdate-permission-fix branch June 16, 2023 17:04
@crim-jenkins-bot
Copy link
Collaborator

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1683/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : autoupdate-permission-fix
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-133.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1200/

NOTEBOOK TEST RESULTS 
    
[2023-06-16T17:17:46.075Z] ============================= test session starts ==============================
[2023-06-16T17:17:46.075Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2023-06-16T17:17:46.075Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2023-06-16T17:17:46.075Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2023-06-16T17:17:46.075Z] collected 245 items
[2023-06-16T17:17:46.075Z] 
[2023-06-16T17:17:55.338Z] notebooks-auth/test_thredds.ipynb ...........                            [  4%]
[2023-06-16T17:18:10.247Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [  7%]
[2023-06-16T17:18:19.957Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [  9%]
[2023-06-16T17:18:25.893Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 13%]
[2023-06-16T17:18:29.009Z] pavics-sdi-master/docs/source/notebooks/WPS_example.ipynb ..........     [ 17%]
[2023-06-16T17:27:44.027Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 22%]
[2023-06-16T17:27:44.028Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 22%]
[2023-06-16T17:27:51.707Z] ...............                                                          [ 28%]
[2023-06-16T17:28:00.688Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 30%]
[2023-06-16T17:28:08.226Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb .....F            [ 33%]
[2023-06-16T17:28:23.089Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 35%]
[2023-06-16T17:28:24.465Z] pavics-sdi-master/docs/source/notebooks/jupyter_extensions.ipynb .       [ 35%]
[2023-06-16T17:28:29.484Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 38%]
[2023-06-16T17:28:33.901Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 40%]
[2023-06-16T17:32:17.639Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 46%]
[2023-06-16T17:33:32.816Z] .............                                                            [ 52%]
[2023-06-16T17:33:34.466Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 53%]
[2023-06-16T17:33:36.673Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 57%]
[2023-06-16T17:33:54.602Z] .................                                                        [ 64%]
[2023-06-16T17:34:00.326Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb .....           [ 66%]
[2023-06-16T17:34:01.709Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 66%]
[2023-06-16T17:34:03.067Z] .FFFFFFFF                                                                [ 70%]
[2023-06-16T17:34:14.343Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 74%]
[2023-06-16T17:34:23.839Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 77%]
[2023-06-16T17:34:25.231Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 77%]
[2023-06-16T17:34:28.602Z] ......                                                                   [ 80%]
[2023-06-16T17:34:35.177Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 80%]
[2023-06-16T17:34:50.847Z] .............                                                            [ 85%]
[2023-06-16T17:35:00.853Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2023-06-16T17:35:50.825Z] ....s.                                                                   [ 88%]
[2023-06-16T17:35:58.978Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 88%]
[2023-06-16T17:36:14.476Z] ...                                                                      [ 90%]
[2023-06-16T17:36:26.691Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2023-06-16T17:36:45.924Z] ......                                                                   [ 93%]
[2023-06-16T17:36:48.343Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2023-06-16T17:39:52.504Z] notebooks/stress-tests.ipynb .....                                       [100%]
[2023-06-16T17:39:52.504Z] 
[2023-06-16T17:39:52.504Z] =================================== FAILURES ===================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tlvu tlvu tlvu approved these changes

@fmigneault fmigneault fmigneault approved these changes

Assignees
No one assigned
Labels
ci/deployment Related to deployment utilities and scripts ci/operations Continuous Integration components documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mishaschwartz @tlvu @crim-jenkins-bot @fmigneault