Run convert_grib2_to_nc.py inside a Docker container for reproducible runtime environment #21
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tlogan2000
approved these changes
Dec 17, 2020
|
@tlvu this looks fine. up to you if you want to merge right away or wait until we talk this pm? |
tlvu
added a commit
to bird-house/birdhouse-deploy-ouranos
that referenced
this pull request
Dec 18, 2020
Add geps forecasts scheduler jobs. Matching PR to automate deployment and future updates of the PR Ouranosinc/pavics-vdb#21. Requires PR bird-house/birdhouse-deploy#110. Part of issue Ouranosinc/pavics-vdb#19 and issue Ouranosinc/raven#313. Add a cronjob (`retrieve_geps_forecasts.env`) and a config file (`retrieve_geps_forecasts.yml`) that defines which git repo to pull from and what command to run to handle the geps forecasts. Leverage the script I did earlier that can deploy any files from any git repos to a PAVICS host. Activating the cronjob via `env.local` is like so https://github.com/bird-house/birdhouse-deploy/blob/2bb9ce6756cb317c47e28a3a65d95d62312c02fd/birdhouse/env.local.example#L156-L158 @tlogan2000 If you change the code in the `pavics-vdb` repo, it'll be live the next time the cronjob runs. If you want to change the cronjob frequency, edit `retrieve_geps_forecasts.env` and it autodeploy itself. Deployed to our staging env Medus: http://medus.ouranos.ca:8083/twitcher/ows/proxy/thredds/catalog/birdhouse/eccc/forecasts/geps/catalog.html. @huard I'll apply this exact same recipe for your Intake catalog project. @matprov This is an example showing you how CRIM can have custom cronjobs, all source controlled, centrally tracked by `env,local`, with automated deployment and update. Sample cronjob logs from Medus: [retrieve_geps_forecasts.log](https://github.com/bird-house/birdhouse-deploy-ouranos/files/5716267/retrieve_geps_forecasts.log) ``` ========== datadeploy START_TIME=2020-12-18T05:37:02+0000 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.checkout_cache + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.checkout_cache + CHECKOUT_CACHE= + '[' -z ] + CHECKOUT_CACHE=/data/deploy_data_cache/retrieve_geps_forecasts + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.git_ssh_identity_file + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.git_ssh_identity_file + GIT_SSH_IDENTITY_FILE= + '[' -z ] + GIT_SSH_IDENTITY_FILE=/home/mourad/.ssh/id_rsa_pavics-vdb-ro + '[' '!' -z /home/mourad/.ssh/id_rsa_pavics-vdb-ro ] + export 'GIT_SSH_COMMAND=ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=/home/mourad/.ssh/id_rsa_pavics-vdb-ro' + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[*].repo_url' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[*].repo_url' + GIT_REPO_URLS=git@github.com:Ouranosinc/pavics-vdb.git + ensure_not_empty git@github.com:Ouranosinc/pavics-vdb.git + '[' -z git@github.com:Ouranosinc/pavics-vdb.git ] + REPO_NUM=0 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --defaultValue origin/master 'deploy[0].branch' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --defaultValue origin/master 'deploy[0].branch' + GIT_BRANCH=origin/minor-enhancements-to-convert_grib2_to_nc.py + ensure_not_empty origin/minor-enhancements-to-convert_grib2_to_nc.py + '[' -z origin/minor-enhancements-to-convert_grib2_to_nc.py ] + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].checkout_name' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].checkout_name' + GIT_CHECKOUT_NAME=pavics-vdb-for-geps-forecasts + ensure_not_empty pavics-vdb-for-geps-forecasts + '[' -z pavics-vdb-for-geps-forecasts ] + CLONE_DEST=/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts + '[' '!' -d /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts ] + echo 'refresh repo '"'"'/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts'"'"' on branch '"'"'origin/minor-enhancements-to-convert_grib2_to_nc.py'"'" refresh repo '/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts' on branch 'origin/minor-enhancements-to-convert_grib2_to_nc.py' + cd /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts + git remote -v origin git@github.com:Ouranosinc/pavics-vdb.git (fetch) origin git@github.com:Ouranosinc/pavics-vdb.git (push) + git clean -fdx + git fetch --prune --all Fetching origin Warning: Permanently added 'github.com,140.82.114.3' (RSA) to the list of known hosts. From github.com:Ouranosinc/pavics-vdb + 3e94dfb...a64a444 minor-enhancements-to-convert_grib2_to_nc.py -> origin/minor-enhancements-to-convert_grib2_to_nc.py (forced update) a6e4c2b..a64a444 self-contain-geps-forecasts-runtime -> origin/self-contain-geps-forecasts-runtime + git checkout --force origin/minor-enhancements-to-convert_grib2_to_nc.py Warning: you are leaving 1 commit behind, not connected to any of your branches: 3e94dfb convert_grib2_to_nc.py: toogle dask progress bar to avoid noise in logs If you want to keep it by creating a new branch, this may be a good time to do so with: git branch <new-branch-name> 3e94dfb HEAD is now at a64a444 convert_grib2_to_nc.py: toogle dask progress bar to avoid noise in logs + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].dir_maps[*].source_dir' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].dir_maps[*].source_dir' + SRC_DIRS= + DIR_NUM=0 + yq r /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --length 'deploy[0].post_actions' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --length 'deploy[0].post_actions' + POST_ACTIONS_LENGTH=1 + '[' -z 1 ] + expr 1 - 1 + POST_ACTIONS_LENGTH_FOR_SEQ=0 + seq 0 0 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].post_actions[0].action' + docker run --rm --name deploy_data_yq_2020-12-18T05_37_02p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].post_actions[0].action' + POST_ACTION='RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + ensure_not_empty 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + '[' -z 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' ] + echo 'executing post_action '"'"'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc'"'" executing post_action 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + eval 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH}' 'RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH}' 'RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH}' 'CONVERT_GRIB2_TO_NC_PROGRESSBAR=false' ECCC-datamart_sync/run_convert_grib2_to_nc + RUN_CONVERT_GRIB2_TO_NC_INPATH= RUN_CONVERT_GRIB2_TO_NC_OUTPATH= RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=/data/datasets/eccc/forecasts/geps CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc ========== run_convert_grib2_to_nc START_TIME=2020-12-18T05:37:25+0000 + id -u + USER_ID=0 + id -g + GROUP_ID=0 + mkdir -p /data/tmp/geps_forecast/grib2 + mkdir -p /data/tmp/geps_forecast/netcdf + docker run --rm --name run_convert_grib2_to_nc --volume /data/tmp/geps_forecast/grib2:/data/tmp/geps_forecast/grib2:rw --volume /data/tmp/geps_forecast/netcdf:/data/tmp/geps_forecast/netcdf:rw --volume /data/datasets/eccc/forecasts/geps:/data/datasets/eccc/forecasts/geps:rw --volume /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py:/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py:ro --env 'CONVERT_GRIB2_TO_NC_INPATH=/data/tmp/geps_forecast/grib2' --env 'CONVERT_GRIB2_TO_NC_OUTPATH=/data/tmp/geps_forecast/netcdf' --env 'CONVERT_GRIB2_TO_NC_THREDDSPATH=/data/datasets/eccc/forecasts/geps' --env CONVERT_GRIB2_TO_NC_PROGRESSBAR --user 0:0 pavics/workflow-tests:201214 python /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py Downloading grib2 files : attempt 1 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 193 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files Downloading grib2 files : attempt 2 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files Downloading grib2 files : attempt 3 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files coverting to netcdf .... converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P159_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P162_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P165_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P168_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P036_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P039_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P042_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P045_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P096_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P099_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P102_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P105_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P099_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P102_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P105_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P108_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P294_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P300_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P306_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P312_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P240_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P246_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P252_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P258_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P027_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P030_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P033_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P036_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P222_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P228_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P234_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P240_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P168_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P171_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P174_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P177_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P051_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P054_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P057_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P060_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P048_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P051_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P054_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P057_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P120_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P123_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P126_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P129_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P312_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P318_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P324_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P330_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P135_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P138_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P141_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P144_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P060_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P063_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P066_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P069_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P132_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P135_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P138_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P141_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P360_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P366_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P372_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P378_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P087_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P090_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P093_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P096_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P342_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P348_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P354_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P360_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P288_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P294_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P300_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P306_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P063_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P066_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P069_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P072_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P318_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P324_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P330_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P336_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P192_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P198_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P204_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P210_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P111_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P114_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P117_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P120_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P366_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P372_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P378_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P384_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P264_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P270_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P276_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P282_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P171_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P174_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P177_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P180_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P000_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P003_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P006_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P009_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P084_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P087_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P090_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P093_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P384_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P147_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P150_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P153_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P156_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P012_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P015_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P018_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P021_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P108_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P111_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P114_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P117_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P123_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P126_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P129_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P132_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P024_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P027_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P030_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P033_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P072_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P075_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P078_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P081_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P336_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P342_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P348_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P354_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P075_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P078_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P081_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P084_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P270_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P276_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P282_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P288_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P216_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P222_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P228_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P234_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P015_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P018_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P021_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P024_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P198_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P204_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P210_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P216_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P144_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P147_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P150_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P153_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P039_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P042_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P045_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P048_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P246_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P252_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P258_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P264_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P180_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P183_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P186_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P189_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P003_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P006_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P009_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P012_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P183_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P186_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P189_allmbrs.grib2 converting CMC_geps-raw_APCP_SFC_0_latlon0p5x0p5_2020121800_P192_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P156_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P159_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P162_allmbrs.grib2 converting CMC_geps-raw_TMP_TGL_2m_latlon0p5x0p5_2020121800_P165_allmbrs.grib2 done coverting GEPS . It took : 222.2973051071167 seconds 2020121400 : no action needed for combining variables and timesteps 2020121412 : no action needed for combining variables and timesteps 2020121500 : no action needed for combining variables and timesteps 2020121512 : no action needed for combining variables and timesteps 2020121600 : no action needed for combining variables and timesteps 2020121612 : no action needed for combining variables and timesteps 2020121700 : no action needed for combining variables and timesteps 2020121712 : no action needed for combining variables and timesteps 2020121800 : combining variables and timesteps ... CMC_geps-raw_latlon0p5x0p5_2020121800_allP_allmbrs.nc Traceback (most recent call last): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 153, in validate_ncml assert ds.reftime.values == pd.to_datetime(start_date, format='%Y%m%d%H') AssertionError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 336, in <module> main() File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 141, in main if validate_ncml(opendap_latest, latest_date): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 157, in validate_ncml raise Exception("can't read ncml opendap link") Exception: can't read ncml opendap link + cleanup_on_exit + set +x run_convert_grib2_to_nc finished START_TIME=2020-12-18T05:37:25+0000 run_convert_grib2_to_nc finished END_TIME=2020-12-18T05:47:11+0000 + expr 0 + 1 + REPO_NUM=1 + cleanup_on_exit + set +x datadeploy finished START_TIME=2020-12-18T05:37:02+0000 datadeploy finished END_TIME=2020-12-18T05:47:12+0000 ========== datadeploy START_TIME=2020-12-18T06:37:06+0000 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.checkout_cache + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.checkout_cache + CHECKOUT_CACHE= + '[' -z ] + CHECKOUT_CACHE=/data/deploy_data_cache/retrieve_geps_forecasts + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.git_ssh_identity_file + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml config.git_ssh_identity_file + GIT_SSH_IDENTITY_FILE= + '[' -z ] + GIT_SSH_IDENTITY_FILE=/home/mourad/.ssh/id_rsa_pavics-vdb-ro + '[' '!' -z /home/mourad/.ssh/id_rsa_pavics-vdb-ro ] + export 'GIT_SSH_COMMAND=ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=/home/mourad/.ssh/id_rsa_pavics-vdb-ro' + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[*].repo_url' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[*].repo_url' + GIT_REPO_URLS=git@github.com:Ouranosinc/pavics-vdb.git + ensure_not_empty git@github.com:Ouranosinc/pavics-vdb.git + '[' -z git@github.com:Ouranosinc/pavics-vdb.git ] + REPO_NUM=0 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --defaultValue origin/master 'deploy[0].branch' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --defaultValue origin/master 'deploy[0].branch' + GIT_BRANCH=origin/minor-enhancements-to-convert_grib2_to_nc.py + ensure_not_empty origin/minor-enhancements-to-convert_grib2_to_nc.py + '[' -z origin/minor-enhancements-to-convert_grib2_to_nc.py ] + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].checkout_name' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].checkout_name' + GIT_CHECKOUT_NAME=pavics-vdb-for-geps-forecasts + ensure_not_empty pavics-vdb-for-geps-forecasts + '[' -z pavics-vdb-for-geps-forecasts ] + CLONE_DEST=/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts + '[' '!' -d /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts ] + echo 'refresh repo '"'"'/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts'"'"' on branch '"'"'origin/minor-enhancements-to-convert_grib2_to_nc.py'"'" refresh repo '/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts' on branch 'origin/minor-enhancements-to-convert_grib2_to_nc.py' + cd /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts + git remote -v origin git@github.com:Ouranosinc/pavics-vdb.git (fetch) origin git@github.com:Ouranosinc/pavics-vdb.git (push) + git clean -fdx + git fetch --prune --all Fetching origin Warning: Permanently added 'github.com,140.82.114.4' (RSA) to the list of known hosts. + git checkout --force origin/minor-enhancements-to-convert_grib2_to_nc.py HEAD is now at a64a444 convert_grib2_to_nc.py: toogle dask progress bar to avoid noise in logs + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].dir_maps[*].source_dir' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].dir_maps[*].source_dir' + SRC_DIRS= + DIR_NUM=0 + yq r /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --length 'deploy[0].post_actions' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r /home/mourad/PROJECTS/retrieve_geps_forecasts.yml --length 'deploy[0].post_actions' + POST_ACTIONS_LENGTH=1 + '[' -z 1 ] + expr 1 - 1 + POST_ACTIONS_LENGTH_FOR_SEQ=0 + seq 0 0 + yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].post_actions[0].action' + docker run --rm --name deploy_data_yq_2020-12-18T06_37_06p0000 -v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml:/home/mourad/PROJECTS/retrieve_geps_forecasts.yml:ro mikefarah/yq:3.3.4 yq r -p v /home/mourad/PROJECTS/retrieve_geps_forecasts.yml 'deploy[0].post_actions[0].action' + POST_ACTION='RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + ensure_not_empty 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + '[' -z 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' ] + echo 'executing post_action '"'"'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc'"'" executing post_action 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH} RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH} RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH} CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc' + eval 'RUN_CONVERT_GRIB2_TO_NC_INPATH=${RUN_CONVERT_GRIB2_TO_NC_INPATH}' 'RUN_CONVERT_GRIB2_TO_NC_OUTPATH=${RUN_CONVERT_GRIB2_TO_NC_OUTPATH}' 'RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=${RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH}' 'CONVERT_GRIB2_TO_NC_PROGRESSBAR=false' ECCC-datamart_sync/run_convert_grib2_to_nc + RUN_CONVERT_GRIB2_TO_NC_INPATH= RUN_CONVERT_GRIB2_TO_NC_OUTPATH= RUN_CONVERT_GRIB2_TO_NC_THREDDSPATH=/data/datasets/eccc/forecasts/geps CONVERT_GRIB2_TO_NC_PROGRESSBAR=false ECCC-datamart_sync/run_convert_grib2_to_nc ========== run_convert_grib2_to_nc START_TIME=2020-12-18T06:37:30+0000 + id -u + USER_ID=0 + id -g + GROUP_ID=0 + mkdir -p /data/tmp/geps_forecast/grib2 + mkdir -p /data/tmp/geps_forecast/netcdf + docker run --rm --name run_convert_grib2_to_nc --volume /data/tmp/geps_forecast/grib2:/data/tmp/geps_forecast/grib2:rw --volume /data/tmp/geps_forecast/netcdf:/data/tmp/geps_forecast/netcdf:rw --volume /data/datasets/eccc/forecasts/geps:/data/datasets/eccc/forecasts/geps:rw --volume /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py:/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py:ro --env 'CONVERT_GRIB2_TO_NC_INPATH=/data/tmp/geps_forecast/grib2' --env 'CONVERT_GRIB2_TO_NC_OUTPATH=/data/tmp/geps_forecast/netcdf' --env 'CONVERT_GRIB2_TO_NC_THREDDSPATH=/data/datasets/eccc/forecasts/geps' --env CONVERT_GRIB2_TO_NC_PROGRESSBAR --user 0:0 pavics/workflow-tests:201214 python /data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py Downloading grib2 files : attempt 1 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files Downloading grib2 files : attempt 2 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files Downloading grib2 files : attempt 3 of 3 Checking for updated GEPS files : Forecast 20201218 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201217 12 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 00 Done. Found 0 new files Checking for updated GEPS files : Forecast 20201216 12 Done. Found 0 new files coverting to netcdf .... done coverting GEPS . It took : 2.776029586791992 seconds 2020121400 : no action needed for combining variables and timesteps 2020121412 : no action needed for combining variables and timesteps 2020121500 : no action needed for combining variables and timesteps 2020121512 : no action needed for combining variables and timesteps 2020121600 : no action needed for combining variables and timesteps 2020121612 : no action needed for combining variables and timesteps 2020121700 : no action needed for combining variables and timesteps 2020121712 : no action needed for combining variables and timesteps 2020121800 : no action needed for combining variables and timesteps Traceback (most recent call last): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 153, in validate_ncml assert ds.reftime.values == pd.to_datetime(start_date, format='%Y%m%d%H') AssertionError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 336, in <module> main() File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 141, in main if validate_ncml(opendap_latest, latest_date): File "/data/deploy_data_cache/retrieve_geps_forecasts/pavics-vdb-for-geps-forecasts/ECCC-datamart_sync/convert_grib2_to_nc.py", line 157, in validate_ncml raise Exception("can't read ncml opendap link") Exception: can't read ncml opendap link + cleanup_on_exit + set +x run_convert_grib2_to_nc finished START_TIME=2020-12-18T06:37:30+0000 run_convert_grib2_to_nc finished END_TIME=2020-12-18T06:37:48+0000 + expr 0 + 1 + REPO_NUM=1 + cleanup_on_exit + set +x datadeploy finished START_TIME=2020-12-18T06:37:06+0000 datadeploy finished END_TIME=2020-12-18T06:37:48+0000 ```
tlvu
added a commit
to Ouranosinc/PAVICS-e2e-workflow-tests
that referenced
this pull request
Jan 12, 2021
docker: new build to get updates and add cfgrib `cfgrib` added for PR Ouranosinc/pavics-vdb#21. Jenkins build with known error only: http://jenkins.ouranos.ca/job/PAVICS-e2e-workflow-tests/job/new-docker-build-with-cfgrib/2/console Corresponding test fix Ouranosinc/pavics-sdi#195. Corresponding PR to deploy to PAVICS bird-house/birdhouse-deploy#112. Relevant changes: ```diff > - cfgrib=0.9.8.5=pyhd8ed1ab_0 < - clisops=0.3.1=pyh32f6830_1 > - clisops=0.4.0=pyhd3deb0d_0 < - dask=2.30.0=py_0 > - dask=2020.12.0=pyhd8ed1ab_0 < - owslib=0.20.0=py_0 > - owslib=0.21.0=pyhd8ed1ab_0 < - xarray=0.16.1=py_0 > - xarray=0.16.2=pyhd8ed1ab_0 < - xclim=0.21.0=py_0 > - xclim=0.22.0=pyhd8ed1ab_0 < - jupyter_conda=3.4.1=pyh9f0ad1d_0 > - jupyter_conda=4.1.0=hd8ed1ab_1 ``` Full `conda env export` diff: [201111-201214-conda-env-export.diff.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/5706860/201111-201214-conda-env-export.diff.txt) ```diff 3d2 < - plotly 12c11 < - ansi2html=1.5.2=py38h32f6830_2 --- > - ansi2html=1.6.0=py38h578d9bd_0 14c13 < - argon2-cffi=20.1.0=py38h1e0a361_2 --- > - argon2-cffi=20.1.0=py38h25fe258_2 23,24c22,23 < - blosc=1.20.1=he1b5a44_0 < - bokeh=2.2.3=py38h32f6830_0 --- > - blosc=1.20.1=h9c3ff4c_0 > - bokeh=2.2.3=py38h578d9bd_0 32,35c31,34 < - brunsli=0.1=he1b5a44_0 < - bzip2=1.0.8=h516909a_3 < - c-ares=1.16.1=h516909a_3 < - ca-certificates=2020.11.8=ha878542_0 --- > - brunsli=0.1=h9c3ff4c_0 > - bzip2=1.0.8=h7f98852_4 > - c-ares=1.17.1=h36c2ea0_0 > - ca-certificates=2020.12.5=ha878542_0 38c37 < - cartopy=0.18.0=py38hc1e57b6_5 --- > - cartopy=0.18.0=py38h9b6dd9e_6 40c39 < - cdtime=3.1.4=py38hac60b08_2 --- > - cdtime=3.1.4=py38h8b240cc_2 42c41 < - certifi=2020.6.20=py38h924ce5b_2 --- > - certifi=2020.12.5=py38h578d9bd_0 44,46c43,47 < - cffi=1.14.3=py38h1bdcb99_1 < - cfitsio=3.470=h2e3daa1_7 < - cftime=1.2.1=py38hab2c0dc_1 --- > - cf_xarray=0.3.1=pyhd3deb0d_0 > - cffi=1.14.4=py38ha312104_0 > - cfgrib=0.9.8.5=pyhd8ed1ab_0 > - cfitsio=3.470=hb418390_7 > - cftime=1.3.0=py38h0b5ebd8_0 48c49 < - charls=2.1.0=he1b5a44_2 --- > - charls=2.1.0=h9c3ff4c_2 51,52c52,53 < - cligj=0.7.0=py_0 < - clisops=0.3.1=pyh32f6830_1 --- > - cligj=0.7.1=pyhd8ed1ab_0 > - clisops=0.4.0=pyhd3deb0d_0 57,58c58,59 < - coverage=5.3=py38h1e0a361_1 < - cryptography=3.2.1=py38h7699a38_0 --- > - coverage=5.3=py38h497a2fe_1 > - cryptography=3.3.1=py38h2b97feb_0 61,63c62,64 < - cytoolz=0.11.0=py38h1e0a361_1 < - dash=1.17.0=pyhd8ed1ab_0 < - dash-core-components=1.13.0=pyhd8ed1ab_1 --- > - cytoolz=0.11.0=py38h25fe258_1 > - dash=1.18.1=pyhd8ed1ab_0 > - dash-core-components=1.14.1=pyhd8ed1ab_0 66,68c67,69 < - dash-table=4.11.0=pyhd8ed1ab_0 < - dask=2.30.0=py_0 < - dask-core=2.30.0=py_0 --- > - dash-table=4.11.1=pyhd8ed1ab_0 > - dask=2020.12.0=pyhd8ed1ab_0 > - dask-core=2020.12.0=pyhd8ed1ab_0 72c73 < - debugpy=1.1.0=py38hadf7658_1 --- > - debugpy=1.2.1=py38h709712a_0 77c78 < - distributed=2.30.1=py38h578d9bd_0 --- > - distributed=2020.12.0=py38h578d9bd_0 80,82c81,84 < - entrypoints=0.3=py38h32f6830_1002 < - esgf-compute-api=2.3.6=pyh9f0ad1d_0 < - esmf=8.0.1=nompi_hbeb3ca6_1 --- > - eccodes=2.19.1=ha0e6eb6_1 > - entrypoints=0.3=pyhd8ed1ab_1003 > - esgf-compute-api=2.3.7=0_h1234567_ > - esmf=8.0.1=nompi_he31a43a_2 86c88 < - fiona=1.8.17=py38h676c6b2_1 --- > - fiona=1.8.18=py38h37fbd03_0 93c95 < - future=0.18.2=py38h32f6830_2 --- > - future=0.18.2=py38h578d9bd_2 108,110c110,112 < - glib=2.66.2=h58526e2_0 < - gmp=6.2.0=h58526e2_4 < - gnutls=3.6.13=h79a8f9a_0 --- > - glib=2.66.3=h58526e2_0 > - gmp=6.2.1=h58526e2_0 > - gnutls=3.6.13=h85f3911_1 115,116c117,118 < - hdf4=4.2.13=hf30be14_1003 < - hdf5=1.10.6=nompi_h54c07f9_1110 --- > - hdf4=4.2.13=h10796ff_1004 > - hdf5=1.10.6=nompi_h6a2412b_1112 118c120 < - holoviews=1.13.5=pyh9f0ad1d_0 --- > - holoviews=1.14.0=pyhd3deb0d_0 122c124 < - imagecodecs=2020.5.30=py38h63741c2_4 --- > - imagecodecs=2020.5.30=py38h67f9a0b_4 124,125c126,127 < - importlib-metadata=2.0.0=py_1 < - importlib_metadata=2.0.0=1 --- > - importlib-metadata=3.3.0=py38h578d9bd_2 > - importlib_metadata=3.3.0=hd8ed1ab_2 128c130 < - ipykernel=5.3.4=py38h1cdfbd6_1 --- > - ipykernel=5.4.2=py38h81c977d_0 137c139 < - joblib=0.17.0=py_0 --- > - joblib=1.0.0=pyhd8ed1ab_0 144,145c146,147 < - jupyter-dash=0.3.0=py_0 < - jupyter_bokeh=2.0.3=py_0 --- > - jupyter-dash=0.3.1=pyhd8ed1ab_0 > - jupyter_bokeh=2.0.4=pyhd8ed1ab_0 147c149 < - jupyter_conda=3.4.1=pyh9f0ad1d_0 --- > - jupyter_conda=4.1.0=hd8ed1ab_1 149,153c151,155 < - jupyter_core=4.6.3=py38h32f6830_2 < - jupyter_server=1.0.5=py38h32f6830_0 < - jupyter_telemetry=0.0.5=py_0 < - jupyterhub=1.2.1=py38h578d9bd_0 < - jupyterhub-base=1.2.1=py38h578d9bd_0 --- > - jupyter_core=4.7.0=py38h578d9bd_0 > - jupyter_server=1.0.10=py38h578d9bd_0 > - jupyter_telemetry=0.1.0=pyhd8ed1ab_1 > - jupyterhub=1.3.0=py38h578d9bd_0 > - jupyterhub-base=1.3.0=py38h578d9bd_0 155c157 < - jupyterlab-git=0.22.3=pyhd8ed1ab_0 --- > - jupyterlab-git=0.23.2=pyhd8ed1ab_0 158,160c160,162 < - jupytext=1.6.0=pyh9f0ad1d_0 < - jxrlib=1.1=h516909a_2 < - kealib=1.4.13=h33137a7_1 --- > - jupytext=1.7.1=pyhd3deb0d_0 > - jxrlib=1.1=h7f98852_2 > - kealib=1.4.14=h0042707_0 162c164 < - krb5=1.17.1=hfafb76e_3 --- > - krb5=1.17.2=h926e7f8_0 164c166 < - lazy-object-proxy=1.5.1=py38h1e0a361_1 --- > - lazy-object-proxy=1.5.2=py38h497a2fe_0 166,168c168,170 < - ld_impl_linux-64=2.35=h769bd43_9 < - lerc=2.2=he1b5a44_0 < - libaec=1.0.4=he1b5a44_1 --- > - ld_impl_linux-64=2.35.1=hea4e1c9_1 > - lerc=2.2=h9c3ff4c_0 > - libaec=1.0.4=h9c3ff4c_1 171c173 < - libcdms=3.1.2=h054cd8a_113 --- > - libcdms=3.1.2=h981a4fd_113 176,177c178,179 < - libdrs=3.1.2=hc2e2db3_113 < - libdrs_f=3.1.2=hae7e664_111 --- > - libdrs=3.1.2=h7918d09_113 > - libdrs_f=3.1.2=h5026c31_111 184,186c186,188 < - libgfortran-ng=7.5.0=hae1eefd_17 < - libgfortran4=7.5.0=hae1eefd_17 < - libglib=2.66.2=hbe7bbb4_0 --- > - libgfortran-ng=9.3.0=he4bcb1c_17 > - libgfortran5=9.3.0=he4bcb1c_17 > - libglib=2.66.3=hbe7bbb4_0 193c195 < - libnetcdf=4.7.4=nompi_hefab0ff_106 --- > - libnetcdf=4.7.4=nompi_h56d31a8_107 195c197 < - libopenblas=0.3.10=pthreads_hb3c22a3_5 --- > - libopenblas=0.3.10=pthreads_h4812303_5 197,199c199,201 < - libpq=12.3=h5513abc_2 < - libprotobuf=3.13.0.1=h8b12597_0 < - libsodium=1.0.18=h516909a_1 --- > - libpq=12.3=h255efa7_3 > - libprotobuf=3.14.0=h780b84a_0 > - libsodium=1.0.18=h36c2ea0_1 206c208 < - libuuid=2.32.1=h14c3975_1000 --- > - libuuid=2.32.1=h7f98852_1000 213,214c215,216 < - libzopfli=1.0.3=he1b5a44_0 < - llvmlite=0.34.0=py38h4f45e52_2 --- > - libzopfli=1.0.3=h9c3ff4c_0 > - llvmlite=0.35.0=py38h4630a5e_0 216c218 < - lxml=4.6.1=py38hbb43d70_0 --- > - lxml=4.6.2=py38hf1fe3a4_0 218a221 > - mamba_gator=4.1.0=pyhd8ed1ab_1 220c223 < - markdown-it-py=0.5.6=py_0 --- > - markdown-it-py=0.5.8=pyhd8ed1ab_0 222,223c225,226 < - matplotlib=3.3.2=py38h578d9bd_1 < - matplotlib-base=3.3.2=py38h5c7f4ab_1 --- > - matplotlib=3.3.3=py38h578d9bd_0 > - matplotlib-base=3.3.3=py38h5c7f4ab_0 226c229 < - mistune=0.8.4=py38h1e0a361_1002 --- > - mistune=0.8.4=py38h25fe258_1002 228c231 < - msgpack-python=1.0.0=py38hbf85e49_2 --- > - msgpack-python=1.0.1=py38h1fd1430_0 233d235 < - nb_conda_kernels=2.3.0=py38h32f6830_3 239,243c241,245 < - ncurses=6.2=h58526e2_3 < - nest-asyncio=1.4.2=pyhd8ed1ab_0 < - netcdf-fortran=4.5.3=nompi_hfef6a68_101 < - netcdf4=1.5.4=nompi_py38hec8b9af_103 < - nettle=3.4.1=h1bed415_1002 --- > - ncurses=6.2=h58526e2_4 > - nest-asyncio=1.4.3=pyhd8ed1ab_0 > - netcdf-fortran=4.5.3=nompi_h1a0d97b_101 > - netcdf4=1.5.5=nompi_py38h1cdf482_100 > - nettle=3.6=he412f7d_0 245c247 < - nodejs=14.14.0=h914e61d_0 --- > - nodejs=15.2.1=h914e61d_0 248,250c250,252 < - nss=3.58=h27285de_1 < - numba=0.51.2=py38hc5bc63f_0 < - numpy=1.19.4=py38hf0fd68c_1 --- > - nss=3.60=hb5efdd6_0 > - numba=0.52.0=py38h51da96c_0 > - numpy=1.19.4=py38h18fd61f_2 253c255 < - openblas=0.3.10=pthreads_h43bd3aa_5 --- > - openblas=0.3.10=pthreads_h04b7a96_5 255,258c257,261 < - openjpeg=2.3.1=h981e76c_3 < - openssl=1.1.1h=h516909a_0 < - owslib=0.20.0=py_0 < - packaging=20.4=pyh9f0ad1d_0 --- > - openjpeg=2.3.1=hf7af979_3 > - openssl=1.1.1i=h7f98852_0 > - owslib=0.21.0=pyhd8ed1ab_0 > - oyaml=1.0=pyhd8ed1ab_0 > - packaging=20.8=pyhd3deb0d_0 260,261c263,264 < - pandas=1.1.4=py38h0ef3d22_0 < - pandoc=2.11.0.4=hd18ef5c_0 --- > - pandas=1.1.5=py38h51da96c_0 > - pandoc=2.11.2=h36c2ea0_0 263c266 < - panel=0.10.1=pyhd8ed1ab_1 --- > - panel=0.10.2=pyhd8ed1ab_0 272c275 < - pip=20.2.4=py_0 --- > - pip=20.3.1=pyhd8ed1ab_0 274c277 < - plotly=4.12.0=pyh9f0ad1d_0 --- > - plotly=4.14.1=pyhd3deb0d_0 276c279 < - pooch=1.2.0=py_0 --- > - pooch=1.3.0=pyhd8ed1ab_0 279c282 < - postgresql=12.3=h8573dbc_2 --- > - postgresql=12.3=hc2f5b80_3 281c284 < - prometheus_client=0.8.0=pyh9f0ad1d_0 --- > - prometheus_client=0.9.0=pyhd3deb0d_0 284c287 < - protobuf=3.13.0.1=py38h950e882_1 --- > - protobuf=3.14.0=py38h709712a_0 287c290 < - pthread-stubs=0.4=h14c3975_1001 --- > - pthread-stubs=0.4=h36c2ea0_1001 289c292 < - py=1.9.0=pyh9f0ad1d_0 --- > - py=1.10.0=pyhd3deb0d_0 295c298 < - pygments=2.7.2=py_0 --- > - pygments=2.7.3=pyhd8ed1ab_0 297c300 < - pyopenssl=19.1.0=py_1 --- > - pyopenssl=20.0.0=pyhd8ed1ab_0 300,301c303,308 < - pyqt=5.12.3=py38ha8c2ead_4 < - pyrsistent=0.17.3=py38h1e0a361_1 --- > - pyqt=5.12.3=py38h578d9bd_6 > - pyqt-impl=5.12.3=py38h7400c14_6 > - pyqt5-sip=4.19.18=py38h709712a_6 > - pyqtchart=5.12=py38h7400c14_6 > - pyqtwebengine=5.12.1=py38h7400c14_6 > - pyrsistent=0.17.3=py38h25fe258_1 304c311 < - pytest=6.1.2=py38h578d9bd_0 --- > - pytest=6.2.0=py38h578d9bd_0 312c319 < - pywavelets=1.1.1=py38hab2c0dc_3 --- > - pywavelets=1.1.1=py38h5c078b8_3 314c321 < - pyzmq=19.0.2=py38ha71036d_2 --- > - pyzmq=20.0.0=py38h1d1b12f_1 316c323 < - qtconsole=4.7.7=pyh9f0ad1d_0 --- > - qtconsole=5.0.1=pyhd8ed1ab_0 321c328 < - requests=2.24.0=pyh9f0ad1d_0 --- > - requests=2.25.0=pyhd3deb0d_0 324,329c331,337 < - rtree=0.9.4=py38h08f867b_1 < - ruamel.yaml=0.16.12=py38h1e0a361_1 < - ruamel.yaml.clib=0.2.2=py38h1e0a361_1 < - scikit-image=0.17.2=py38h0ef3d22_4 < - scikit-learn=0.23.2=py38h5d63f67_2 < - scipy=1.5.3=py38h828c644_0 --- > - roocs-utils=0.1.5=pyhd3deb0d_1 > - rtree=0.9.4=py38h02d302b_2 > - ruamel.yaml=0.16.12=py38h25fe258_1 > - ruamel.yaml.clib=0.2.2=py38h25fe258_1 > - scikit-image=0.17.2=py38h51da96c_4 > - scikit-learn=0.23.2=py38h5d63f67_3 > - scipy=1.5.3=py38hb2138dd_0 338c346 < - sortedcontainers=2.2.2=pyh9f0ad1d_0 --- > - sortedcontainers=2.3.0=pyhd8ed1ab_0 340,343c348,351 < - sqlalchemy=1.3.20=py38h1e0a361_0 < - sqlite=3.33.0=h4cf870e_1 < - tbb=2020.2=hc9558a2_0 < - tbb-devel=2020.2=hc9558a2_0 --- > - sqlalchemy=1.3.20=py38h497a2fe_0 > - sqlite=3.34.0=h74cdb3f_0 > - tbb=2020.2=h4bd325d_1 > - tbb-devel=2020.2=h4bd325d_1 349,351c357,359 < - tifffile=2020.10.1=py_0 < - tiledb=2.1.2=h47b529c_0 < - tk=8.6.10=hed695b0_1 --- > - tifffile=2020.12.8=pyhd8ed1ab_0 > - tiledb=2.1.3=h17508cd_0 > - tk=8.6.10=h21135ba_1 355c363 < - tqdm=4.51.0=pyh9f0ad1d_0 --- > - tqdm=4.54.1=pyhd8ed1ab_0 359c367 < - tzcode=2020a=h516909a_0 --- > - tzcode=2020d=h7f98852_0 370,371c378,379 < - wheel=0.35.1=pyh9f0ad1d_0 < - widgetsnbextension=3.5.1=py38h32f6830_4 --- > - wheel=0.36.2=pyhd3deb0d_0 > - widgetsnbextension=3.5.1=py38h578d9bd_4 373,374c381,382 < - xarray=0.16.1=py_0 < - xclim=0.21.0=py_0 --- > - xarray=0.16.2=pyhd8ed1ab_0 > - xclim=0.22.0=pyhd8ed1ab_0 376,377c384,385 < - xeus=0.24.4=h67b5559_0 < - xeus-python=0.8.7=py38h8030535_0 --- > - xeus=0.25.3=heb02020_0 > - xeus-python=0.9.4=py38h3a7077d_0 389c397 < - xorg-xproto=7.0.31=h14c3975_1007 --- > - xorg-xproto=7.0.31=h7f98852_1007 392,393c400,401 < - zeromq=4.3.3=he1b5a44_2 < - zfp=0.5.5=he1b5a44_4 --- > - zeromq=4.3.3=h58526e2_3 > - zfp=0.5.5=h9c3ff4c_4 404c412 < - handcalcs==1.1.3 --- > - handcalcs==1.2.1 409c417 < - mpld3==0.5.1 --- > - mpld3==0.5.2 411c419 < - nbresuse==0.3.6 --- > - nbresuse==0.4.0 413,415d420 < - pyqt5-sip==4.19.18 < - pyqtchart==5.12 < - pyqtwebengine==5.12.1 ``` Full new `conda env export`: [201214-conda-env-export.yml.txt](https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/files/5706861/201214-conda-env-export.yml.txt) ``` name: birdy channels: - cdat - conda-forge - defaults dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=1_gnu - affine=2.3.0=py_0 - alembic=1.4.3=pyh9f0ad1d_0 - ansi2html=1.6.0=py38h578d9bd_0 - appdirs=1.4.4=pyh9f0ad1d_0 - argon2-cffi=20.1.0=py38h25fe258_2 - async_generator=1.10=py_0 - attrs=20.3.0=pyhd3deb0d_0 - backcall=0.2.0=pyh9f0ad1d_0 - backports=1.0=py_2 - backports.functools_lru_cache=1.6.1=py_0 - beautifulsoup4=4.9.3=pyhb0f4dca_0 - bleach=3.2.1=pyh9f0ad1d_0 - blinker=1.4=py_1 - blosc=1.20.1=h9c3ff4c_0 - bokeh=2.2.3=py38h578d9bd_0 - boltons=20.2.1=pyh9f0ad1d_0 - boost-cpp=1.70.0=h7b93d67_3 - bottleneck=1.3.2=py38hab2c0dc_2 - branca=0.3.1=py_0 - brotli=1.0.9=he1b5a44_3 - brotli-python=1.0.9=py38h348cfbe_3 - brotlipy=0.7.0=py38h8df0ef7_1001 - brunsli=0.1=h9c3ff4c_0 - bzip2=1.0.8=h7f98852_4 - c-ares=1.17.1=h36c2ea0_0 - ca-certificates=2020.12.5=ha878542_0 - cached-property=1.5.1=py_0 - cairo=1.16.0=h9f066cc_1006 - cartopy=0.18.0=py38h9b6dd9e_6 - cdat_info=8.2.1=pyh9f0ad1d_1 - cdtime=3.1.4=py38h8b240cc_2 - cdutil=8.2.1=pyh9f0ad1d_0 - certifi=2020.12.5=py38h578d9bd_0 - certipy=0.1.3=py_0 - cf_xarray=0.3.1=pyhd3deb0d_0 - cffi=1.14.4=py38ha312104_0 - cfgrib=0.9.8.5=pyhd8ed1ab_0 - cfitsio=3.470=hb418390_7 - cftime=1.3.0=py38h0b5ebd8_0 - chardet=3.0.4=py38h924ce5b_1008 - charls=2.1.0=h9c3ff4c_2 - click=7.1.2=pyh9f0ad1d_0 - click-plugins=1.1.1=py_0 - cligj=0.7.1=pyhd8ed1ab_0 - clisops=0.4.0=pyhd3deb0d_0 - cloudpickle=1.6.0=py_0 - colorama=0.4.4=pyh9f0ad1d_0 - colorcet=2.0.1=py_0 - configurable-http-proxy=1.3.0=0 - coverage=5.3=py38h497a2fe_1 - cryptography=3.3.1=py38h2b97feb_0 - curl=7.71.1=he644dc0_8 - cycler=0.10.0=py_2 - cytoolz=0.11.0=py38h25fe258_1 - dash=1.18.1=pyhd8ed1ab_0 - dash-core-components=1.14.1=pyhd8ed1ab_0 - dash-html-components=1.1.1=pyh9f0ad1d_0 - dash-renderer=1.8.3=pyhd8ed1ab_0 - dash-table=4.11.1=pyhd8ed1ab_0 - dask=2020.12.0=pyhd8ed1ab_0 - dask-core=2020.12.0=pyhd8ed1ab_0 - datashader=0.11.1=pyh9f0ad1d_0 - datashape=0.5.4=py_1 - dbus=1.13.6=hfdff14a_1 - debugpy=1.2.1=py38h709712a_0 - decorator=4.4.2=py_0 - defusedxml=0.6.0=py_0 - descartes=1.1.0=py_4 - distarray=2.12.2=py_1 - distributed=2020.12.0=py38h578d9bd_0 - docopt=0.6.2=py_1 - dv3d=8.2.1=pyh9f0ad1d_0 - eccodes=2.19.1=ha0e6eb6_1 - entrypoints=0.3=pyhd8ed1ab_1003 - esgf-compute-api=2.3.7=0_h1234567_ - esmf=8.0.1=nompi_he31a43a_2 - esmpy=8.0.1=nompi_py38h5410a82_2 - expat=2.2.9=he1b5a44_2 - ffmpeg=4.2.3=h167e202_0 - fiona=1.8.18=py38h37fbd03_0 - flask=1.1.2=pyh9f0ad1d_0 - flask-compress=1.8.0=pyhd8ed1ab_0 - fontconfig=2.13.1=h7e3eb15_1002 - freetype=2.10.4=h7ca028e_0 - freexl=1.0.5=h516909a_1002 - fsspec=0.8.4=py_0 - future=0.18.2=py38h578d9bd_2 - g2clib=1.6.0=hf3f1b0b_9 - gdal=3.1.4=py38h9edfc58_0 - genutil=8.2.1=py38hab2c0dc_1 - geojson=2.5.0=py_0 - geopandas=0.8.1=py_0 - geos=3.8.1=he1b5a44_0 - geotiff=1.6.0=h5d11630_3 - geoviews=1.8.2=py_0 - geoviews-core=1.8.2=py_0 - gettext=0.19.8.1=hf34092f_1004 - ghostscript=9.53.3=h58526e2_2 - giflib=5.2.1=h36c2ea0_2 - gitdb=4.0.5=py_0 - gitpython=3.1.11=py_0 - glib=2.66.3=h58526e2_0 - gmp=6.2.1=h58526e2_0 - gnutls=3.6.13=h85f3911_1 - gst-plugins-base=1.14.5=h0935bb2_2 - gstreamer=1.14.5=h36ae1b5_2 - h5netcdf=0.8.1=py_0 - h5py=3.1.0=nompi_py38hafa665b_100 - hdf4=4.2.13=h10796ff_1004 - hdf5=1.10.6=nompi_h6a2412b_1112 - heapdict=1.0.1=py_0 - holoviews=1.14.0=pyhd3deb0d_0 - hvplot=0.6.0=pyh9f0ad1d_0 - icu=67.1=he1b5a44_0 - idna=2.10=pyh9f0ad1d_0 - imagecodecs=2020.5.30=py38h67f9a0b_4 - imageio=2.9.0=py_0 - importlib-metadata=3.3.0=py38h578d9bd_2 - importlib_metadata=3.3.0=hd8ed1ab_2 - importlib_resources=3.3.0=py38h578d9bd_0 - iniconfig=1.1.1=pyh9f0ad1d_0 - ipykernel=5.4.2=py38h81c977d_0 - ipyleaflet=0.13.3=pyh9f0ad1d_0 - ipython=7.19.0=py38h81c977d_0 - ipython_genutils=0.2.0=py_1 - ipywidgets=7.5.1=pyh9f0ad1d_1 - itsdangerous=1.1.0=py_0 - jasper=1.900.1=h07fcdf6_1006 - jedi=0.17.2=py38h578d9bd_1 - jinja2=2.11.2=pyh9f0ad1d_0 - joblib=1.0.0=pyhd8ed1ab_0 - jpeg=9d=h36c2ea0_0 - json-c=0.13.1=hbfbb72e_1002 - json5=0.9.5=pyh9f0ad1d_0 - jsoncpp=1.8.4=hc9558a2_1002 - jsonschema=3.2.0=py_2 - jupyter-archive=0.7.0=py_0 - jupyter-dash=0.3.1=pyhd8ed1ab_0 - jupyter_bokeh=2.0.4=pyhd8ed1ab_0 - jupyter_client=6.1.7=py_0 - jupyter_conda=4.1.0=hd8ed1ab_1 - jupyter_console=6.2.0=py_0 - jupyter_core=4.7.0=py38h578d9bd_0 - jupyter_server=1.0.10=py38h578d9bd_0 - jupyter_telemetry=0.1.0=pyhd8ed1ab_1 - jupyterhub=1.3.0=py38h578d9bd_0 - jupyterhub-base=1.3.0=py38h578d9bd_0 - jupyterlab=2.2.9=py_0 - jupyterlab-git=0.23.2=pyhd8ed1ab_0 - jupyterlab_pygments=0.1.2=pyh9f0ad1d_0 - jupyterlab_server=1.2.0=py_0 - jupytext=1.7.1=pyhd3deb0d_0 - jxrlib=1.1=h7f98852_2 - kealib=1.4.14=h0042707_0 - kiwisolver=1.3.1=py38h82cb98a_0 - krb5=1.17.2=h926e7f8_0 - lame=3.100=h14c3975_1001 - lazy-object-proxy=1.5.2=py38h497a2fe_0 - lcms2=2.11=hcbb858e_1 - ld_impl_linux-64=2.35.1=hea4e1c9_1 - lerc=2.2=h9c3ff4c_0 - libaec=1.0.4=h9c3ff4c_1 - libblas=3.8.0=17_openblas - libcblas=3.8.0=17_openblas - libcdms=3.1.2=h981a4fd_113 - libcf=1.0.3=py38h2f41aa0_109 - libclang=10.0.1=default_hde54327_1 - libcurl=7.71.1=hcdd3856_8 - libdap4=3.20.6=h1d1bd15_1 - libdrs=3.1.2=h7918d09_113 - libdrs_f=3.1.2=h5026c31_111 - libedit=3.1.20191231=he28a2e2_2 - libev=4.33=h516909a_1 - libevent=2.1.10=hcdb4288_3 - libffi=3.2.1=he1b5a44_1007 - libgcc-ng=9.3.0=h5dbcf3e_17 - libgdal=3.1.4=h670eac6_0 - libgfortran-ng=9.3.0=he4bcb1c_17 - libgfortran5=9.3.0=he4bcb1c_17 - libglib=2.66.3=hbe7bbb4_0 - libgomp=9.3.0=h5dbcf3e_17 - libiconv=1.16=h516909a_0 - libidn2=2.3.0=h516909a_0 - libkml=1.3.0=h4fcabce_1010 - liblapack=3.8.0=17_openblas - libllvm10=10.0.1=he513fc3_3 - libnetcdf=4.7.4=nompi_h56d31a8_107 - libnghttp2=1.41.0=h8cfc5f6_2 - libopenblas=0.3.10=pthreads_h4812303_5 - libpng=1.6.37=h21135ba_2 - libpq=12.3=h255efa7_3 - libprotobuf=3.14.0=h780b84a_0 - libsodium=1.0.18=h36c2ea0_1 - libspatialindex=1.9.3=he1b5a44_3 - libspatialite=5.0.0=h4dde289_0 - libssh2=1.9.0=hab1572f_5 - libstdcxx-ng=9.3.0=h2ae2ef3_17 - libtiff=4.1.0=h4f3a223_6 - libunistring=0.9.10=h14c3975_0 - libuuid=2.32.1=h7f98852_1000 - libuv=1.40.0=hd18ef5c_0 - libwebp-base=1.1.0=h36c2ea0_3 - libxcb=1.13=h14c3975_1002 - libxkbcommon=0.10.0=he1b5a44_0 - libxml2=2.9.10=h68273f3_2 - libxslt=1.1.33=h572872d_1 - libzopfli=1.0.3=h9c3ff4c_0 - llvmlite=0.35.0=py38h4630a5e_0 - locket=0.2.0=py_2 - lxml=4.6.2=py38hf1fe3a4_0 - lz4-c=1.9.2=he1b5a44_3 - mako=1.1.3=pyh9f0ad1d_0 - mamba_gator=4.1.0=pyhd8ed1ab_1 - markdown=3.3.3=pyh9f0ad1d_0 - markdown-it-py=0.5.8=pyhd8ed1ab_0 - markupsafe=1.1.1=py38h8df0ef7_2 - matplotlib=3.3.3=py38h578d9bd_0 - matplotlib-base=3.3.3=py38h5c7f4ab_0 - memory_profiler=0.58.0=py_0 - mesalib=18.3.1=h590aaf7_0 - mistune=0.8.4=py38h25fe258_1002 - more-itertools=8.6.0=pyhd8ed1ab_0 - msgpack-python=1.0.1=py38h1fd1430_0 - multipledispatch=0.6.0=py_0 - munch=2.5.0=py_0 - mysql-common=8.0.21=2 - mysql-libs=8.0.21=hf3661c5_2 - nbclient=0.5.1=py_0 - nbdime=2.1.0=py_0 - nbformat=5.0.8=py_0 - nbval=0.9.6=pyh9f0ad1d_0 - nc-time-axis=1.2.0=py_1 - ncurses=6.2=h58526e2_4 - nest-asyncio=1.4.3=pyhd8ed1ab_0 - netcdf-fortran=4.5.3=nompi_h1a0d97b_101 - netcdf4=1.5.5=nompi_py38h1cdf482_100 - nettle=3.6=he412f7d_0 - networkx=2.5=py_0 - nodejs=15.2.1=h914e61d_0 - notebook=6.1.5=py38h578d9bd_0 - nspr=4.29=he1b5a44_1 - nss=3.60=hb5efdd6_0 - numba=0.52.0=py38h51da96c_0 - numpy=1.19.4=py38h18fd61f_2 - oauthlib=3.0.1=py_0 - olefile=0.46=pyh9f0ad1d_1 - openblas=0.3.10=pthreads_h04b7a96_5 - openh264=2.1.1=h8b12597_0 - openjpeg=2.3.1=hf7af979_3 - openssl=1.1.1i=h7f98852_0 - owslib=0.21.0=pyhd8ed1ab_0 - oyaml=1.0=pyhd8ed1ab_0 - packaging=20.8=pyhd3deb0d_0 - pamela=1.0.0=py_0 - pandas=1.1.5=py38h51da96c_0 - pandoc=2.11.2=h36c2ea0_0 - pandocfilters=1.4.2=py_1 - panel=0.10.2=pyhd8ed1ab_0 - param=1.10.0=py_0 - parso=0.7.1=pyh9f0ad1d_0 - partd=1.1.0=py_0 - pcre=8.44=he1b5a44_0 - pexpect=4.8.0=pyh9f0ad1d_2 - pickleshare=0.7.5=py_1003 - pillow=8.0.1=py38h70fbd49_0 - pint=0.16.1=py_0 - pip=20.3.1=pyhd8ed1ab_0 - pixman=0.40.0=h36c2ea0_0 - plotly=4.14.1=pyhd3deb0d_0 - pluggy=0.13.1=py38h924ce5b_3 - pooch=1.3.0=pyhd8ed1ab_0 - poppler=0.89.0=h4190859_1 - poppler-data=0.4.10=0 - postgresql=12.3=hc2f5b80_3 - proj=7.1.1=h966b41f_3 - prometheus_client=0.9.0=pyhd3deb0d_0 - prompt-toolkit=3.0.8=pyha770c72_0 - prompt_toolkit=3.0.8=hd8ed1ab_0 - protobuf=3.14.0=py38h709712a_0 - pscript=0.7.4=pyh9f0ad1d_0 - psutil=5.7.3=py38h8df0ef7_0 - pthread-stubs=0.4=h36c2ea0_1001 - ptyprocess=0.6.0=py_1001 - py=1.10.0=pyhd3deb0d_0 - pycparser=2.20=pyh9f0ad1d_2 - pyct=0.4.6=py_0 - pyct-core=0.4.6=py_0 - pycurl=7.43.0.6=py38hd8bd627_0 - pydap=3.2.2=pyh9f0ad1d_1001 - pygments=2.7.3=pyhd8ed1ab_0 - pyjwt=1.7.1=py_0 - pyopenssl=20.0.0=pyhd8ed1ab_0 - pyparsing=2.4.7=pyh9f0ad1d_0 - pyproj=2.6.1.post1=py38h56787f0_3 - pyqt=5.12.3=py38h578d9bd_6 - pyqt-impl=5.12.3=py38h7400c14_6 - pyqt5-sip=4.19.18=py38h709712a_6 - pyqtchart=5.12=py38h7400c14_6 - pyqtwebengine=5.12.1=py38h7400c14_6 - pyrsistent=0.17.3=py38h25fe258_1 - pyshp=2.1.2=pyh9f0ad1d_0 - pysocks=1.7.1=py38h924ce5b_2 - pytest=6.2.0=py38h578d9bd_0 - python=3.8.6=h852b56e_0_cpython - python-dateutil=2.8.1=py_0 - python-editor=1.0.4=py_0 - python-json-logger=2.0.1=pyh9f0ad1d_0 - python_abi=3.8=1_cp38 - pytz=2020.4=pyhd8ed1ab_0 - pyviz_comms=0.7.6=pyh9f0ad1d_0 - pywavelets=1.1.1=py38h5c078b8_3 - pyyaml=5.3.1=py38h8df0ef7_1 - pyzmq=20.0.0=py38h1d1b12f_1 - qt=5.12.9=h1f2b2cb_0 - qtconsole=5.0.1=pyhd8ed1ab_0 - qtpy=1.9.0=py_0 - rasterio=1.1.8=py38h37fbd03_0 - readline=8.0=he28a2e2_2 - regionmask=0.6.1=py_1 - requests=2.25.0=pyhd3deb0d_0 - retrying=1.3.3=py_2 - rioxarray=0.1.1=pyhd8ed1ab_0 - roocs-utils=0.1.5=pyhd3deb0d_1 - rtree=0.9.4=py38h02d302b_2 - ruamel.yaml=0.16.12=py38h25fe258_1 - ruamel.yaml.clib=0.2.2=py38h25fe258_1 - scikit-image=0.17.2=py38h51da96c_4 - scikit-learn=0.23.2=py38h5d63f67_3 - scipy=1.5.3=py38hb2138dd_0 - send2trash=1.5.0=py_0 - setuptools=49.6.0=py38h924ce5b_2 - shapely=1.7.1=py38hc7361b7_1 - siphon=0.8.0=py38h32f6830_1004 - six=1.15.0=pyh9f0ad1d_0 - smmap=3.0.4=pyh9f0ad1d_0 - snappy=1.1.8=he1b5a44_3 - snuggs=1.4.7=py_0 - sortedcontainers=2.3.0=pyhd8ed1ab_0 - soupsieve=2.0.1=py_1 - sqlalchemy=1.3.20=py38h497a2fe_0 - sqlite=3.34.0=h74cdb3f_0 - tbb=2020.2=h4bd325d_1 - tbb-devel=2020.2=h4bd325d_1 - tblib=1.6.0=py_0 - terminado=0.9.1=py38h32f6830_1 - testpath=0.4.4=py_0 - threadpoolctl=2.1.0=pyh5ca1d4c_0 - threddsclient=0.4.2=py_0 - tifffile=2020.12.8=pyhd8ed1ab_0 - tiledb=2.1.3=h17508cd_0 - tk=8.6.10=h21135ba_1 - toml=0.10.2=pyhd8ed1ab_0 - toolz=0.11.1=py_0 - tornado=6.1=py38h25fe258_0 - tqdm=4.54.1=pyhd8ed1ab_0 - traitlets=5.0.5=py_0 - traittypes=0.2.1=pyh9f0ad1d_2 - typing_extensions=3.7.4.3=py_0 - tzcode=2020d=h7f98852_0 - udunits2=2.2.27.6=h4e0c4b3_1001 - urllib3=1.25.11=py_0 - vcs=8.2.1=pyh9f0ad1d_0 - voila=0.2.4=py_0 - vtk-cdat=8.2.0.8.2.1=py38_mesalibh0be9106_0 - wcwidth=0.2.5=pyh9f0ad1d_2 - webencodings=0.5.1=py_1 - webob=1.8.6=py_0 - werkzeug=1.0.1=pyh9f0ad1d_0 - wget=1.20.1=h22169c7_0 - wheel=0.36.2=pyhd3deb0d_0 - widgetsnbextension=3.5.1=py38h578d9bd_4 - x264=1!152.20180806=h14c3975_0 - xarray=0.16.2=pyhd8ed1ab_0 - xclim=0.22.0=pyhd8ed1ab_0 - xerces-c=3.2.3=hfe33f54_1 - xeus=0.25.3=heb02020_0 - xeus-python=0.9.4=py38h3a7077d_0 - xorg-kbproto=1.0.7=h14c3975_1002 - xorg-libice=1.0.10=h516909a_0 - xorg-libsm=1.2.3=h84519dc_1000 - xorg-libx11=1.6.12=h516909a_0 - xorg-libxau=1.0.9=h14c3975_0 - xorg-libxdmcp=1.1.3=h516909a_0 - xorg-libxext=1.3.4=h516909a_0 - xorg-libxrender=0.9.10=h516909a_1002 - xorg-libxt=1.2.0=h516909a_0 - xorg-renderproto=0.11.1=h14c3975_1002 - xorg-xextproto=7.3.0=h14c3975_1002 - xorg-xproto=7.0.31=h7f98852_1007 - xz=5.2.5=h516909a_1 - yaml=0.2.5=h516909a_0 - zeromq=4.3.3=h58526e2_3 - zfp=0.5.5=h9c3ff4c_4 - zict=2.0.0=py_0 - zipp=3.4.0=py_0 - zlib=1.2.11=h516909a_1010 - zstd=1.4.5=h6597ccf_2 - pip: - astunparse==1.6.3 - birdhouse-birdy==0.6.9 - cdms2==3.1.5 - colour==0.1.5 - funcsigs==1.0.2 - handcalcs==1.2.1 - innerscope==0.2.0 - ipython-blocking==0.2.1 - jupyter==1.0.0 - jupyternotify==0.1.15 - mpld3==0.5.2 - nbconvert==5.6.1 - nbresuse==0.4.0 - pixiedust==1.1.18 - pytest-tornasync==0.6.0.post2 - regrid2==3.1.5 - requests-magpie==0.1.1 - wrapt==1.12.1 prefix: /opt/conda/envs/birdy ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added the only missing
cfgribpackage to our PAVICS Jupyter image and used our PAVICS Jupyter image as the runtime environment so other users can also perform the same task as Travis on our Jupyter if needed.The Jupyter image is always present on Boreas so re-using it means no extra space required for the runtime of the script (except the extra 159MB for
cfgrib).@tlogan2000 Note that I had to repeat the the 3 paths (inpath, outpath, threddspath) to the wrapper script as well so if you ever need to modify those paths, you should modify in both scripts. I've left comments in the code as reminder that there are duplicates.
@huard For your Intake catalog project, you can write a script similar to
convert_grib2_to_nc.pyand I'll automate its deployment and execution the same way.Automated deployment and update is handled by PR bird-house/birdhouse-deploy-ouranos#2.
Requires PR Ouranosinc/PAVICS-e2e-workflow-tests#56 for the runtime environment.
Part of issue #19 and issue Ouranosinc/raven#313.