Add DKRZ as a site to RTW #3837
Conversation
ehogan
added
the
Recipe Test Workflow (RTW)
| * on DKRZ: | ||
|
|
||
| * add the following line to your ``~/.bashrc`` file to ensure the Cylc and | ||
| Rose executables can be found:: | ||
|
|
||
| export PATH=/work/bd0854/metomi/bin:$PATH | ||
|
|
||
| * run the RTW on DKRZ:: | ||
|
|
||
| cd ESMValTool/esmvaltool/utils/recipe_test_workflow | ||
| cylc vip -O dkrz |
There was a problem hiding this comment.
Here are instructions on how to run the RTW on DKRZ. Rendered version of these instructions are at https://esmvaltool--3837.org.readthedocs.build/en/3837/utils/RTW/user_guide/quick_start.html. Note that some of the process tasks are failing, I assume due to development changes since the last release. I will open new issues for those next week 👍
| --wckey = RTW | ||
| --account = bk1088 | ||
| --partition = interactive | ||
| --mem = 15G |
There was a problem hiding this comment.
I have sent an e-mail to DKRZ support to ask why the memory needs to be so high (the jobs fail if the memory requested is less than this with messages like slurmstepd: error: Detected 1 oom_kill event in StepId=<process-id>.batch. Some of the step tasks have been OOM Killed.. Actual usage is (apparently) no more than 4GB 😞
| @@ -0,0 +1,8 @@ | |||
| [command] | |||
| default=singularity-env singularity build ${CONTAINER_PATH} ${DOCKER_SOURCE} | |||
There was a problem hiding this comment.
Singularity works on DKRZ! Yay! 🎉
| @@ -2,7 +2,8 @@ | |||
| [runtime] | |||
| [[root]] | |||
| [[[environment]]] | |||
| # Warning: fragile assumption of location of esmvaltool code, see #3437 | |||
| # Warning: fragile assumption of location of esmvaltool code, | |||
| # issue #3437 contains more details. | |||
There was a problem hiding this comment.
Adding link to issue here: #3437
|
@sloosvel this branch should enable you to run the RTW on DKRZ! 🎉 The reason for the delay was because I was implementing the use of a container (I was using the "clone" method at the workshop in November), and I made a few rookie mistakes 😖 I have made some comments on the PR that I hope will help you with your review 😊 Note that remove_preproc_dir is set to False by default, which causes some of the |
Description
Before you get started
Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.
To help with the number of pull requests: