Skip to content

Commit

Permalink
Refactor UFS-DA ATM ens component to use python g-w (#1373)
Browse files Browse the repository at this point in the history
This PR contains UFS-DA ATM ens changes originally in PR #1354.

Below is a list of changes in this PR
- rename UFS-DA ATM ens jobs atmensanalprep, atmensanalrun, and atmensanalpost as atmensanlinit, atmensanlrun, and atmensanlfinal, respectively
- replace UFS-DA ATM ens shell scripts with python scripts
- rename UFS-DA ATM ens j-jobs consistent with initialize, run, and finalize functions. Update j-jobs to execute python scripts instead of shell scripts
- rename UFS-DA ATM ens rocoto jobs to be consistent with initialize, run, and finalize functions. Update jobs to set python paths and execute renamed j-jobs
- update rocoto workflow generation to new names for UFS-DA ATM ens jobs
- update UFS-DA ATM ens job names in machine dependent env files to new job names
- rename UFS-DA ATM ens configuration files consistent with change in job names
- add python class for UFS-DA ATM ens analysis
- unify JEDIEXE link for UFS-DA Aerosol, ATM, and ENS
- properly set `cycledefs` for `gfsatmanlinit`
- remove unused `FV3JEDI_FIX` from atmanl and atmensanl config

The above changes are part of a larger g-w effort to transition from shell scripts to python. UFS-DA Aerosol was the first GDASApp system to be converted. PR #1372 converted UFS-DA atmospheric variational DA to the python based approach.  This PR converts converts UFS-DA atmospheric local ensemble DA to the python based approach.

Fixes #1313
Depends (in part) on #1370 and #1372 and NOAA-EMC/GDASApp#388
  • Loading branch information
RussTreadon-NOAA authored Apr 21, 2023
1 parent 7db7049 commit 587e469
Show file tree
Hide file tree
Showing 40 changed files with 761 additions and 663 deletions.
2 changes: 1 addition & 1 deletion env/CONTAINER.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down
14 changes: 6 additions & 8 deletions env/HERA.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -56,15 +56,13 @@ elif [[ "${step}" = "atmanlrun" ]]; then
[[ ${NTHREADS_ATMANL} -gt ${nth_max} ]] && export NTHREADS_ATMANL=${nth_max}
export APRUN_ATMANL="${launcher} -n ${npe_atmanlrun}"

elif [[ "${step}" = "atmensanalrun" ]]; then
elif [[ "${step}" = "atmensanlrun" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_atmensanalrun))
nth_max=$((npe_node_max / npe_node_atmensanlrun))

export NTHREADS_ATMENSANAL=${nth_atmensanalrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANAL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANAL=${nth_max}
export APRUN_ATMENSANAL="${launcher} -n ${npe_atmensanalrun}"
export NTHREADS_ATMENSANL=${nth_atmensanlrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANL=${nth_max}
export APRUN_ATMENSANL="${launcher} -n ${npe_atmensanlrun}"

elif [[ "${step}" = "aeroanlrun" ]]; then

Expand Down
16 changes: 6 additions & 10 deletions env/JET.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -55,17 +55,13 @@ elif [[ "${step}" = "atmanlrun" ]]; then
[[ ${NTHREADS_ATMANL} -gt ${nth_max} ]] && export NTHREADS_ATMANL=${nth_max}
export APRUN_ATMANL="${launcher} -n ${npe_atmanlrun}"

elif [[ "${step}" = "atmensanalrun" ]]; then
elif [[ "${step}" = "atmensanlrun" ]]; then

export CFP_MP=${CFP_MP:-"YES"}
export USE_CFP=${USE_CFP:-"YES"}
export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_atmensanalrun))
nth_max=$((npe_node_max / npe_node_atmensanlrun))

export NTHREADS_ATMENSANAL=${nth_atmensanalrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANAL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANAL=${nth_max}
export APRUN_ATMENSANAL="${launcher} -n ${npe_atmensanalrun}"
export NTHREADS_ATMENSANL=${nth_atmensanlrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANL=${nth_max}
export APRUN_ATMENSANL="${launcher} ${npe_atmensanlrun}"

elif [[ "${step}" = "aeroanlrun" ]]; then

Expand Down
16 changes: 6 additions & 10 deletions env/ORION.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -56,17 +56,13 @@ elif [[ "${step}" = "atmanlrun" ]]; then
[[ ${NTHREADS_ATMANL} -gt ${nth_max} ]] && export NTHREADS_ATMANL=${nth_max}
export APRUN_ATMANL="${launcher} -n ${npe_atmanlrun}"

elif [[ "${step}" = "atmensanalrun" ]]; then
elif [[ "${step}" = "atmensanlrun" ]]; then

export CFP_MP=${CFP_MP:-"YES"}
export USE_CFP=${USE_CFP:-"YES"}
export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_atmensanalrun))
nth_max=$((npe_node_max / npe_node_atmensanlrun))

export NTHREADS_ATMENSANAL=${nth_atmensanalrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANAL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANAL=${nth_max}
export APRUN_ATMENSANAL="${launcher} -n ${npe_atmensanalrun}"
export NTHREADS_ATMENSANL=${nth_atmensanlrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANL=${nth_max}
export APRUN_ATMENSANL="${launcher} -n ${npe_atmensanlrun}"

elif [[ "${step}" = "aeroanlrun" ]]; then

Expand Down
16 changes: 6 additions & 10 deletions env/S4.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -54,17 +54,13 @@ elif [[ "${step}" = "atmanlrun" ]]; then
[[ ${NTHREADS_ATMANL} -gt ${nth_max} ]] && export NTHREADS_ATMANL=${nth_max}
export APRUN_ATMANL="${launcher} -n ${npe_atmanlrun}"

elif [[ "${step}" = "atmensanalrun" ]]; then
elif [[ "${step}" = "atmensanlrun" ]]; then

export CFP_MP=${CFP_MP:-"YES"}
export USE_CFP=${USE_CFP:-"YES"}
export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_atmensanalrun))
nth_max=$((npe_node_max / npe_node_atmensanlrun))

export NTHREADS_ATMENSANAL=${nth_atmensanalrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANAL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANAL=${nth_max}
export APRUN_ATMENSANAL="${launcher} -n ${npe_atmensanalrun}"
export NTHREADS_ATMENSANL=${nth_atmensanlrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANL=${nth_max}
export APRUN_ATMENSANL="${launcher} -n ${npe_atmensanlrun}"

elif [[ "${step}" = "aeroanlrun" ]]; then

Expand Down
16 changes: 6 additions & 10 deletions env/WCOSS2.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanlrun aeroanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen esfc efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -43,17 +43,13 @@ elif [[ "${step}" = "atmanlrun" ]]; then
[[ ${NTHREADS_ATMANL} -gt ${nth_max} ]] && export NTHREADS_ATMANL=${nth_max}
export APRUN_ATMANL="${launcher} -n ${npe_atmanlrun}"

elif [[ "${step}" = "atmensanalrun" ]]; then
elif [[ "${step}" = "atmensanlrun" ]]; then

export CFP_MP=${CFP_MP:-"YES"}
export USE_CFP=${USE_CFP:-"YES"}
export APRUNCFP="${launcher} -np \$ncmd ${mpmd_opt}"

nth_max=$((npe_node_max / npe_node_atmensanalrun))
nth_max=$((npe_node_max / npe_node_atmensanlrun))

export NTHREADS_ATMENSANAL=${nth_atmensanalrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANAL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANAL=${nth_max}
export APRUN_ATMENSANAL="${launcher} -n ${npe_atmensanalrun}"
export NTHREADS_ATMENSANL=${nth_atmensanlrun:-${nth_max}}
[[ ${NTHREADS_ATMENSANL} -gt ${nth_max} ]] && export NTHREADS_ATMENSANL=${nth_max}
export APRUN_ATMENSANL="${launcher} -n ${npe_atmensanlrun}"

elif [[ "${step}" = "aeroanlrun" ]]; then

Expand Down
66 changes: 0 additions & 66 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_POST

This file was deleted.

66 changes: 0 additions & 66 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_PREP

This file was deleted.

66 changes: 0 additions & 66 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_RUN

This file was deleted.

Loading

0 comments on commit 587e469

Please sign in to comment.