Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial land DA cycling #1351

Merged
merged 45 commits into from
Apr 24, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
fb2f8fb
Add initial land DA cycling.
jiaruidong2017 Mar 1, 2023
3e6a3a5
Fix shellcheck error.
jiaruidong2017 Mar 1, 2023
47e04e9
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 2, 2023
da71eba
Address the reviwer's comments.
jiaruidong2017 Mar 2, 2023
4fafb58
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 2, 2023
67cfc23
Remove "COMPONENT" and replaced with hard-coded "land".
jiaruidong2017 Mar 3, 2023
99c6e8d
Merge branch 'landda_pr1' of https://github.com/jiaruidong2017/global…
jiaruidong2017 Mar 3, 2023
687508d
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 3, 2023
70c0b79
Revert back to use atmos/ for land component.
jiaruidong2017 Mar 3, 2023
a89b280
Address the reviewer's questions.
jiaruidong2017 Mar 4, 2023
02108f5
Address the reviewer's comments.
jiaruidong2017 Mar 6, 2023
b6e454a
Merge branch 'develop' into landda_pr1
jiaruidong2017 Mar 10, 2023
a951ffa
Handle using different observations in different cycles.
jiaruidong2017 Mar 10, 2023
de063f3
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 10, 2023
138bb8f
Address the reviewer's comments.
jiaruidong2017 Mar 11, 2023
c10af3d
Fix shellcheck warning.
jiaruidong2017 Mar 11, 2023
ffa2e1c
Fix the shellcheck errors.
jiaruidong2017 Mar 11, 2023
ca947c7
Revert back.
jiaruidong2017 Mar 12, 2023
754ef8b
Try.
jiaruidong2017 Mar 12, 2023
40da629
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 12, 2023
52f8188
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 14, 2023
4d20e8b
Adress the reviewer's comments.
jiaruidong2017 Mar 14, 2023
03480d6
Merge branch 'landda_pr1' of https://github.com/jiaruidong2017/global…
jiaruidong2017 Mar 14, 2023
949b4c9
bugfix
jiaruidong2017 Mar 14, 2023
587fb86
bugfix2.
jiaruidong2017 Mar 14, 2023
f18d7dc
bugfix try.
jiaruidong2017 Mar 14, 2023
25f832c
Bugfix try.
jiaruidong2017 Mar 14, 2023
186a203
Combine landanlinit and landanlfinal into a single block
jiaruidong2017 Mar 14, 2023
2ab93a0
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 15, 2023
049593e
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 15, 2023
5524ff8
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 18, 2023
c6457d8
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Mar 28, 2023
e5fb548
Modify the scripts to align with the aerosol analysis task.
jiaruidong2017 Mar 29, 2023
48cea6e
Merge branch 'develop' into landda_pr1
jiaruidong2017 Apr 19, 2023
973e2f5
Remove ATML app and add do_JEDILANDDA option.
jiaruidong2017 Apr 19, 2023
fcaf0ea
Made a minor correction.
jiaruidong2017 Apr 19, 2023
66833f0
Correct the shell norm errors
jiaruidong2017 Apr 19, 2023
deb8080
Correct shell check warning.
jiaruidong2017 Apr 20, 2023
470d8c5
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Apr 20, 2023
8d4cfac
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Apr 20, 2023
cfab1d3
Made changes by comparing the Russ's PR for the ATM DA.
jiaruidong2017 Apr 21, 2023
af29e14
Merge branch 'NOAA-EMC:develop' into landda_pr1
jiaruidong2017 Apr 21, 2023
3b8974b
Address the reviwer's comments and suggestions.
jiaruidong2017 Apr 21, 2023
13a567a
Remove the '_gfs' resource.
jiaruidong2017 Apr 23, 2023
0be71f7
Merge branch 'develop' into landda_pr1
jiaruidong2017 Apr 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion env/HERA.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanalrun aeroanlrun landanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -76,6 +76,16 @@ elif [[ "${step}" = "aeroanlrun" ]]; then
[[ ${NTHREADS_AEROANL} -gt ${nth_max} ]] && export NTHREADS_AEROANL=${nth_max}
export APRUN_AEROANL="${launcher} -n ${npe_aeroanlrun}"

elif [[ "${step}" = "landanlrun" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know @RussTreadon-NOAA removed these lines from the atmanlrun. Please check if the land jobs will execute via MPMD as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed these lines, and confirmed that there were not any effects for the landanlrun.


nth_max=$((npe_node_max / npe_node_landanlrun))

export NTHREADS_LANDANL=${nth_landanlrun:-${nth_max}}
[[ ${NTHREADS_LANDANL} -gt ${nth_max} ]] && export NTHREADS_LANDANL=${nth_max}
export APRUN_LANDANL="${launcher} -n ${npe_landanlrun}"

elif [[ "${step}" = "ocnanalbmat" ]]; then

export APRUNCFP="${launcher} -n \$ncmd --multi-prog"
Expand Down
12 changes: 11 additions & 1 deletion env/ORION.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if [[ $# -ne 1 ]]; then

echo "Must specify an input argument to set runtime environment variables!"
echo "argument can be any one of the following:"
echo "atmanlrun atmensanalrun aeroanlrun"
echo "atmanlrun atmensanalrun aeroanlrun landanlrun"
echo "anal sfcanl fcst post vrfy metp"
echo "eobs eupd ecen efcs epos"
echo "postsnd awips gempak"
Expand Down Expand Up @@ -78,6 +78,16 @@ elif [[ "${step}" = "aeroanlrun" ]]; then
[[ ${NTHREADS_AEROANL} -gt ${nth_max} ]] && export NTHREADS_AEROANL=${nth_max}
export APRUN_AEROANL="${launcher} -n ${npe_aeroanlrun}"

elif [[ "${step}" = "landanlrun" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same solution as above for Hera.


nth_max=$((npe_node_max / npe_node_landanlrun))

export NTHREADS_LANDANL=${nth_landanlrun:-${nth_max}}
[[ ${NTHREADS_LANDANL} -gt ${nth_max} ]] && export NTHREADS_LANDANL=${nth_max}
export APRUN_LANDANL="${launcher} -n ${npe_landanlrun}"

elif [[ "${step}" = "ocnanalbmat" ]]; then

export APRUNCFP="${launcher} -n \$ncmd ${mpmd_opt}"
Expand Down
50 changes: 50 additions & 0 deletions jobs/JGLOBAL_LAND_ANALYSIS_FINALIZE
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"
export WIPE_DATA="NO"
export DATA=${DATA:-${DATAROOT}/${RUN}landanl_${cyc}}
source "${HOMEgfs}/ush/jjob_header.sh" -e "landanlfinal" -c "base landanl landanlfinal"

##############################################
# Set variables used in the script
##############################################
GDATE=$(date +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
GDUMP="gdas"

##############################################
# Begin JOB SPECIFIC work
##############################################

export COMOUT=${COMOUT:-${ROTDIR}/${RUN}.${PDY}/${cyc}/atmos}
mkdir -p "${COMOUT}"

# COMIN_GES and COMIN_GES_ENS are used in script
export COMIN_GES="${ROTDIR}/${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"
export COMIN_GES_ENS="${ROTDIR}/enkf${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"

###############################################################
# Run relevant script

EXSCRIPT=${GDASLANDFINALPY:-${HOMEgfs}/scripts/exglobal_land_analysis_finalize.py}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

##############################################
# End JOB SPECIFIC work
##############################################

##############################################
# Final processing
##############################################
if [[ -e "${pgmout}" ]] ; then
cat "${pgmout}"
fi

##########################################
# Remove the Temporary working directory
##########################################
cd "${DATAROOT}" || exit 1
[[ ${KEEPDATA} = "NO" ]] && rm -rf "${DATA}"

exit 0
43 changes: 43 additions & 0 deletions jobs/JGLOBAL_LAND_ANALYSIS_INITIALIZE
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"
export DATA=${DATA:-${DATAROOT}/${RUN}landanl_${cyc}}
source "${HOMEgfs}/ush/jjob_header.sh" -e "landanlinit" -c "base landanl landanlinit"

##############################################
# Set variables used in the script
##############################################
GDATE=$(date +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
GDUMP="gdas"

##############################################
# Begin JOB SPECIFIC work
##############################################

export COMOUT=${COMOUT:-${ROTDIR}/${RUN}.${PDY}/${cyc}/atmos}
mkdir -p "${COMOUT}"

# COMIN_GES and COMIN_GES_ENS are used in script
export COMIN_GES="${ROTDIR}/${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"
export COMIN_GES_ENS="${ROTDIR}/enkf${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"

###############################################################
# Run relevant script

EXSCRIPT=${GDASLANDINITPY:-${HOMEgfs}/scripts/exglobal_land_analysis_initialize.py}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

##############################################
# End JOB SPECIFIC work
##############################################

##############################################
# Final processing
##############################################
if [[ -e "${pgmout}" ]] ; then
cat "${pgmout}"
fi

exit 0
44 changes: 44 additions & 0 deletions jobs/JGLOBAL_LAND_ANALYSIS_RUN
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"
export WIPE_DATA="NO"
export DATA=${DATA:-${DATAROOT}/${RUN}landanl_${cyc}}
source "${HOMEgfs}/ush/jjob_header.sh" -e "landanlrun" -c "base landanl landanlrun"

##############################################
# Set variables used in the script
##############################################
GDATE=$(date +%Y%m%d%H -d "${PDY} ${cyc} - ${assim_freq} hours")
GDUMP="gdas"

##############################################
# Begin JOB SPECIFIC work
##############################################

export COMOUT=${COMOUT:-${ROTDIR}/${RUN}.${PDY}/${cyc}/atmos}
mkdir -p "${COMOUT}"

# COMIN_GES and COMIN_GES_ENS are used in script
export COMIN_GES="${ROTDIR}/${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"
export COMIN_GES_ENS="${ROTDIR}/enkf${GDUMP}.${GDATE:0:8}/${GDATE:8:2}/atmos"

###############################################################
# Run relevant script

EXSCRIPT=${GDASLANDRUNSH:-${HOMEgfs}/scripts/exglobal_land_analysis_run.sh}
${EXSCRIPT}
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

##############################################
# End JOB SPECIFIC work
##############################################

##############################################
# Final processing
##############################################
if [[ -e "${pgmout}" ]] ; then
cat "${pgmout}"
fi

exit 0
23 changes: 23 additions & 0 deletions jobs/rocoto/landanlfinal.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"

###############################################################
# Source UFSDA workflow modules
. "${HOMEgfs}/ush/load_ufsda_modules.sh"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

export job="landanlfinal"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
pygwPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/pygw/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${pygwPATH}"
export PYTHONPATH
###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_LAND_ANALYSIS_FINALIZE"
status=$?
exit "${status}"
24 changes: 24 additions & 0 deletions jobs/rocoto/landanlinit.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"

###############################################################
# Source UFSDA workflow modules
. "${HOMEgfs}/ush/load_ufsda_modules.sh"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

export job="landanlinit"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
pygwPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/pygw/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${pygwPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_LAND_ANALYSIS_INITIALIZE"
status=$?
exit "${status}"
24 changes: 24 additions & 0 deletions jobs/rocoto/landanlrun.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"

###############################################################
# Source UFSDA workflow modules
. "${HOMEgfs}/ush/load_ufsda_modules.sh"
status=$?
[[ ${status} -ne 0 ]] && exit "${status}"

export job="landanlrun"
export jobid="${job}.$$"

###############################################################
# setup python path for workflow utilities and tasks
pygwPATH="${HOMEgfs}/ush/python:${HOMEgfs}/ush/python/pygw/src"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}${pygwPATH}"
export PYTHONPATH

###############################################################
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_LAND_ANALYSIS_RUN"
status=$?
exit "${status}"
1 change: 1 addition & 0 deletions parm/config/config.base.emc.dyn
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ export imp_physics=@IMP_PHYSICS@
export DO_JEDIVAR="NO"
export DO_JEDIENS="NO"
export DO_JEDIOCNVAR="NO"
export DO_JEDILANDDA="NO"

# Hybrid related
export DOHYBVAR="@DOHYBVAR@"
Expand Down
23 changes: 23 additions & 0 deletions parm/config/config.landanl
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we are doing an -x in the config scripts.

Suggested change
#!/bin/bash -x
#!/usr/bin/env bash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made changes as suggested.


########## config.landanl ##########
# configuration common to all land analysis tasks

echo "BEGIN: config.landanl"

obs_list_name=gdas_land_adpsfc_only.yaml
if [[ "${cyc}" == "18" ]]; then
obs_list_name=gdas_land_prototype.yaml
fi

export OBS_YAML_DIR=${HOMEgfs}/sorc/gdas.cd/parm/land/obs/config/
export OBS_LIST=${HOMEgfs}/sorc/gdas.cd/parm/land/obs/lists/${obs_list_name}
export LANDVARYAML=${HOMEgfs}/sorc/gdas.cd/parm/land/letkfoi/letkfoi.yaml
export FV3JEDI_FIX=${HOMEgfs}/fix/gdas
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to be used? @RussTreadon-NOAA had removed its reference.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should be consistent, I don't think it's needed now as long as the fix directory is linked

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul I checked that @RussTreadon-NOAA still kept this reference in the config.atmanl, and similar for the config.aeroanl. Therefore, we keep this reference here for consistency.


export io_layout_x=@IO_LAYOUT_X@
export io_layout_y=@IO_LAYOUT_Y@

export JEDIVAREXE=${HOMEgfs}/exec/fv3jedi_letkf.x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a "variational executable", if running an LETKF, is it?
Also, I think @RussTreadon-NOAA updated this to JEDIEXE.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the changes as suggested and updated this to JEDIEXE.


echo "END: config.landanl"
10 changes: 10 additions & 0 deletions parm/config/config.landanlfinal
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#!/bin/bash -x
#!/usr/bin/env bash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made changes as suggested.


########## config.landanlfinal ##########
# Post Land Analysis specific

echo "BEGIN: config.landanlfinal"

# Get task specific resources
. "${EXPDIR}/config.resources" landanlfinal
echo "END: config.landanlfinal"
10 changes: 10 additions & 0 deletions parm/config/config.landanlinit
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#!/bin/bash -x
#!/usr/bin/env bash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made changes as suggested.


########## config.landanlinit ##########
# Pre Land Analysis specific

echo "BEGIN: config.landanlinit"

# Get task specific resources
. "${EXPDIR}/config.resources" landanlinit
echo "END: config.landanlinit"
11 changes: 11 additions & 0 deletions parm/config/config.landanlrun
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#!/bin/bash -x
#!/usr/bin/env bash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made changes as suggested.


########## config.landanlrun ##########
# Land Analysis specific

echo "BEGIN: config.landanlrun"

# Get task specific resources
. "${EXPDIR}/config.resources" landanlrun

echo "END: config.landanlrun"
Loading