Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DART input.nml to template in WRF Tutorial #464

Merged
merged 11 commits into from
Apr 11, 2023
Merged

Add DART input.nml to template in WRF Tutorial #464

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2bab64d
Add DART input.nml to template in WRF Tutorial
braczka Mar 13, 2023
aa686ac
Updated obs_seq.final flist for WRF_DART Tutorial
braczka Mar 16, 2023
92ce866
bug-fix: developer test quickbuilds
hkershaw-brown Oct 17, 2022
859bf4d
Remove WRF diagnostics before next integration
braczka Apr 3, 2023
911dc4e
WRF-DART Tutorial documentation edits
braczka Apr 3, 2023
c354a65
Changing EXTRA=/models/template/model_mod.f90 to EXTRA=/models/templa…
mjs2369 Apr 4, 2023
07170e4
Fixing EXTRA var in random_seq as well
mjs2369 Apr 4, 2023
08aba5e
WRF tutorial eliminates DART template input.nml
braczka Apr 4, 2023
63a5748
WRF shell scripts switch to rundir input.nml
braczka Apr 4, 2023
2f6ed97
Merge branch 'devel-tests-quickbuild' into WRF_tutorial
mjs2369 Apr 11, 2023
282c2ca
bump conf.py and update CHANGELOG
mjs2369 Apr 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ individual files.

The changes are now listed with the most recent at the top.

**April 11 2023 :: Bug-fixes for WRF Tutorial and developer test quickbuilds. Tag v10.6.5**

- Fixes the developer_tests quickbuild.sh files
- Fixes the broken link to WRF DART tutorial input.nml.template

**February 22 2023 :: Bug-fix release. Tag v10.6.4**

- Removes unused and uninitialized argument to adaptive_inflate_init.
Expand Down
2 changes: 1 addition & 1 deletion conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
author = 'Data Assimilation Research Section'

# The full version, including alpha/beta/rc tags
release = '10.6.4'
release = '10.6.5'
master_doc = 'README'

# -- General configuration ---------------------------------------------------
Expand Down
5 changes: 3 additions & 2 deletions developer_tests/forward_operators/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
LOCATION="threed_sphere"
MODEL="none"
EXTRA=$DART/models/template/threed_model_mod.f90
dev_test=1
TEST="forward_operators"
LOCATION="threed_sphere"

programs=(
rttov_test
Expand Down
5 changes: 3 additions & 2 deletions developer_tests/io/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
LOCATION="threed_sphere"
MODEL="none"
EXTRA="$DART"/models/template/threed_model_mod.f90
dev_test=1
TEST="io"
LOCATION="threed_sphere"

programs=(
test_cf_conventions
Expand Down
3 changes: 2 additions & 1 deletion developer_tests/obs_sequence/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
MODEL="none"
EXTRA="$DART"/models/template/oned_model_mod.f90
LOCATION="oned"
dev_test=1
TEST="obs_sequence"
Expand Down
6 changes: 3 additions & 3 deletions developer_tests/quad_interpolate/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL=template
LOCATION=threed_sphere
MODEL="none"
EXTRA="$DART"/models/template/threed_model_mod.f90
TEST=quad_interpolate
dev_test=1

LOCATION="threed_sphere"

programs=(
)
Expand Down
5 changes: 3 additions & 2 deletions developer_tests/random_seq/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
LOCATION="threed_sphere"
MODEL="none"
EXTRA="$DART"/models/template/threed_model_mod.f90
dev_test=1
TEST="random_seq"
LOCATION="threed_sphere"

serial_programs=(
test_corr
Expand Down
5 changes: 3 additions & 2 deletions developer_tests/time_manager/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
LOCATION="threed_sphere"
MODEL="none"
EXTRA="$DART"/models/template/threed_model_mod.f90
dev_test=1
TEST="time_manager"
LOCATION="threed_sphere"

serial_programs=(
timetest
Expand Down
3 changes: 2 additions & 1 deletion developer_tests/utilities/work/quickbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ main() {
export DART=$(git rev-parse --show-toplevel)
source "$DART"/build_templates/buildfunctions.sh

MODEL="template"
MODEL="none"
EXTRA="$DART"/models/template/threed_model_mod.f90
LOCATION="threed_sphere"
dev_test=1
TEST="utilities"
Expand Down
6 changes: 3 additions & 3 deletions models/wrf/shell_scripts/diagnostics_obs.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
source $paramfile

cd $OBS_DIAG_DIR
${COPY} ${TEMPLATE_DIR}/input.nml.template input.nml
${COPY} ${RUN_DIR}/input.nml input.nml
set gdate = (`echo $datea 0 -g | ${DART_DIR}/models/wrf/work/advance_time`)
set yyyy2 = `echo $datea | cut -b1-4`
set mm2 = `echo $datea | cut -b5-6`
Expand Down Expand Up @@ -57,7 +57,7 @@
set datef = `echo $datef $ASSIM_INT_HOURS | ${DART_DIR}/models/wrf/work/advance_time`

end
ls -1 obs_seq.final_* >! flist
readlink -f obs_seq.final_* >! flist

cat >! script.sed << EOF
/obs_sequence_name/c\
Expand Down Expand Up @@ -95,7 +95,7 @@
EOF


sed -f script.sed ${TEMPLATE_DIR}/input.nml.template >! input.nml
sed -f script.sed ${RUN_DIR}/input.nml >! input.nml

# create the state-space diagnostic summary

Expand Down
7 changes: 7 additions & 0 deletions models/wrf/shell_scripts/driver.csh
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had major trouble running tutorial without this check to make sure there were no pre-existing start_member and done_member diagnostic files that existed during first run through of driver.csh. There was a tendency for these diagnostic files to exist already from prior execution of init_ensemble_var.csh. This led to file transfers to begin within driver.csh before the WRF integrations were complete.

Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,13 @@ while ( 1 == 1 )

echo "ready to integrate ensemble members"

# Removing old start_member and done_member diagnostics
if ( -e ${RUN_DIR}/start_member_1) then
${REMOVE} ${RUN_DIR}/start_member_* \
${RUN_DIR}/done_member_*
endif


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If first time through the driver.csh, there could be lingering start_member or done_member files from the prior init_ensemble_var.csh run. Need to get rid of these or the driver.csh script starts transferring files that do not exist because WRF integration is not yet completed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mgharamti I know you are super busy, but as a regular WRF user, would be worth for you to look this over, especially commit 63a5748 . This commit changes some of the wrf shell scripting which extends outside the tutorial. I think it is OK and just a more simple way of linking the DART input.nml information to the WRF case. No rush on this. I think this is about as much as I want to change for foreseeable future. Thanks @mjs2369 for looking at this too. If everything looks OK please include in next DART release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@braczka I've looked over these changes and I agree that the WRF shell scripting needs to link ${RUN_DIR}/input.nml as opposed to ${TEMPLATE_DIR}/input.nml.template . I think this is ready to merge.

But there's no rush on this release (the bug-fix that Helen implemented is developer code only), so I also think that we should wait for @mgharamti to take a look if he has the time. If not, we can just go ahead and release sometime next week.

set n = 1
while ( $n <= $NUM_ENS )

Expand Down
2 changes: 1 addition & 1 deletion models/wrf/shell_scripts/gen_retro_icbc.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ while ( 1 == 1 )
if ( ! -d ${OUTPUT_DIR}/${datea} ) mkdir -p ${OUTPUT_DIR}/${datea}

cd ${ICBC_DIR}
${LINK} ${TEMPLATE_DIR}/input.nml.template input.nml
${LINK} ${RUN_DIR}/input.nml input.nml
${REMOVE} gfs*pgrb2* *grib2

# prepare to run WPS ungrib and metgrid
Expand Down
6 changes: 1 addition & 5 deletions models/wrf/shell_scripts/init_ensemble_var.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ set dd = `echo $initial_date | cut -b7-8`
set hh = `echo $initial_date | cut -b9-10`

${COPY} ${TEMPLATE_DIR}/namelist.input.meso namelist.input
${COPY} ${TEMPLATE_DIR}/input.nml.template input.nml
${REMOVE} ${RUN_DIR}/WRF
${LINK} ${OUTPUT_DIR}/${initial_date} WRF

Expand All @@ -61,11 +60,8 @@ while ( $n <= $NUM_ENS )

mkdir -p ${RUN_DIR}/advance_temp${n}

# TJH why does the run_dir/*/input.nml come from the template_dir and not the rundir?
# TJH furthermore, template_dir/input.nml.template and rundir/input.nml are identical. SIMPLIFY.

${LINK} ${RUN_DIR}/WRF_RUN/* ${RUN_DIR}/advance_temp${n}/.
${LINK} ${TEMPLATE_DIR}/input.nml.template ${RUN_DIR}/advance_temp${n}/input.nml
${LINK} ${RUN_DIR}/input.nml ${RUN_DIR}/advance_temp${n}/input.nml

${COPY} ${OUTPUT_DIR}/${initial_date}/wrfinput_d01_${gdate[1]}_${gdate[2]}_mean \
${RUN_DIR}/advance_temp${n}/wrfvar_output.nc
Expand Down
9 changes: 5 additions & 4 deletions models/wrf/shell_scripts/new_advance_model.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -219,11 +219,12 @@ while($state_copy <= $num_states) # MULTIPLE DOMAINS - we don't expect advan
# endif

# link WRF-runtime files (required) and be.dat (if using WRF-Var)
${LN} ${CENTRALDIR}/WRF_RUN/* .

# link DART namelist
${COPY} ${CENTRALDIR}/input.nml .
${LN} ${CENTRALDIR}/WRF_RUN/* .

# Copy DART namelist if necessary
if ( ! -e input.nml) then
${COPY} ${CENTRALDIR}/input.nml .
endif
# append LSM data from previous cycle
if ( -e ${CENTRALDIR}/append_lsm_data ) then
${LN} ${CENTRALDIR}/LSM/lsm_data_${ensemble_member}.nc lsm_data.nc
Expand Down
2 changes: 1 addition & 1 deletion models/wrf/shell_scripts/setup.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ ${COPY} ${SHELL_SCRIPTS_DIR}/add_bank_perts.ncl ${RUN_DIR} || exit 8
${COPY} ${SHELL_SCRIPTS_DIR}/new_advance_model.csh ${RUN_DIR} || exit 9

# Edit input.nml.template so that its ens_size is set to the same value as $NUM_ENS in param.csh
sed "s/ens_size.*/ens_size =$NUM_ENS,/g" ${DART_DIR}/models/wrf/tutorial/template/input.nml.template > ${RUN_DIR}/input.nml || exit 8
sed "s/ens_size.*/ens_size = $NUM_ENS,/g" ${DART_DIR}/models/wrf/tutorial/template/input.nml.template > ${RUN_DIR}/input.nml || exit 8
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setup.csh script was designed to apply ensemble member updates to the run directory input.nml, thus all WRF shell scripting (e.g. gen_retro_icbc.csh, init_ensemble_var.csh, new_advance_model.csh etc) should link or copy the run directory input.nml, not the input.nml located in the template directory. This is now consistent with the WRF tutorial instructions.


echo "$myname complete at "`date`
echo
Expand Down
12 changes: 8 additions & 4 deletions models/wrf/tutorial/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,9 +237,8 @@ bash ``export BASE_DIR=<path_to_your_working_directory>``
cp $DART_DIR/models/wrf/tutorial/template/namelist.input.meso $BASE_DIR/template/.
cp $DART_DIR/models/wrf/tutorial/template/namelist.wps.template $BASE_DIR/template/.

3. You will also need the scripting to run a WRF/DART experiment. Copy
the contents of ``$DART_DIR/models/wrf/shell_scripts`` to the
``$BASE_DIR/scripts`` directory.
3. You will also need scripting to run a WRF/DART experiment. Copy the contents of
``$DART_DIR/models/wrf/shell_scripts`` to the ``$BASE_DIR/scripts`` directory.

::

Expand Down Expand Up @@ -457,7 +456,7 @@ changes:
+====================+======================================+=========================================================================================================================================================================================================================================================+
| driver.csh | datefnl = 2017042712 | Change to the final target date; here the final date is already set correctly for this tutorial. |
+--------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| gen_retro_icbc.csh | datefnl = 2017043000 | This is the final date to create WRF initial/boundary conditions for. This is set to the last date that files are included in the tutorial. |
| gen_retro_icbc.csh | datefnl = 2017042712 | Set to the final target date of the tutorial. However, it is possible (not necessary) to create WRF initial/boundary conditions to 2017043000. This is the latest date that files are included in the tutorial. |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to run gen_retro_icbc.csh all the way to 2017043000 when tutorial only runs to 2017042712. This helps streamline troubleshooting.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch here

+--------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| gen_retro_icbc.csh | paramfile = <full path to param.csh> | The full path to param.csh. Change this on the line after the comment. While these two files are in the same directory here, in general it is helpful to have one param.csh for each experiment. |
+--------------------+--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Expand Down Expand Up @@ -557,6 +556,11 @@ and look near the end. You will need to modify the lines staring with
also want to modify this script to test running a single member first —
just in case you have some debugging to do.

However, be warned that to successfully complete the tutorial, including
running the *driver.csh* script in Step 5, using a smaller ensemble
(e.g. < 20 members) can lead to spurious updates during the analysis step,
causing the WRF simulation to fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tutorial can run OK with as little as 20 ensemble members. Wanted to avoid confusion with new users that using less than recommended 50 members, although good for quick troubleshooting, introduces problems if not done carefully.

When complete for the full ensemble, you should find 50 new files in the
directory ``output/2017042700/PRIORS`` with names like *prior_d01.0001*,
*prior_d01.0002*, etc... You may receive an e-mail to helpfully inform
Expand Down