Bring external land component support to UFS

[uturuncoglu]

PR Checklist

• This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.

• This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR

• An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR are specified below.

• Results for one or more of the regression tests change and the reasons for the changes are understood and explained below.

• New or updated input data is required by this PR. If checked, please work with the code managers to update input data sets on all platforms.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsibility to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

This provide aims to add Noah MP land model as an external land component. The implementation does also support existing model configurations via Noah MP land model under CCPP/Physics. Along with this PR, three different RT are also implemented,

• datm_cdeps_lnd_gswp3: 1 day run with CDEPS GSWP3 dataset
• datm_cdeps_lnd_gswp3_rst: restart run with CDEPS GSWP3 dataset
• control_p8_atmlnd_sbs: side-by-side test with active atmosphere, uses same configuration with control_p8 (not using -D32BIT=ON) and the interaction with FV3 is uni-directional to allow comparing the results coming from FV3/CCPP/NOAHMP with external NOAHMP component.

A new namelist option cpllnd is introduced under gfs_physics_nml group to allow coupling with external land component.

The initial testing is done 5-days simulation with side-by-side model configuration - control_p8_atmlnd_sbs and results are compared with the FV3 output (in this case, the land fluxes are calculated under CCPP/Physics). The results indicates that the external land components able to reproduce the results calculated under CCPP/Physics. The following figures demonstrates the performance of the external land component compared with the results calculated with CCPP/Physics.

Difference of Latent Heat Flux (W/m2): external land - fv3/ccpp/lnd

• After 3-hour:
  plot_cubed_diff_lhtfl_evap_22-03-2021_09:00.pdf
• After 5-days:
  plot_cubed_diff_lhtfl_evap_27-03-2021_06:00.pdf
• Time series of single point in each tile (incl. RMSE):
  plot_cubed_ts_lhtfl_evap.pdf

Difference of Sensible Heat Flux (W/m2): external land - fv3/ccpp/lnd

• After 3-hour:
  plot_cubed_diff_shtfl_hflx_22-03-2021_09:00.pdf
• After 5-days:
  plot_cubed_diff_shtfl_hflx_27-03-2021_06:00.pdf
• Time series of single point in each tile (incl. RMSE):
  plot_cubed_ts_shtfl_hflx.pdf

Difference of Surface Temperature (K or degC): external land - fv3/ccpp/lnd

• After 3-hour:
  plot_cubed_diff_tmpsfc_tskin_22-03-2021_09:00.pdf
• After 5-days:
  plot_cubed_diff_tmpsfc_tskin_27-03-2021_06:00.pdf
• Time series of single point in each tile (incl. RMSE):
  plot_cubed_ts_tmpsfc_tskin.pdf

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues must always be created before starting work on a PR branch!)

• it does not fix the issue [Inconsistent land-sea mask among sfc*, oro* and static information files for control_p8 RT #1423] but this needs to be fixed by updating the input files.

Testing

How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)

• hera.intel
• hera.gnu
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss2.intel
• acorn.intel
• opnReqTest for newly added/changed feature
• CI

Note that following tests are failing in baseline check:

• control_p8
• control_p8_lndp
• control_decomp_p8
• control_2threads_p8
• merra2_thompson
• control_debug_p8
• atmaero_control_p8
• atmaero_control_p8_rad_micro

This is mainly due to the changes that is done in tests/fv3_conf/control_run.IN to find the land sea mask issue. We could modify it to affect only newly introduced RTs for external land work until #1423 is resolved. Another solution is proposed also in #1423 and it only requires update in NoahMP ESMF/NUOPC cap and removing modification in tests/fv3_conf/control_run.IN. This is again not ideal solution and could create potential problem in the future when fully coupled atm-lnd configuration implemented.

Dependencies

If testing this branch requires non-default branches in other repositories, list them. Those branches should have matching names (ideally).

Do PRs in upstream repositories need to be merged first?
If so add the "waiting for other repos" label and list the upstream PRs

• waiting on [Add NUOPC cap to the model to support external land component under UFS NOAA-EMC/noahmp#1]
• waiting on [Add support for external land component NOAA-EMC/fv3atm#591]
• waiting on [Bring external land model support ccpp-physics#10]
• waiting on [Bring external land component support to UFS ESCOMP/CMEPS#310] and [Update CMEPS to latest ESCOMP/master NOAA-EMC/CMEPS#72]
• waiting on [Update CDEPS NOAA-EMC/CDEPS#50]

Following issue is also created: #1442

Following directories are also needed to include into INPUT directory (on Orion):
/work/noaa/nems/tufuk/RT/NEMSfv3gfs/input-data-20220414/NOAHMP_IC
/work/noaa/nems/tufuk/RT/NEMSfv3gfs/input-data-20220414/DATM_GSWP3_input_data

[DeniseWorthen]

@jkbk2004 What do you mean by "similar issues on other files on baselines"? Baselines for the added tests here, or baselines for unrelated (current) tests?

[uturuncoglu]

I agree. Creating issue will be great. I could do it in noahmp repo. In any case this part of code will be replaced with ESMF calls soon.

[jkbk2004]

@DeniseWorthen yeah, I mean another baseline on this PR and across other variables as well. @uturuncoglu I am creating issue now.

[DeniseWorthen]

@uturuncoglu CMEPS has been merged. You can revert the gitmodules and update to point to the latest emc/develop. Thanks for your hard work on this project!

[uturuncoglu]

@DeniseWorthen done

[uturuncoglu]

@barlage can you merge NoahMP PR? By the way, he might be out of office. I am not sure. In nay case, Is there anybody else that has right to merge that PR.

[BrianCurtis-NOAA]

FV3 hash is now: NOAA-EMC/fv3atm@324591b please revert that .gitmodule and update that hash. Still waiting on NoahMP

[uturuncoglu]

@BrianCurtis-NOAA done

[uturuncoglu]

@BrianCurtis-NOAA I think I also need to point NOAA-EMC/CDEPS. Is it ready?

[BrianCurtis-NOAA]

Look like it's been merged: hash is NOAA-EMC/CDEPS@530c262

[uturuncoglu]

@BrianCurtis-NOAA Okay. CDEPS is also updated.

[uturuncoglu]

@BrianCurtis-NOAA @jkbk2004 I am plaining to merge the noahmp PR manually from command line which is easier for us for the initial noahmp PR. So, this will update the noahmp repository develop branch. Then, I'll point it from UFS level like I did for other. Then, UFS will be ready to merge. I am doing a small test in noahmp side to be sure it is okay. I'll let you know soon.

[uturuncoglu]

@BrianCurtis-NOAA @jkbk2004 it is done. All the sub-modules are merged and also pointed to NOAA-EMC forks.

[jkbk2004]

@uturuncoglu Thanks for all the hard work!

[BrianCurtis-NOAA]

Automated RT Failure Notification
Machine: gaea
Compiler: intel
Job: RT
[RT] Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/1074134404/20221027011508/ufs-weather-model
Please make changes and add the following label back: gaea-intel-RT

[uturuncoglu]

@uturuncoglu Thanks for all the hard work!

Thanks all of you to help me. It is really great to have your fabulous support! It was impossible without your help. Have a nice weekend.