Update to newer version of UFS Weather Model using srw_common

[chan-hoo]

DESCRIPTION OF CHANGES:

• Update to a newer version of UFS Weather Model, which is used for the real-time parallel run at EMC, required for new divergence damping values.
• Introduce a common module file 'srw_common' similar with 'ufs_common' of UFS Weather Model.
• 'PLATFORM(MACHINE)' is auto-detected in 'devbuild.sh'
• Steps to build the app:

1. git clone -b feature/new_ufs_weather_model https://github.com/chan-hoo/ufs-srweather-app
2. cd ufs-srweather-app
3. vim Externals.cfg => change the regional workflow to: https://github.com/chan-hoo/regional_wofklow, branch: feature/nord_d4bg
4. ./manage_externals/checkout_externals
5. ./devbuild.sh

TESTS CONDUCTED:

• GST test on WCOSS dell and cray, Hera, and Orion.
• A test on other machines such as Jet and Cheyenne is needed.

DEPENDENCIES:

• NOAA-EMC/regional_workflow/pull/657

ISSUE:

• Fixes issue mentioned in Update to newer ufs weather model necessary #199
• Fixes issue mentioned in Allow the ufs-weather-model build to use the ufs_common module #154

[chan-hoo]

@gsketefian, @JeffBeck-NOAA, @mkavulich, If you agree with this change using 'srw_common', can you test this on Jet and Cheyenne? I don't have my account on them.

[chan-hoo]

@JeffBeck-NOAA @gsketefian @mkavulich , can you review this PR? This hash of the ufs weather model is required to use the new divergence damping parameters.

[JeffBeck-NOAA]

@chan-hoo, a few questions on these changes:

1. Does a common build file (srw_common) cause a problem when different machines require different module versions?
2. Is there a reason we can't use ufs_common from ufs-weather-model instead of maintaining our own srw_common?
3. With these changes, what happens to the devbuild.sh script we have been using to build the model?

Thanks!

@mkavulich, can you review this PR? Thanks!

[chan-hoo]

@JeffBeck-NOAA, 1) I think so, but I've checked the versions of modules over the machines (except for Jet and Cheyenne) and put the common modules and their versions in srw_common. 2) Different from ufs-weather-model, the ufs srw app should include the modules for UFS_UTILS. srw_common=ufs_common+ufs_utils. Moreover, ufs_common is out of our control (totally dependent on the external component). In my opinion, it is better to have an independent env file that is totally under our control. 3) I didn't check 'devbuild.sh'. I'll test and modify it soon.

[JeffBeck-NOAA]

@chan-hoo, do we still need build_app.sh if you make the necessary modifications to devbuild.sh?

[chan-hoo]

@JeffBeck-NOAA, I agree. I'll put the machine-detect command into devbuild.sh.

[JeffBeck-NOAA]

@JeffBeck-NOAA, I agree. I'll put the machine-detect command into devbuild.sh.

Thanks, @chan-hoo! Also, I just built your code on Jet and it works. However, should it be loading srw_common instead of build_jet_intel.env? It's loading the latter.

[chan-hoo]

@JeffBeck-NOAA, I've modified devbuild.sh. It works well on Hera, Orion, WCOSS dell and cray. Can you test it on Jet and Cheyenne? The modified devbuild.sh doesn't require PLATFORM any more. You can run it with "./devbuild.sh".

[JeffBeck-NOAA]

@JeffBeck-NOAA, I've modified devbuild.sh. It works well on Hera, Orion, WCOSS dell and cray. Can you test it on Jet and Cheyenne? The modified devbuild.sh doesn't require PLATFORM any more. You can run it with "./devbuild.sh".

@chan-hoo, before I test, I still see that devbuild.sh is sourcing the build_jet_intel.env file on Jet, and not srw_common. Is this correct. How is srw_common used?

[chan-hoo]

@JeffBeck-NOAA, Loading srw_common is included in 'build_jet_intel.env'. so if you source 'build_jet_intel.env', the modules in 'srw_common' will be loaded together.

[chan-hoo]

@JeffBeck-NOAA, If you want to source 'build_jet_intel.env' separately, you should run "module use {srw_dir}/env" before you run "source build_jet_intel.env". Otherwise, srw_common will not be loaded.

[JeffBeck-NOAA]

@JeffBeck-NOAA, If you want to source 'build_jet_intel.env' separately, you should run "module use {srw_dir}/env" before you run "source build_jet_intel.env". Otherwise, srw_common will not be loaded.

Sorry, @chan-hoo. My mistake! I'm testing the build right now on Jet with your latest commits included.

[JeffBeck-NOAA]

@JeffBeck-NOAA, If you want to source 'build_jet_intel.env' separately, you should run "module use {srw_dir}/env" before you run "source build_jet_intel.env". Otherwise, srw_common will not be loaded.

Sorry, @chan-hoo. My mistake! I'm testing the build right now on Jet with your latest commits included.

Build succeeded on Jet. @mkavulich, if you have time to review this PR, it would great to have your expertise in evaluating the changes to the build system.

[chan-hoo]

@JeffBeck-NOAA, since we didn't hear from others, can you approve this PR if you don't have any other comments?

[chan-hoo]

Thank you!

[gsketefian]

Sorry I'm late to this, just can't get around to everything in time. I just ran into a problem building (see #2 below). Some comments:

1. I think leaving out the automatic detection of the machine from devbuild.sh was intentional on the part of @mkavulich. Maybe he can comment.
2. devbuild.sh is not officially supporte. Instead, the instructions here are the official way to build. With that, srw_common is not found -- need to have something like module use ./ before module use srw_common (as you point out above). Again, @mkavulich wrote that script, so I'll let him comment, but we need a fix since following the supported instructions results in failure.
3. Question for @chan-hoo: Does the UFS now have a common set of modules that it uses on all machines so that we are guaranteed that srw_common will be the same on all platforms?

[chan-hoo]

@gsketefian, (2) I've added an example to the official wiki page. (3)Yes. I checked the common modules over the tier-1 platforms except for Cheyenne and Jet, but I checked their modules in the ufs_common file of the ufs weather model. @JeffBeck-NOAA confirmed that it worked on Jet. So I expect it should work on Cheyenne, too.