Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update with fv3 & ccpp/physics updates in order to turn the NSST model on in the coupled model (Replace PR #453) #483

Merged
merged 19 commits into from
Apr 9, 2021
Merged

update with fv3 & ccpp/physics updates in order to turn the NSST model on in the coupled model (Replace PR #453) #483

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
482dd64
point to fv3 branch
XuLi-NOAA Mar 25, 2021
dad728f
Add one more test cpld_control_nsst to test added ccpp suite (FV3_GFS…
XuLi-NOAA Mar 25, 2021
48c414f
Modify rt.conf to add FV3_GFS_v15p2_couplednsst
XuLi-NOAA Mar 29, 2021
9c7e5b8
Modify rt.conf to add a new test, cpld_bmarkfrac_v16_nsst, remove te…
XuLi-NOAA Mar 29, 2021
c73d62b
merge to the develop branch in ufs-weather-model
XuLi-NOAA Mar 31, 2021
0dc1d1e
Remove the "," after FV3_GFS_v16_couplednsst in rt.conf
XuLi-NOAA Mar 31, 2021
e060e17
Modify tests/tests/cpld_bmarkfrac_v16_nsst: 1. cpld_bmarkfrac_v16 to …
XuLi-NOAA Mar 31, 2021
f922305
Modify input.benchmark_v16.nml.IN & cpld_bmarkfrac_v16_nsst for a con…
XuLi-NOAA Mar 31, 2021
8993cf2
merge to the develop branch in ufs-weather-model
XuLi-NOAA Mar 31, 2021
2f6fb4d
Modify cpld_bmarkfrac_v16_nsst by moving the NSTF_NAME to the nameli…
XuLi-NOAA Apr 1, 2021
5a0f6d2
merge to the develop branch in ufs-weather-model
XuLi-NOAA Apr 5, 2021
2af3f31
remove tests/tests/cpld_control_nsst since it has been replaced by cp…
XuLi-NOAA Apr 6, 2021
6d4e019
Change BL_DATE to be: develop-20210406
XuLi-NOAA Apr 6, 2021
b465ef7
RegressionTests_orion.intel.log of the rt run and BL_DATE=20210406 in…
XuLi-NOAA Apr 7, 2021
7d0b2c9
RT JOBS PASSED: hera.intel. Log file uploaded.
BrianCurtis-NOAA Apr 8, 2021
c5842b9
run-ci, commit 7 RegressionTest log files
XuLi-NOAA Apr 8, 2021
127fa9a
Push RegressionTests_wcoss_dell_p3.log
XuLi-NOAA Apr 8, 2021
afed340
Update .gitmodules back to develop
XuLi-NOAA Apr 8, 2021
fda9a8b
Update FV3
XuLi-NOAA Apr 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
url = https://github.com/XuLi-NOAA/fv3atm
branch = feature/ufs-nst
[submodule "NEMS"]
path = NEMS
url = https://github.com/NOAA-EMC/NEMS
Expand Down
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated 5 files
+2 −2 .gitmodules
+1 −1 ccpp/physics
+94 −0 ccpp/suites/suite_FV3_GFS_v15p2_couplednsst.xml
+95 −0 ccpp/suites/suite_FV3_GFS_v16_couplednsst.xml
+1 −1 io/FV3GFS_io.F90
3 changes: 2 additions & 1 deletion tests/rt.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ RUN | fv3_gfs_v16_ras_debug
# CPLD tests #
###################################################################################################################################################################################

COMPILE | APP=S2S SUITES=FV3_GFS_2017_coupled,FV3_GFS_2017_satmedmf_coupled,FV3_GFS_v15p2_coupled,FV3_GFS_v16_coupled | - wcoss_cray jet.intel | fv3 |
COMPILE | APP=S2S SUITES=FV3_GFS_2017_coupled,FV3_GFS_2017_satmedmf_coupled,FV3_GFS_v15p2_coupled,FV3_GFS_v16_coupled,FV3_GFS_v16_couplednsst | - wcoss_cray jet.intel | fv3 |
RUN | cpld_control | - wcoss_cray jet.intel | fv3 |
RUN | cpld_restart | - wcoss_cray jet.intel | | cpld_control
RUN | cpld_controlfrac | - wcoss_cray jet.intel | fv3 |
Expand Down Expand Up @@ -158,6 +158,7 @@ RUN | cpld_restart_bmarkfrac
#6h/6h/12h restart test
# test fails on gaea with esmfpio error
RUN | cpld_bmarkfrac_v16 | - wcoss_cray gaea.intel jet.intel | fv3 |
RUN | cpld_bmarkfrac_v16_nsst | - wcoss_cray gaea.intel jet.intel | fv3 |
RUN | cpld_restart_bmarkfrac_v16 | - wcoss_cray gaea.intel jet.intel | | cpld_bmarkfrac_v16

COMPILE | APP=S2SW SUITES=FV3_GFS_2017_coupled,FV3_GFS_2017_satmedmf_coupled,FV3_GFS_v15p2_coupled,FV3_GFS_v16_coupled | - wcoss_cray gaea.intel jet.intel | fv3 |
Expand Down
150 changes: 150 additions & 0 deletions tests/tests/cpld_bmarkfrac_v16_nsst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
#
# cpld_bmarkfrac_v16_nsst test
#

export TEST_DESCR="Fully coupled FV3-CCPP-MOM6-CICE-CMEPS system - C384L127 MX025 - Benchmark test - frac grid - v16"

export CNTL_DIR="cpld_bmarkfrac_v16_nsst"

export LIST_FILES="phyf012.tile1.nc \
phyf012.tile2.nc \
phyf012.tile3.nc \
phyf012.tile4.nc \
phyf012.tile5.nc \
phyf012.tile6.nc \
dynf012.tile1.nc \
dynf012.tile2.nc \
dynf012.tile3.nc \
dynf012.tile4.nc \
dynf012.tile5.nc \
dynf012.tile6.nc \
RESTART/coupler.res \
RESTART/fv_core.res.nc \
RESTART/fv_core.res.tile1.nc \
RESTART/fv_core.res.tile2.nc \
RESTART/fv_core.res.tile3.nc \
RESTART/fv_core.res.tile4.nc \
RESTART/fv_core.res.tile5.nc \
RESTART/fv_core.res.tile6.nc \
RESTART/fv_srf_wnd.res.tile1.nc \
RESTART/fv_srf_wnd.res.tile2.nc \
RESTART/fv_srf_wnd.res.tile3.nc \
RESTART/fv_srf_wnd.res.tile4.nc \
RESTART/fv_srf_wnd.res.tile5.nc \
RESTART/fv_srf_wnd.res.tile6.nc \
RESTART/fv_tracer.res.tile1.nc \
RESTART/fv_tracer.res.tile2.nc \
RESTART/fv_tracer.res.tile3.nc \
RESTART/fv_tracer.res.tile4.nc \
RESTART/fv_tracer.res.tile5.nc \
RESTART/fv_tracer.res.tile6.nc \
RESTART/phy_data.tile1.nc \
RESTART/phy_data.tile2.nc \
RESTART/phy_data.tile3.nc \
RESTART/phy_data.tile4.nc \
RESTART/phy_data.tile5.nc \
RESTART/phy_data.tile6.nc \
RESTART/sfc_data.tile1.nc \
RESTART/sfc_data.tile2.nc \
RESTART/sfc_data.tile3.nc \
RESTART/sfc_data.tile4.nc \
RESTART/sfc_data.tile5.nc \
RESTART/sfc_data.tile6.nc \
RESTART/MOM.res.nc \
RESTART/MOM.res_1.nc \
RESTART/MOM.res_2.nc \
RESTART/MOM.res_3.nc \
RESTART/iced.2013-04-01-43200.nc \
RESTART/ufs.cpld.cpl.r.2013-04-01-43200.nc"

export_fv3
export_cpl

export SYEAR="2013"
export SMONTH="04"
export SDAY="01"
export SHOUR="00"

export DAYS="0.5"
export FHMAX="12"
export RESTART_INTERVAL="6"
export RESTART_N=${RESTART_INTERVAL}

export TASKS=$TASKS_cpl_bmrk
export TPN=$TPN_cpl_bmrk
export INPES=$INPES_cpl_bmrk
export JNPES=$JNPES_cpl_bmrk
export THRD=$THRD_cpl_bmrk
export WRTTASK_PER_GROUP=$WPG_cpl_bmrk

export med_petlist_bounds=$MPB_cpl_bmrk
export atm_petlist_bounds=$APB_cpl_bmrk
export ocn_petlist_bounds=$OPB_cpl_bmrk
export ice_petlist_bounds=$IPB_cpl_bmrk

# atm/ocn/ice resolution
export ATMRES='C384'
export NPX=385
export NPY=385
export IMO=1536
export JMO=768

export OCNRES='025'
export ICERES='0.25'
export NX_GLB=1440
export NY_GLB=1080
export NPROC_ICE='48'

# resolution dependent setting
export CDMBWD=${CDMBWD_c384}

# set component and coupling timesteps
export DT_ATMOS='225'
export DT_CICE=${DT_ATMOS}
export DT_DYNAM_MOM6='900'
export DT_THERM_MOM6='1800'
export CPL_SLOW=${DT_THERM_MOM6}
export CPL_FAST=${DT_ATMOS}

# nems.configure
export coupling_interval_slow_sec=${CPL_SLOW}
export coupling_interval_fast_sec=${CPL_FAST}

export FRAC_GRID='.T.'
export CPLMODE='nems_frac'

export NPZ=127
export NPZP=128
export PSM_BC=1
export DDDMP=0.2

# resolution dependent files
export MOM_INPUT="MOM_input_template_${OCNRES}"
export MESHOCN_ICE="mesh.mx${OCNRES}.nc"
export CICEGRID="grid_cice_NEMS_mx${OCNRES}.nc"
export CICEMASK="kmtu_cice_NEMS_mx${OCNRES}.nc"
export CHLCLIM="seawifs-clim-1997-2010.${NX_GLB}x${NY_GLB}.v20180328.nc"
export FRUNOFF="runoff.daitren.clim.${NX_GLB}x${NY_GLB}.v20180328.nc"

export FNALBC="'global_snowfree_albedo.bosu.t766.1536.768.rg.grb',"
export FNVETC="'global_vegtype.igbp.t766.1536.768.rg.grb',"
export FNSOTC="'global_soiltype.statsgo.t766.1536.768.rg.grb',"
export FNSMCC="'global_soilmgldas.statsgo.t766.1536.768.grb',"
export FNABSC="'global_mxsnoalb.uariz.t766.1536.768.rg.grb',"

export OZ_PHYS_NEW=".T."

export MOM6_RIVER_RUNOFF='True'
export MOM6_RESTART_SETTING="r"
export MOM6_REPRO_LA='True'

export RUNID="cpcice"

export INPUT_NML=input.benchmark_v16.nml.IN

export FIELD_TABLE="field_table_gfsv16"
export DIAG_TABLE="diag_table_bmark_v16_template"
export SUITE_NAME="FV3_GFS_v16_couplednsst"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but shouldn't this be export CCPP_SUITE="FV3_GFS_v16_couplednsst" ? Please check other regression tests.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@climbfuji The parm/input.benchmark_v16.nml.IN uses SUITE_NAME:

&atmos_model_nml
  blocksize = 32
  chksum_debug = .false.
  dycore_only = .false.
  fdiag = 6
  fhmax = 840
  fhout = 6
  fhmaxhf = 0
  fhouthf = -1
  ccpp_suite = '@[SUITE_NAME]'

/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This inconsistency should be fixed at some point. It's unintuitive that SUITE_NAME environment variable is used to set the value of ccpp_suite namelist variable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fixed after creating the baseline and before verifying against it. Many workflows depend on this being called CCPP_SUITE and not SUITE_NAME.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable is SUITE_NAME in all the coupled tests, not just the v16 input nml. It requires a change in default_vars, the three input nmls that the coupled tests use( input.benchmark_ccpp.nml.IN, input.benchmark_v16.nml.IN, input.mom6_ccpp.nml.IN) and then in all the cpld tests which are using a non-default setting.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this should never have been merged, but apparently I didn't catch it. Let's fix this in a separate PR then.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add it to the CMEPS PR if you like.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be great, yes. Thanks!

export nstf_name=2,1,0,0,0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to make nstf_name in the file input.benchmark_v16.nml.IN to be a variable, which is defined here in the test. So, here define: NSTF_NAME='2,1,0,0,0'. Then in input.benchmark_v16.nml.IN set nstf_name = @[NSTF_NAME]. This will also need to be done in the same way for the control_nsst test that you added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to make nstf_name in the file input.benchmark_v16.nml.IN to be a variable, which is defined here in the test. So, here define: NSTF_NAME='2,1,0,0,0'. Then in input.benchmark_v16.nml.IN set nstf_name = @[NSTF_NAME]. This will also need to be done in the same way for the control_nsst test that you added.

@DeniseWorthen: Yes, it is better, since it is consistent with all the places of nstf_name in the parm/input*nml files. I change NSTF_NAME to be nstf_name since " nstf_name = 0,0,0,0,0" has been defined in input.benchmark_v16.nml.IN, I will remove this, and add nstf_name = @[NSTF_NAME] in input.benchmark_v16.nml.IN, then I can keep the upper case NSTF_NAME='2,1,0,0,0' in cpld_bmarkfrac_v16_nsst, as I did for cpld_control_nsst. This is the first time for me to create a test, a good experience.


export FV3_RUN=cpld_bmark_run.IN
64 changes: 64 additions & 0 deletions tests/tests/cpld_control_nsst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#
# cpld_control_nsst test
#

export TEST_DESCR="Fully coupled FV3-CCPP-MOM6-CICE-CMEPS system - C96MX100"

export CNTL_DIR="cpld_control_nsst"

export LIST_FILES="phyf024.tile1.nc \
phyf024.tile2.nc \
phyf024.tile3.nc \
phyf024.tile4.nc \
phyf024.tile5.nc \
phyf024.tile6.nc \
dynf024.tile1.nc \
dynf024.tile2.nc \
dynf024.tile3.nc \
dynf024.tile4.nc \
dynf024.tile5.nc \
dynf024.tile6.nc \
RESTART/coupler.res \
RESTART/fv_core.res.nc \
RESTART/fv_core.res.tile1.nc \
RESTART/fv_core.res.tile2.nc \
RESTART/fv_core.res.tile3.nc \
RESTART/fv_core.res.tile4.nc \
RESTART/fv_core.res.tile5.nc \
RESTART/fv_core.res.tile6.nc \
RESTART/fv_srf_wnd.res.tile1.nc \
RESTART/fv_srf_wnd.res.tile2.nc \
RESTART/fv_srf_wnd.res.tile3.nc \
RESTART/fv_srf_wnd.res.tile4.nc \
RESTART/fv_srf_wnd.res.tile5.nc \
RESTART/fv_srf_wnd.res.tile6.nc \
RESTART/fv_tracer.res.tile1.nc \
RESTART/fv_tracer.res.tile2.nc \
RESTART/fv_tracer.res.tile3.nc \
RESTART/fv_tracer.res.tile4.nc \
RESTART/fv_tracer.res.tile5.nc \
RESTART/fv_tracer.res.tile6.nc \
RESTART/phy_data.tile1.nc \
RESTART/phy_data.tile2.nc \
RESTART/phy_data.tile3.nc \
RESTART/phy_data.tile4.nc \
RESTART/phy_data.tile5.nc \
RESTART/phy_data.tile6.nc \
RESTART/sfc_data.tile1.nc \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this test cpld_control_nsst is used? I don't see SUITE_NAME being set, how is this test supposed to run/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should remove this file tests/tests/cpld_control_nsst

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@XuLi-NOAA You need to either remove this file from the PR or add it as a test in rt.conf. If you add it as a test, you need to set the SUITE_NAME in the test, otherwise it will use the default suite in input.mom6_ccpp.nml.IN

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this test cpld_control_nsst is used? I don't see SUITE_NAME being set, how is this test supposed to run/

@climbfuji : cpld_control_nsst is not used since it doesn't use v16 physics, as suggested by Bin & Jun, another one, has been created and tested, cpld_bmarkfrac_v16_nsst.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, in this case, please remove tests/tests/cpld_control_nsst

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, in this case, please remove tests/tests/cpld_control_nsst

@climbfuji : I will remove cpld_control_nsst and it doesn't affect anything we have done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@climbfuji : cpld_control_nsst removed

RESTART/sfc_data.tile2.nc \
RESTART/sfc_data.tile3.nc \
RESTART/sfc_data.tile4.nc \
RESTART/sfc_data.tile5.nc \
RESTART/sfc_data.tile6.nc \
RESTART/MOM.res.nc \
RESTART/iced.2016-10-04-00000.nc \
RESTART/ufs.cpld.cpl.r.2016-10-04-00000.nc"

export_fv3
export_cpl

export RESTART_INTERVAL="12"
export RESTART_N=${RESTART_INTERVAL}
export NSTF_NAME=2,1,0,0,0

export FV3_RUN=cpld_control_run.IN