Updates from v2 release branch (#521)

Updates to the documentation of the ufs-weather-model, developed as part of the v2.0.0 release, and a few bug fixes that were also part of that release.

ufs-weather-model: doc/ subdirectory, add the RRFS_v1alpha to rt.conf
NEMS: doc directory only
fv3atm: add RRFS_v1alpha SDF
ccpp-physics: documentation, fix assumed-size array dimensions (gnu10 fix), rename gmtb-scm to scm (Single Column Model)
atmos_cubed_sphere: doc directory only

doc/UsersGuide/source/BuildingAndRunning.rst

@@ -15,14 +15,14 @@ There are two categories of libraries that are needed:
    Most have an NCEPLIBS prefix in the repository, e.g. NCEPLIBS-bacio. Select tools from the UFS
    Utilities repository (UFS-UTILS) are also included in this category. A list of the bundled
    libraries tested with this WM release is in the top-level ``README`` of the `NCEPLIBS repository
-   <https://github.com/NOAA-EMC/NCEPLIBS/tree/ufs-v1.1.0>`_ (**be sure to look at the tag in that repository that
+   <https://github.com/NOAA-EMC/NCEPLIBS/tree/ufs-v2.0.0>`_ (**be sure to look at the tag in that repository that
    matches the tag on this WM release**).
 
 #. Third-party libraries (NCEPLIBS-external). These are libraries that were developed external to
    the UFS Weather Model. They are general software packages that are also used by other models in
    the community. Building these is optional, since existing builds of these libraries can be pointed
    to instead. A list of the external libraries tested with this WM release is in the top-level ``README``
-   of the `NCEPLIBS-external repository <https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v1.1.0>`_. Again, be
+   of the `NCEPLIBS-external repository <https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v2.0.0>`_. Again, be
    sure to look at the tag in that repository that matches the tag on this WM release.
 
 .. note::
@@ -34,16 +34,16 @@ to build NCEPLIBS and NCEPLIBS-external or are working on a system that is alrea
 pre-configured platforms, the libraries are already available.
 
 If you do have to build the libraries, it is a good idea to check the platform- and compiler-specific
-``README`` files in the doc/ directory of the `NCEPLIBS-external repository <https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v 1.1.0>`_
+``README`` files in the doc/ directory of the `NCEPLIBS-external repository <https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v 2.0.0>`_
 as a first step, to see if your system or one similar to it is included. These files have detailed
 instructions for building NCEPLIBS-external, NCEPLIBS, and the UFS Weather Model. They may be all the
 documentation you need. Be sure to use the tag that corresponds to this version of the WM, and define a
 WORK directory path before you get started.
 
 If your platform is not included in these platform- and compiler-specific ``README`` files, there is a more
 generic set of instructions in the ``README`` file at the top level of the `NCEPLIBS-external repository
-<https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v1.1.0>`_, and at the top level of the `NCEPLIBS repository
-<https://github.com/NOAA-EMC/NCEPLIBS/tree/ufs-v1.1.0>`_. It may still be a good idea to look at some of the platform-
+<https://github.com/NOAA-EMC/NCEPLIBS-external/tree/ufs-v2.0.0>`_, and at the top level of the `NCEPLIBS repository
+<https://github.com/NOAA-EMC/NCEPLIBS/tree/ufs-v2.0.0>`_. It may still be a good idea to look at some of the platform-
 and compiler-specific ``README`` files as a guide. Again, be sure to use the tag that corresponds to this version of the WM.
 
 The top-level ``README`` in the NCEPLIBS-external repository includes a troubleshooting section that may be helpful.
@@ -57,13 +57,13 @@ set up specifically for issues related to build dependencies.
 Downloading the Weather Model Code
 ==================================
 
-To clone the ufs-weather-model repository for this v1.1.0 release, execute the following commands:
+To clone the ufs-weather-model repository for this v2.0.0 release, execute the following commands:
 
 .. code-block:: console
 
   git clone https://github.com/ufs-community/ufs-weather-model.git ufs-weather-model
   cd ufs-weather-model
-  git checkout ufs-v1.1.0
+  git checkout ufs-v2.0.0
   git submodule update --init --recursive
 
 Compiling the model will take place within the `ufs-weather-model` directory you just created.
@@ -149,7 +149,7 @@ If you are not running on one of the pre-configured platforms, you will need to
 in a different way.
 
 If you used one of the platform- and compiler-specific ``README`` files in the ``doc/`` directory of NCEPLIBS-external
-to build the prerequisite libraries, there is a script in the ``NCEPLIBS-ufs-v1.1.0/bin`` directory called
+to build the prerequisite libraries, there is a script in the ``NCEPLIBS-ufs-v2.0.0/bin`` directory called
 ``setenv_nceplibs.sh`` that will set the NCEPLIBS-external variables for you.
 
 Of course, you can also set the values of these variables yourself if you know where the paths are on your system.

doc/UsersGuide/source/FAQ.rst

@@ -92,5 +92,68 @@ How do I select the file format for the model output (netCDF or NEMSIO)?
 ========================================================================
 In your run directory, there is a file named ``model_configure``.  Change the
 variable ``output_file`` to ``'netcdf'`` or ``'nemsio'``. The variable ``output_file``
-if only valid when the write component is activated by setting ``quilting`` to .true.
+is only valid when the write component is activated by setting ``quilting`` to .true.
 in the ``model_configure`` file.
+
+==============================================================
+How do I set the output history interval?
+==============================================================
+The interval at which output (history) files are written is controlled in two
+places, and depends on whether you are using the write component to generate your output files.
+:numref:`Table %s <OutputControl>` describes the relevant variables.  If the write_component is used, then the variables listed as *model_configure* are required.  It is however, also required that the settings in *input.nml* match those same settings in *model_configure*.  If these settings are inconsistent, then unpredictable output files and intervals may occur!
+
+.. _OutputControl:
+
+.. list-table:: *Namelist variables used to control the output file frequency.*
+   :widths: 15 10 10 30 
+   :header-rows: 1
+
+   * - Namelist variable
+     - Location
+     - Default Value
+     - Description
+   * - fdiag
+     - input.nml
+     - 0
+     - Array with dimension ``maxhr`` = 4096 listing the diagnostic output times (in hours) for the GFS physics.
+       This can either be a list of times after initialization, or an interval if only the first entry is
+       nonzero. The default setting of 0 will result in no outputs.
+   * - fhmax
+     - input.nml
+     - 384
+     - The maximal forecast time for output.
+   * - fhmaxhf
+     - input.nml
+     - 120
+     - The maximal forecast hour for high frequency output.
+   * - fhout
+     - input.nml
+     - 3
+     - Output frequency during forecast time from 0 to ``fhmax``, or from ``fhmaxhf`` to ``fhmax`` if ``fhmaxf>0``.
+   * - fhouthf
+     - input.nml
+     - 1
+     - The high frequency output frequency during the forecast time from 0 to ``fhmaxhf`` hour.
+   * - nfhmax_hf
+     - model_configure
+     - 0
+     - forecast length of high history file
+   * - nfhout_hf
+     - model_configure
+     - 1
+     - high history file output frequency
+   * - nfhout
+     - model_configure
+     - 3
+     - history file output frequency
+
+==============================================================
+How do I set the total number of tasks for my job?
+==============================================================
+The total number of MPI tasks used by the UFS Weather Model is a combination of compute and quilt tasks, and can be calculated using the following relationship:
+
+- total tasks = compute tasks + quilt tasks
+- compute tasks = x layout * y layout * number of tiles
+- quilt tasks = write_groups * write_tasks_per_group if quilting==.true.
+
+The layout and tiles settings are in ``input.nml``, and the quilt task settings are in ``model_configure``

doc/UsersGuide/source/InputNML.inc

@@ -12,9 +12,13 @@ The atmosphere model reads many parameters from a Fortran namelist file, named *
 several Fortran namelist records, some of which are always required, others of which are only used when selected
 physics options are chosen.
 
+The following link provides an in depth description of the namelist settings:
+
+https://dtcenter.ucar.edu/GMTB/v5.0.0/sci_doc/
+
 The following link describes the various physics-related namelist records:
 
-https://dtcenter.ucar.edu/GMTB/v4.1.0/sci_doc/CCPPsuite_nml_desp.html
+https://dtcenter.ucar.edu/GMTB/v5.0.0/sci_doc/CCPPsuite_nml_desp.html
 
 The following link describes the stochastic physics namelist records:
 

doc/UsersGuide/source/InputsOutputs.rst

@@ -11,12 +11,12 @@ Input files
 =============
 
 There are three types of files needed to execute a run: static datasets (*fix* files containing climatological
-information), files that depend on grid resolution and initial conditions, and model configuration files (such as namelists).
+information), files that depend on grid resolution, initial and boundary conditions, and model configuration files (such as namelists).
 
 ------------------------------------
 Static datasets (i.e., *fix files*)
 ------------------------------------
-The static input files are listed and described in :numref:`Table %s <FixFiles>`.
+The static input files for global configurations are listed and described in :numref:`Table %s <FixFiles>`.  Similar files are used for a regional grid but are grid specific and generated by pre-processing utilities.
 
 .. _FixFiles:
 
@@ -79,19 +79,19 @@ The static input files are listed and described in :numref:`Table %s <FixFiles>`
 ---------------------------------------------
 Grid description and initial condition files
 ---------------------------------------------
-The input files containing grid information and the initial conditions are listed and described in :numref:`Table %s <GridICFiles>`.
+The input files containing grid information and the initial conditions for global configurations are listed and described in :numref:`Table %s <GridICFiles>`.  The input files for a limited area model (LAM) configuration including grid information and initial and lateral boundary conditions are listed and described in :numref:`Table %s <RegionalGridICFiles>`.  Note that the regional grid is referred to as Tile 7 here, and are generated by several pre-processing utilities.
 
 .. _GridICFiles:
 
-.. list-table:: *Input files containing grid information and initial conditions*
+.. list-table:: *Input files containing grid information and initial conditions for global configurations*
    :widths: 35 50 15
    :header-rows: 1
 
    * - Filename
      - Description
      - Date-dependent
-   * - C96_grid.tile[1-6].nc
-     - C96 grid information for tiles 1-6
+   * - Cxx_grid.tile[1-6].nc
+     - Cxx grid information for tiles 1-6, where 'xx' is the grid number
      -
    * - gfs_ctrl.nc
      - NCEP NGGPS tracers, ak, and bk
@@ -109,6 +109,38 @@ The input files containing grid information and the initial conditions are liste
      - Surface properties for grid tiles 1-6
      - ✔
 
+
+.. _RegionalGridICFiles:
+
+.. list-table:: *Regional input files containing grid information and initial and lateral boundary conditions for regional configurations*
+   :widths: 35 50 15
+   :header-rows: 1
+
+   * - Filename
+     - Description
+     - Date-dependent
+   * - Cxx_grid.tile7.nc
+     - Cxx grid information for tile 7, where 'xx' is the grid number
+     -
+   * - gfs_ctrl.nc
+     - NCEP NGGPS tracers, ak, and bk
+     - ✔
+   * - gfs_bndy.tile7.HHH.nc
+     - Lateral boundary conditions at hour HHH
+     - ✔
+   * - gfs_data.tile7.nc
+     - Initial condition fields (ps, u, v, u, z, t, q, O3). May include spfo3, spfo, spf02 if multiple gases are used
+     - ✔
+   * - oro_data.tile7.nc
+     - Model terrain (topographic/orographic information) for grid tile 7
+     -
+   * - sfc_ctrl.nc
+     - Control parameters for surface input: forecast hour, date, number of soil levels
+     -
+   * - sfc_data.tile7.nc
+     - Surface properties for grid tile 7
+     - ✔
+
 ------------------------------------
 Model configuration files
 ------------------------------------
@@ -121,8 +153,9 @@ The configuration files used by the UFS Weather Model are listed here and descri
 - *suite_[suite_name].xml* (used only at build time)
 
 While the *input.nml* file is also a configuration file used by the UFS Weather Model, it is described in
-:numref:`Section %s <InputNML>`.
+:numref:`Section %s <InputNML>`.  The run-time configuration of model output fields is controlled by the combination of *diag_table* and *model_configure*, and is described in detail in :numref:`Section %s <OutputFiles>`.
 
+.. _diag_tableFile:
 
 *diag_table* file
 ------------------------------------
@@ -145,7 +178,7 @@ consists of six space-separated integers in the following format:  ``year month
 The File Description lines are used to specify the name of the file(s) to which the output will be written. They
 contain one or more sets of six required and five optional fields (optional fields are denoted by square brackets
 ``[ ]``).  The lines containing File Descriptions can be intermixed with the lines containing Field Descriptions as
-long as files are defined before fields that are to be written them.  File entries have the following format:
+long as files are defined before fields that are to be written to them.  File entries have the following format:
 
 .. code-block:: console
 
@@ -412,6 +445,8 @@ The lines in this file can be coded quite flexibly. Due to this flexibility, a n
 See ``FMS/field_manager/field_manager.F90`` for more information.
 
 
+.. _model_configureFile:
+
 *model_configure* file
 ------------------------------------
 
@@ -619,31 +654,46 @@ this is an atmosphere-only model, so this file is simple and does not need to be
 
 *The SDF (Suite Definition File) file*
 ---------------------------------------
-There are two SDFs currently supported: *suite_FV3_GFS_v15p2.xml* and *suite_FV3_GFS_v16beta.xml*.
+There are two SDFs currently supported for the UFS Medium Range Weather App configuration: *suite_FV3_GFS_v15p2.xml* and *suite_FV3_GFS_v16beta.xml*.
+
+There are two SDFs currently supported for the UFS Short Range Weather App configuration: *suite_FV3_GFS_v15p2.xml* and *suite_FV3_RRFS_v1alpha.xml*.
+
+Detailed descriptions of the supported suites can be found with the `CCPP v5.0.0 Scientific Documentation <https://dtcenter.ucar.edu/GMTB/v5.0.0/sci_doc/>`_.
+
 
 .. -------------------------------------------------------------------
 .. Include InputNML file describing the contents of the input.nml file
 .. -------------------------------------------------------------------
 
 .. include:: InputNML.inc
 
+.. _OutputFiles:
+
 =============
 Output files
 =============
 
-The following files are output when running *fv3.exe* in the default configuration (six files of each kind,
-corresponding to the six tiles of the model grid):
+The output files generated when running *fv3.exe* are defined in the *diag_tabl* file.  For the default global configuration, the following files are output (six files of each kind, corresponding to the six tiles of the model grid):
 
 - *atmos_4xdaily.tile[1-6].nc*
 - *atmos_static.tile[1-6].nc*
 - *sfcfHHH.nc*
 - *atmfHHH.nc*
 - *grid_spec.tile[1-6].nc*
 
-Note that the sfcf* and atmf* files are not output on the 6 tiles, but instead as a single global gaussian grid file.  The specifications of the output files (type, projection, etc) may be overridden in the *model_configure* input file.
+Note that the sfcf* and atmf* files are not output on the 6 tiles, but instead as a single global gaussian grid file.  The specifications of the output files (format, projection, etc) may be overridden in the *model_configure* input file, see :numref:`Section %s <model_configureFile>`.
+
+The regional configuration will generate similar output files, but the *tile[1-6]* is not included in the filename.
+
+Two files (*model_configure* and *diag_table*) control the output that is generated by the UFS Weather Model.  The output files that contain the model variables are written to a file as shown in the figure below.  The format of these output files is selected in *model_configure* as NetCDF or nemsio.  The information in these files may be remapped, augmented with derived variables, and converted to GRIB2 by the Unified Post Processor (UPP).  Model variables are listed in the *diag_table* in two groupings, *fv3_history* and *fv3_history2d*, as described in :numref:`Section %s <diag_tableFile>`.  The names of the files that contain these model variables are specified in the *model_configure* file. When *quilting* is set to *.true.* for the write component, the variables listed in the groups *fv3_history* and *fv3_history2d* are converted into the two output files named in the *model_configure* file, e.g. *atmfHHH.* and *sfcfHHH.*. The bases of the file names (*atm* and *sfc*) are specified in the *model_configure* file, and *HHH* refers to the forecast hour.
+
+.. image:: _static/fv3IO.png
+
+Standard output files are *logfHHH* (one per forecast hour), and out and err as specified by the job submission. ESMF may also produce log
+files (controlled by variable print_esmf in the *model_configure* file), called *PETnnn.ESMF_LogFile* (one per MPI task).
+
+Additional output files include: nemsusage.xml, a timing log file; time_stamp.out, contains the model init time; *RESTART/*nc*, files needed for restart runs.
 
-Standard output files are *logf???*, and out and err as specified by the job submission. ESMF may also produce log
-files (controlled by variable print_esmf in the *model_configure* file), called *PET???.ESMF_LogFile*.
 
 ==============================================================
 Additional Information about the FMS Diagnostic Manager
@@ -717,3 +767,9 @@ This release of the UFS Weather Model uses the following namelist:
    &diag_manager_nml
      prepend_date = .false.
    /
+
+==============================================================
+Additional Information about the Write Component
+==============================================================
+
+The UFS Weather Model is built using the Earth System Modeling Framework (ESMF).  As part of this framework, the output history files written by the model use an ESMF component, referred to as the *write component*.  This model component is configured with settings in the model_configure file, as described in :numref:`Section %s <model_configureFile>`.  By using the ESMF capabilities, the write component can generate output files in several different formats and several different map projections.  For example, a Gaussian global grid in NEMSIO format, or a native grid in NetCDF format.  The write component also runs on independent MPI tasks, and so the computational tasks can continue while the write component completes its work asynchronously.  The configuration of write component tasks, balanced with compute tasks, is part of the tuning for each specific application of the model (HPC, write frequency, i/o speed, model domain, etc).  For the global grid, if the write component is not selected (quilting=.false.), the FV3 code will write tiled output in the native projection in NetCDF format.  The regional grid requires the use of the write component.