Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: Add plotting and template variable documentation #528

Merged
merged 17 commits into from
Jan 5, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/UsersGuide/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
sphinxcontrib-bibtex
sphinx_rtd_theme
sphinx_rtd_theme
12 changes: 3 additions & 9 deletions docs/UsersGuide/source/Components.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,6 @@ Among other techniques, MET provides the capability to compute standard verifica

METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (ESRL), and NOAA/Environmental Modeling Center (:term:`EMC`), and it is open to community contributions.


Visualization Example
=====================
A Python script is provided to create basic visualizations of the model output. The script is designed to output graphics in PNG format for several standard meteorological variables
when using the pre-defined :term:`CONUS` domain. A difference plotting script is also included to visually compare two runs for the same domain and resolution. These scripts are provided only as an example for users familiar with Python. They may be used to perform a visual check to verify that the application is producing reasonable results.

After running ``manage_externals/checkout_externals``, the visualization scripts will be available in the ``ufs-srweather-app/ush/Python`` directory. Usage information and instructions are described in :numref:`Chapter %s <Graphics>` and are also included at the top of each script.

Build System and Workflow
=========================

Expand All @@ -81,7 +73,9 @@ The SRW Application has a portable build system and a user-friendly, modular, an
An umbrella CMake-based build system is used for building the components necessary for running the end-to-end SRW Application, including the UFS Weather Model and the pre- and post-processing software. Additional libraries necessary for the application (e.g., :term:`NCEPLIBS-external` and :term:`NCEPLIBS`) are not included in the SRW Application build system but are available pre-built on pre-configured platforms. On other systems, they can be installed via the HPC-Stack (see :doc:`HPC-Stack Documentation <hpc-stack:index>`). There is a small set of system libraries and utilities that are assumed to be present on the target computer: the CMake build software; a Fortran, C, and C++ compiler; and an :term:`MPI` library.

Once built, the provided experiment generator script can be used to create a Rocoto-based
workflow file that will run each task in the system in the proper sequence (see :numref:`Chapter %s <RocotoInfo>` or the `Rocoto documentation <https://github.com/christopherwharrop/rocoto/wiki/Documentation>`__ for more information on Rocoto). If Rocoto and/or a batch system is not present on the available platform, the individual components can be run in a stand-alone, command line fashion with provided run scripts. The generated namelist for the atmospheric model can be modified in order to vary settings such as forecast starting and ending dates, forecast length hours, the :term:`CCPP` physics suite, integration time step, history file output frequency, and more. It also allows for configuration of other elements of the workflow; for example, users can choose whether to run some or all of the pre-processing, forecast model, and post-processing steps.
workflow file that will run each task in the system in the proper sequence (see :numref:`Chapter %s <RocotoInfo>` or the `Rocoto documentation <https://github.com/christopherwharrop/rocoto/wiki/Documentation>`__ for more information on Rocoto). If Rocoto and/or a batch system is not present on the available platform, the individual components can be run in a stand-alone, command line fashion with provided run scripts. The generated namelist for the atmospheric model can be modified in order to vary settings such as forecast starting and ending dates, forecast length hours, the :term:`CCPP` physics suite, integration time step, history file output frequency, and more. It also allows for configuration of other elements of the workflow; for example, users can choose whether to run some or all of the pre-processing, forecast model, and post-processing steps.

An optional Python plotting task is also included to create basic visualizations of the model output. The task outputs graphics in PNG format for several standard meteorological variables on the pre-defined :term:`CONUS` domain. A difference plotting option is also included to visually compare two runs for the same domain and resolution. These plots may be used to perform a visual check to verify that the application is producing reasonable results. Configuration instructions are provided in :numref:`Section %s <PlotOutput>`.

The latest SRW Application release has been tested on a variety of platforms widely used by researchers, such as the NOAA Research and Development High-Performance Computing Systems (RDHPCS), including Hera, Orion, and Jet; the National Center for Atmospheric Research (:term:`NCAR`) Cheyenne system; and generic Linux and MacOS systems using Intel and GNU compilers. Four `levels of support <https://github.com/ufs-community/ufs-srweather-app/wiki/Supported-Platforms-and-Compilers>`__ have been defined for the SRW Application, including pre-configured (Level 1), configurable (Level 2), limited-test (Level 3), and build-only (Level 4) platforms. Each level is further described below.

Expand Down
52 changes: 52 additions & 0 deletions docs/UsersGuide/source/ConfigWorkflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,12 @@ Verification Tasks

.. COMMENT: COMMENT: Define "ensemble-stat verification for gridded data," "ensemble point verification," "ensemble-stat point verification," and "point verification of ensemble-stat output"?

Plotting Task
----------------

``RUN_TASK_PLOT_ALLVARS:`` (Default: false)
Flag that determines whether to run python plotting scripts.

.. _make-grid:

MAKE_GRID Configuration Parameters
Expand Down Expand Up @@ -1134,6 +1140,9 @@ These parameters are associated with the fixed (i.e., static) files. On `Level 1
``FIXlut``: (Default: "")
System directory where the lookup tables for optics properties are located.

``FIXshp``: (Default: "")
System directory where the graphics shapefiles are located. On Level 1 systems, these are set within the machine files. Users on other systems will need to provide the path to the directory that contains the *Natural Earth* shapfiles.
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

``TOPO_DIR``: (Default: "")
The location on disk of the static input files used by the ``make_orog`` task (i.e., ``orog.x`` and ``shave.x``). Can be the same as ``FIXgsm``.

Expand Down Expand Up @@ -1726,6 +1735,49 @@ Non-default parameters for the ``run_enspointvx_prob`` task are set in the ``tas
``MAXTRIES_VX_ENSPOINT_PROB``: (Default: 1)
Maximum number of times to attempt the task.

.. _PlotVars:

PLOT_ALLVARS Configuration Parameters
========================================

Non-default parameters for the ``plot_allvars`` task are set in the ``task_plot_allvars:`` section of the ``config.yaml`` file.

Basic Task Parameters
--------------------------

For each workflow task, certain parameter values must be passed to the job scheduler (e.g., Slurm), which submits a job for the task. Typically, users do not need to adjust the default values.

``PLOT_ALLVARS_TN``: (Default: "plot_allvars")
Set the name of this Rocoto workflow task. Users typically do not need to change this value.

``NNODES_PLOT_ALLVARS``: (Default: 1)
Number of nodes to use for the job.

``PPN_PLOT_ALLVARS``: (Default: 24)
Number of :term:`MPI` processes per node.

``WTIME_PLOT_ALLVARS``: (Default: 01:00:00)
Maximum time for the task to complete.

``MAXTRIES_PLOT_ALLVARS``: (Default: 1)
Maximum number of times to attempt the task.

Additional Parameters
------------------------

Typially, the following parameters must be set explicitly by the user in the configuration file (``config.yaml``) when executing the plotting tasks.
gspetro-NOAA marked this conversation as resolved.
Show resolved Hide resolved

``COMOUT_REF``: (Default: "")
The directory where the GRIB2 files from post-processing are located. In *community* mode (i.e., when ``RUN_ENVIR: "community"``), this directory will correspond to the location in the experiment directory where the post-processed output can be found (e.g., ``$EXPTDIR/$DATE_FIRST_CYCL/postprd``). In *nco* mode, this directory should be set to the location of the COMOUT directory and end with ``$PDY/$cyc``.

``PLOT_FCST_START``: (Default: 0)
The starting forecast hour for the plotting task. For example, if a forecast starts at 18h/18z, this is considered the 0th forecast hour, so "starting forecast hour" should be 0, not 18. If a forecast starts at 18h/18z, but the user only wants plots from the 6th forecast hour on, "starting forecast hour" should be 6.

``PLOT_FCST_INC``: (Default: 3)
Forecast hour increment for the plotting task. This may be the same as ``INCR_CYCL_FREQ``, or it may be a multiple of ``INCR_CYCL_FREQ``. For example, if ``INCR_CYCL_FREQ`` is set to 3, there will be forecast output every three hours for the duration of the forecast. If the user wants plots of all of this output, they should set ``PLOT_FCST_INC: 3``. If the user only wants plots for some of the output (e.g., every 6 hours), they should set ``PLOT_FCST_INC: 6``. However, there must be forecast output available at the designated increments to produce the plots. In this example, setting ``PLOT_FCST_INC: 7`` would produce an error because there is only forecast output available for hours 3, 6, 9, ..., etc.

``PLOT_FCST_END``: (Default: "")
The last forecast hour for the plotting task. For example, if a forecast run for 24 hours, and the user wants plots for each available hour of forecast output, they should set ``PLOT_FCST_END: 24``. If the user only wants plots from the first 12 hours of the forecast, the "last forecast hour" should be 12.

Global Configuration Parameters
===================================
Expand Down
4 changes: 0 additions & 4 deletions docs/UsersGuide/source/ContainerQuickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,3 @@ New Experiment
===============

To run a new experiment in the container at a later time, users will need to rerun the commands in :numref:`Section %s <SetUpPythonEnvC>` to reactivate the regional workflow. Then, users can configure a new experiment by updating the environment variables in ``config.yaml`` to reflect the desired experiment configuration. Basic instructions appear in :numref:`Section %s <SetUpConfigFileC>` above, and detailed instructions can be viewed in :numref:`Section %s <UserSpecificConfig>`. After adjusting the configuration file, regenerate the experiment by running ``./generate_FV3LAM_wflow.py``.

Plot the Output
===============
Two python scripts are provided to generate plots from the FV3-LAM post-processed GRIB2 output. Information on how to generate the graphics can be found in :numref:`Chapter %s <Graphics>`.
Loading