From 99c2e4c8b227b79fcb9c31572893d3a3799171b3 Mon Sep 17 00:00:00 2001 From: Natalie Perlin Date: Fri, 27 May 2022 10:29:57 -0400 Subject: [PATCH 1/3] Updated the Introduction, build for MacOS --- docs/UsersGuide/source/BuildRunSRW.rst | 19 ++++------ docs/UsersGuide/source/Introduction.rst | 48 ++++++++++++------------- 2 files changed, 31 insertions(+), 36 deletions(-) diff --git a/docs/UsersGuide/source/BuildRunSRW.rst b/docs/UsersGuide/source/BuildRunSRW.rst index 658ff72a34..94e7deda10 100644 --- a/docs/UsersGuide/source/BuildRunSRW.rst +++ b/docs/UsersGuide/source/BuildRunSRW.rst @@ -238,6 +238,7 @@ Set Up the Workflow Environment .. attention:: If users successfully built the executables in :numref:`Step %s `, they should skip to step :numref:`Step %s `. + For the CMake steps on MacOS systems, follow the approach in :numref:`Step %s `. If the ``devbuild.sh`` approach failed, users need to set up their environment to run a workflow on their specific platform. First, users should make sure ``Lmod`` is the app used for loading modulefiles. This is the case on most Level 1 systems; however, on systems such as Gaea/Odin, the default modulefile loader is from Cray and must be switched to Lmod. For example, on Gaea, assuming a ``bash`` login shell, run: @@ -339,25 +340,20 @@ Then, users must source the Lmod setup file, just as they would on other systems source etc/lmod-setup.sh macos module use module load build_macos_gnu + export LDFLAGS="-L${MPI_ROOT}/lib" In a csh/tcsh shell, users would run ``source etc/lmod-setup.csh macos`` in place of the first line in the code above. .. note:: If you execute ``source etc/lmod-setup.sh`` on systems that don't need it, it will simply do a ``module purge``. -Additionally, for Option 1 systems, set the variable ``ENABLE_QUAD_PRECISION`` to ``OFF`` in line 35 of the ``$SRW/src/ufs-weather-model/FV3/atmos_cubed_sphere/CMakeLists.txt`` file. This change is optional if using Option 2 to build the SRW App. Using a text editor (e.g., vi, vim, emacs): +Additionally, for Option 1 systems, set the variable ``ENABLE_QUAD_PRECISION`` to ``OFF`` in ``$SRW/src/ufs-weather-model/FV3/atmos_cubed_sphere/CMakeLists.txt`` file. This change is optional if using Option 2 to build the SRW App. You could use a streamline editor `sed` to change it: .. code-block:: console - option(ENABLE_QUAD_PRECISION "Enable compiler definition -DENABLE_QUAD_PRECISION" OFF) - -An alternative way to make this change is using a `sed` (streamline editor). From the command line, users can run one of two commands (user's preference): - -.. code-block:: console - - sed -i -e 's/QUAD_PRECISION\" ON)/QUAD_PRECISION\" OFF)/' CMakeLists.txt - sed -i -e 's/bin\/sh/bin\/bash/g' *sh + sed -i .bak 's/QUAD_PRECISION\" ON)/QUAD_PRECISION\" OFF)/' $SRW/src/ufs-weather-model/FV3/atmos_cubed_sphere/CMakeLists.txt +Proceed to building executables using CMake in :numref:`Step %s ` .. _Data: @@ -908,7 +904,7 @@ For Option 1 (8 CPUs): # Architecture information WORKFLOW_MANAGER="none" - NCORES_PER_NODE=${NCORES_PER_NODE:-8} (Option 2: when 4 CPUs, set to 4) + NCORES_PER_NODE=${NCORES_PER_NODE:-8} SCHED=${SCHED:-"none"} # UFS SRW App specific paths @@ -925,9 +921,8 @@ For Option 1 (8 CPUs): RUN_CMD_UTILS="mpirun -np 4" RUN_CMD_FCST='mpirun -np ${PE_MEMBER01}' RUN_CMD_POST="mpirun -np 4" - PRE_TASK_CMDS='{ulimit -a;}' -The same settings can be used for Option 2, except that ``NCORES_PER_NODE=${NCORES_PER_NODE:-8}`` should be set to 4 instead of 8. +Using Option 2 with 4 CPUs requires ``NCORES_PER_NODE=${NCORES_PER_NODE:-4}`` in the above example. .. _MacActivateWFenv: diff --git a/docs/UsersGuide/source/Introduction.rst b/docs/UsersGuide/source/Introduction.rst index 9e1e86b3dc..6c6cfed135 100644 --- a/docs/UsersGuide/source/Introduction.rst +++ b/docs/UsersGuide/source/Introduction.rst @@ -51,7 +51,7 @@ The instructions in this documentation assume that users have certain background * Familiarity with LINUX/UNIX systems * Command line basics * System configuration knowledge (e.g., compilers, environment variables, paths, etc.) -* Numerical Weather Prediction +* Numerical Weather Prediction (concepts of parameterizations: physical, microphysical, convective) * Meteorology (particularly meteorology at the scales being predicted) .. @@ -73,12 +73,12 @@ The UFS SRW Application has been designed so that any sufficiently up-to-date ma * POSIX-compliant UNIX-style operating system -* >40 GB disk space +* >102 GB disk space - * 18 GB input data from GFS, RAP, and HRRR for "out-of-the-box" SRW App case described in :numref:`Chapter %s ` - * 6 GB for :term:`HPC-Stack` full installation - * 1 GB for ufs-srweather-app installation - * 11 GB for 48hr forecast on CONUS 25km domain + * 54 GB input data from GFS, RAP, and HRRR for "out-of-the-box" SRW App case described in :numref:`Chapter %s ` + * 8 GB for :term:`HPC-Stack` full installation + * 3 GB for ufs-srweather-app installation + * 17 GB for 12hr forecast on CONUS 25km domain * 4GB memory (CONUS 25km domain) @@ -88,7 +88,7 @@ The UFS SRW Application has been designed so that any sufficiently up-to-date ma * C and C++ compilers compatible with the Fortran compiler - * gcc v9+, ifort v18+, and clang v9+ (macOS, native Apple clang or LLVM clang) have been tested + * gcc v9+, ifort v18+, and clang v9+ (macOS, native Apple clang, LLVM clang, GNU) have been tested * Python v3.6+, including prerequisite packages ``jinja2``, ``pyyaml`` and ``f90nml`` @@ -96,41 +96,40 @@ The UFS SRW Application has been designed so that any sufficiently up-to-date ma * Perl 5 -* git v1.8+ +* git v2.12+ -.. - COMMENT: Should curl/wget/TIFF library also be required? These are listed as prerequisites for building HPC-Stack on generic MacOS/Linux +* curl + +* wget +.. + COMMENT: Should curl/wget/TIFF library also be required? These are listed as prerequisites for building HPC-Stack on generic MacOS/Linux | A: TIFF could be installed with the HPC-stack build; others are needed The following software is also required to run the SRW Application, but the :term:`HPC-Stack` (which contains the software libraries necessary for building and running the SRW App) can be configured to build these requirements: -* CMake v3.15+ +* CMake v3.20+ * MPI (MPICH, OpenMPI, or other implementation) - * Only **MPICH** can be built with HPC-Stack. Other options must be installed separately by the user. - -* Software libraries + * Only **MPICH** or **OpenMPI** can be built with HPC-Stack. Other implementations must be installed separately by the user. - * netCDF (C and Fortran libraries) - * HDF5 - * ESMF 8.2.0 - * Jasper - * libJPG - * libPNG - * zlib +.. + COMMENT: All the software libraries are removed from the list, as they are built by the HPC stack. For MacOS systems, some additional software is needed. It is recommended that users install this software using the `Homebrew `__ package manager for MacOS: +* brew install bash + + * OR: brew upgrade bash * brew install gcc@11 * brew install cmake * brew install make * brew install wget +* brew install lmod * brew install coreutils -* brew install pkg-config .. - COMMENT: Is this still accurate? It seems like we should delete the last one. And gcc@11 is basically the same as requiring fortran/C/C++ compilers, no? CMake is listed above. + COMMENT: Is this still accurate? It seems like we should delete the last one. And gcc@11 is basically the same as requiring fortran/C/C++ compilers, no? CMake is listed above. | A: updated bash requirements Optional but recommended prerequisites for all systems: @@ -138,8 +137,9 @@ Optional but recommended prerequisites for all systems: * Bash v4+ * Rocoto Workflow Management System (1.3.1) * Python packages ``scipy``, ``matplotlib``, ``pygrib``, ``cartopy``, and ``pillow`` for graphics -* Lmod +.. + COMMENT: Lmod is listed as required .. _ComponentsOverview: From 35d178fdb27e74ae8644c61795fc345928fe3160 Mon Sep 17 00:00:00 2001 From: Natalie Perlin <68030316+natalie-perlin@users.noreply.github.com> Date: Tue, 31 May 2022 13:39:24 -0400 Subject: [PATCH 2/3] Update Introduction.rst Some comments removed --- docs/UsersGuide/source/Introduction.rst | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/UsersGuide/source/Introduction.rst b/docs/UsersGuide/source/Introduction.rst index 6c6cfed135..e2ac98286f 100644 --- a/docs/UsersGuide/source/Introduction.rst +++ b/docs/UsersGuide/source/Introduction.rst @@ -14,9 +14,6 @@ The SRW App v1.0.0 citation is as follows and should be used when presenting res UFS Development Team. (2021, March 4). Unified Forecast System (UFS) Short-Range Weather (SRW) Application (Version v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.4534994 -.. - COMMENT: Update version numbers/citation for release! Also update release date for citation! - How to Use This Document ======================== @@ -113,9 +110,6 @@ The following software is also required to run the SRW Application, but the :ter * Only **MPICH** or **OpenMPI** can be built with HPC-Stack. Other implementations must be installed separately by the user. -.. - COMMENT: All the software libraries are removed from the list, as they are built by the HPC stack. - For MacOS systems, some additional software is needed. It is recommended that users install this software using the `Homebrew `__ package manager for MacOS: * brew install bash From 5982034f94be54c64e990079eb7a2f9643b057d7 Mon Sep 17 00:00:00 2001 From: Natalie Perlin <68030316+natalie-perlin@users.noreply.github.com> Date: Thu, 2 Jun 2022 09:56:48 -0400 Subject: [PATCH 3/3] Update Introduction.rst Updated data storage requirements for the SRW --- docs/UsersGuide/source/Introduction.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/UsersGuide/source/Introduction.rst b/docs/UsersGuide/source/Introduction.rst index e2ac98286f..208162a9e2 100644 --- a/docs/UsersGuide/source/Introduction.rst +++ b/docs/UsersGuide/source/Introduction.rst @@ -70,12 +70,14 @@ The UFS SRW Application has been designed so that any sufficiently up-to-date ma * POSIX-compliant UNIX-style operating system -* >102 GB disk space +* >82 GB disk space - * 54 GB input data from GFS, RAP, and HRRR for "out-of-the-box" SRW App case described in :numref:`Chapter %s ` + * 53 GB input data for a standard collection of global database, or "fix" data (topography, climatology, observational database), and boundary conditions for a short 12-h test forecast on CONUS 25km domain. See data download instructions in :numref:`Step %s `. * 8 GB for :term:`HPC-Stack` full installation * 3 GB for ufs-srweather-app installation - * 17 GB for 12hr forecast on CONUS 25km domain + * 1 GB boundary conditions for a short 12-h test forecast on CONUS 25km domain. See data download instructions in :numref:`Step %s ` + * 17 GB for a 12-h test forecast on CONUS 25km domain, with model output saved hourly, see :numref:`Step %s ` + * 4GB memory (CONUS 25km domain)