From b526dcc07d273ea740bea0b5390fa04f11bb7534 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Fri, 11 Aug 2023 12:13:54 -0400 Subject: [PATCH 01/11] Add formatting updates to docs/source/Regridding.rst docs/source/Regridding.rst - Format text that it falls within 80 columns or less - Add space between bulleted items - Use :file: instead of :literal: for file names - Add a note that gridspec and sparselt are present in the GCPy environment file starting with 14.1.0 Signed-off-by: Bob Yantosca --- docs/source/Regridding.rst | 300 +++++++++++++++++++++---------------- 1 file changed, 175 insertions(+), 125 deletions(-) diff --git a/docs/source/Regridding.rst b/docs/source/Regridding.rst index 9bed1d93..cb4b23ea 100644 --- a/docs/source/Regridding.rst +++ b/docs/source/Regridding.rst @@ -8,21 +8,26 @@ Regridding ########## -This page describes the regridding capabilities of GCPy. GCPy -currently supports regridding of data from GEOS-Chem restarts and -output NetCDF files. Regridding is supported across any horizontal -resolution and any grid type available in GEOS-Chem, including lat/lon -(global or non-global), global standard cubed-sphere, and global -stretched-grid. GCPy also supports arbitrary vertical regridding -across different vertical resolutions. - -Regridding with GCPy is currently undergoing an overhaul. As of the current -release, regridding is split into two different categories: (1) regridding -between lat-lon grids using regridding weights computed on the fly by GCPy, -and (2) regridding either lat-lon or cubed-sphere using regridding weights -computed as a preprocessing step. The latter method may be used for -creating GCHP standard grid and stretched grid restart files from -either GCHP or GC-Classic restart files. +This page describes the regridding capabilities of +:program:`GCPy`. GCPy currently supports regridding of data from +GEOS-Chem restarts and output NetCDF files. Regridding is supported +across any horizontal resolution and any grid type available in +GEOS-Chem, including lat/lon (global or non-global), global standard +cubed-sphere, and global stretched-grid. GCPy also supports arbitrary +vertical regridding across different vertical resolutions. + +Regridding with GCPy is currently undergoing an overhaul. As of the +current release, regridding is split into two different +categories: + +#. Regridding between lat-lon grids using regridding weights computed + on the fly by GCPy, and +#. Regridding either lat-lon or cubed-sphere using regridding weights + computed as a preprocessing step. + +The latter method may be used for creating GCHP standard grid +and stretched grid restart files from either GCHP or GC-Classic +restart files. .. _regrid-classic: @@ -31,7 +36,7 @@ Using Online Regridding Weights =============================== You can regrid existing GEOS-Chem Classic restart or output diagnostic files -between lat/lon resolutions using GCPy function:code:`gcpy.file_regrid`. +between lat/lon resolutions using GCPy function :code:`gcpy.file_regrid`. :code:`gcpy.file_regrid` can either be called directly from the command line using :code:`python -m gcpy.file_regrid` or as a function (:code:`gcpy.file_regrid.file_regrid()`) from a Python script or interpreter. @@ -78,17 +83,17 @@ gcpy.file_regrid optional arguments: Example: -------- -There is now only one grid format supported for regridding files using the -:code:`gcpy.file_regrid` method: :literal:`classic`. You must specify -:literal:`classic` as the value of both :code:`dim_format_in` and -:code:`dim_format_out`, as well as specifying a resolution as the value of -:code:`ll_res_out`. +There is now only one grid format supported for regridding files using +the :code:`gcpy.file_regrid` method: :literal:`classic`. You must +specify :literal:`classic` as the value of both :code:`dim_format_in` +and :code:`dim_format_out`, as well as specifying a resolution as the +value of :code:`ll_res_out`. As stated previously, you can either call :code:`file_regrid.file_regrid()` directly or call it from the command -line using :code:`python -m gcpy.file_regrid ARGS`. An example command -line call (separated by line for readability) for regridding a 2x2.5 lat/lon -restart file to a 4x5 lat/lon grid looks like: +line using :code:`python -m gcpy.file_regrid ARGS`. An example +command line call (separated by line for readability) for regridding a +2x2.5 lat/lon restart file to a 4x5 lat/lon grid looks like: .. code-block:: @@ -105,17 +110,19 @@ restart file to a 4x5 lat/lon grid looks like: Using Offline Regridding Weights ================================ -This approach requires generating regridding weights using python packages -`gridspec `_ and -`sparselt `_. -Regridding with :literal:`GCPy`, :literal:`gridspec` and :literal:`sparselt` is a three stage -process: +This approach requires generating regridding weights using python +packages `gridspec `_ and +`sparselt `_. Regridding with +:literal:`GCPy`, :literal:`gridspec` and :literal:`sparselt` is a +three stage process: #. Create grid specifications for the source and target grids using - :literal:`gridspec` + :literal:`gridspec` |br| + |br| #. Create regridding weights for the transformation using - :literal:`ESMF_RegridWeightGen` + :literal:`ESMF_RegridWeightGen` |br| + |br| #. Run the regridding operation using the :code:`regrid_restart_file` submodule of GCPy @@ -125,11 +132,17 @@ process: Python environment for gridspec and sparselt -------------------------------------------- -Until GCPy contains a complete regridding implementation that works for all -GEOS-Chem grid formats, we recommend that you create a small -`conda `_ environment in which to generated regridding weights +**NOTE: Starting with GCPy 1.4.0 (currently under development),** +:code:`gridspec` **and** :code:`sparselt` **are included in the +standard environment file.** -The following conda `environment file `_ +Until GCPy contains a complete regridding implementation that works +for all GEOS-Chem grid formats, we recommend that you create a small +`mamba `_ environment in which to +generate regridding weights. + +The following `environment file +`_ will get you set up with an environment for regridding with :literal:`gridspec` and :literal:`sparselt`: @@ -151,17 +164,20 @@ will get you set up with an environment for regridding with .. tip:: A copy of the above environment file is maintained at the path - :file:`docs/environment/gchp_regridding.yml`. You can create the environment using command - :code:`conda env create -f /path/to/gchp_regridding.yml`. + :file:`docs/environment/gchp_regridding.yml`. You can create the + environment using command :code:`mamba env create -f + /path/to/gchp_regridding.yml`. -After installing and switching to this new conda environment, you should have -the :literal:`gridspec` commands available to you at the command line. +After installing and switching to this new environment, you +should have the :literal:`gridspec` commands available to you at the +command line. gcpy.regrid_restart_file required arguments: -------------------------------------------- -There are three arguments required by the GCPy function :literal:`regrid_restart_file`: +There are three arguments required by the GCPy function +:literal:`regrid_restart_file`: .. option:: file_to_regrid : str @@ -169,65 +185,77 @@ There are three arguments required by the GCPy function :literal:`regrid_restart .. option:: regridding_weights_file : str - Regridding weights to be used in the regridding transformation, generated - by :literal:`ESMF_RegridWeightGen` + Regridding weights to be used in the regridding transformation, + generated by :literal:`ESMF_RegridWeightGen` .. option:: template_file : str - The GC-Classic or GCHP restart file to use as a template for the regridded - restart file. Attributes, dimensions, and variables for the output file will be - taken from this template. This may be the same file as the file you are regridding! + The GC-Classic or GCHP restart file to use as a template for the + regridded restart file. Attributes, dimensions, and variables + for the output file will be taken from this template. This may + be the same file as the file you are regridding! gcpy.regrid_restart_file optional arguments: -------------------------------------------- -There are four optional arguments, all of which are for regridded to a stretched cubed-sphere grid. +There are four optional arguments, all of which are for regridded to a +stretched cubed-sphere grid. .. option:: --stretched-grid : switch - A switch to indicate that the target grid is a stretched cubed-sphere grid + A switch to indicate that the target grid is a stretched + cubed-sphere grid. .. option:: --stretch-factor : float - The grid stretching factor for the target stretched grid. Only takes - effect when :code:`--stretched-grid` is set. See the - `GCHP documentation `_ - for more information. Make sure this value exactly matches the value you - plan to use in GCHP configuration file :file:`setCommonRunSettings.sh`. + The grid stretching factor for the target stretched grid. Only + takes effect when :code:`--stretched-grid` is set. See the + `GCHP documentation + `_ + for more information. Make sure this value exactly matches the + value you plan to use in GCHP configuration file + :file:`setCommonRunSettings.sh`. .. option:: --target-latitude : float - The latitude of the centre point for stretching the target grid. Only - takes effect when :code:`--stretched-grid` is set. See the - `GCHP documentation `_ - for more information. Make sure this value exactly matches the value you - plan to use in GCHP configuration file :file:`setCommonRunSettings.sh`. + The latitude of the centre point for stretching the target + grid. Only takes effect when :code:`--stretched-grid` is + set. See the `GCHP documentation + `_ + for more information. Make sure this value exactly matches the + value you plan to use in GCHP configuration file + :file:`setCommonRunSettings.sh`. .. option:: --target-longitude : float - The longitude of the centre point for stretching the target grid. Only - takes effect when :code:`--stretched-grid` is set. See the - `GCHP documentation `_ - for more information. Make sure this value exactly matches the value you - plan to use in GCHP configuration file :file:`setCommonRunSettings.sh`. + The longitude of the centre point for stretching the target + grid. Only takes effect when :code:`--stretched-grid` is + set. See the `GCHP documentation `_ + for more information. Make sure this value exactly matches the + value you plan to use in GCHP configuration file + :file:`setCommonRunSettings.sh`. .. _regrid-gchp-procedure: Example 1: Standard Lat-Lon to Cubed-Sphere Regridding -------------------------------------------- +------------------------------------------------------ This example will show regridding a GC-Classic 4x5 restart file to a GCHP c24 restart file. -#. Load the :literal:`gchp_regridding` python environment. +#. Load the :literal:`gchp_regridding` python environment. |br| + |br| -#. Create a lat-lon source grid specification using :code:`gridspec-create`. +#. Create a lat-lon source grid specification using + :code:`gridspec-create`. .. code-block:: console $ gridspec-create latlon 46x72 - This will produce 1 file - :literal:`regular_lat_lon_46x72.nc`. + This will produce 1 file - + :file:`regular_lat_lon_46x72.nc`. |br| + |br| #. Create a target grid specification using :code:`gridspec-create`. @@ -235,8 +263,9 @@ GCHP c24 restart file. $ gridspec-create gcs 23 - Again, this will produce 7 files - :literal:`c24_gridspec` and - :literal:`c24.tile[1-6].nc` + Again, this will produce 7 files - :file:`c24_gridspec` and + :file:`c24.tile[1-6].nc` |br| + |br| #. Create the regridding weights for the regridding transformation using :code:`ESMF_RegridWeightGen`. @@ -249,16 +278,19 @@ GCHP c24 restart file. --method conserve \ --weight 46x72_to_c24_weights.nc - This will produce a log file, :literal:`PET0.RegridWeightGen.Log`, and our - regridding weights, :literal:`46x72_to_c24_weights.nc` + This will produce a log file, :file:`PET0.RegridWeightGen.Log`, and our + regridding weights, :file:`46x72_to_c24_weights.nc` |br| + |br| -#. Switch to your GCPy python environment. +#. Switch to your GCPy python environment. |br| + |br| -#. Use the grid weights produced in previous steps to complete the regridding. The first file - listed in the command contains the data you wish to regrid and so is a GC-Classic - restart file. The second file is a template file for the regridded file, containing - all attributes, dimensions, and variables that you would like to have in the GCHP - restart file. +#. Use the grid weights produced in previous steps to complete the + regridding. The first file listed in the command contains the data + you wish to regrid and so is a GC-Classic restart file. The second + file is a template file for the regridded file, containing all + attributes, dimensions, and variables that you would like to have + in the GCHP restart file. .. code-block:: console @@ -267,17 +299,19 @@ GCHP c24 restart file. 46x72_to_c24_weights.nc \ GEOSChem.fullchem.Restart.20190701_0000z.c24.old_version.nc4 - This will produce a single file, :literal:`new_restart_file.nc`, regridded - from 4x5 to c24, that you can rename and use as you please. + This will produce a single file, :file:`new_restart_file.nc`, + regridded from 4x5 to c24, that you can rename and use as you + please. Example 2: Standard Cubed-Sphere to Cubed-Sphere Regridding ------------------------------------------------- +----------------------------------------------------------- We will use the example of regridding the out-of-the-box -:literal:`GEOSChem.Restart.20190701_0000z.c48.nc4` restart file from C48 to -C60 to demonstrate the standard cubed-sphere regridding process: +:file:`GEOSChem.Restart.20190701_0000z.c48.nc4` restart file from +C48 to C60 to demonstrate the standard cubed-sphere regridding process: -#. Load the :literal:`gchp_regridding` python environment. +#. Load the :literal:`gchp_regridding` python environment. |br| + |br| #. Create a source grid specification using :code:`gridspec-create`. @@ -286,7 +320,8 @@ C60 to demonstrate the standard cubed-sphere regridding process: $ gridspec-create gcs 48 This will produce 7 files - :literal:`c48_gridspec.nc` and - :literal:`c48.tile[1-6].nc` + :literal:`c48.tile[1-6].nc` |br| + |br| #. Create a target grid specification using :code:`gridspec-create`. @@ -295,10 +330,11 @@ C60 to demonstrate the standard cubed-sphere regridding process: $ gridspec-create gcs 60 Again, this will produce 7 files - :literal:`c60_gridspec` and - :literal:`c60.tile[1-6].nc` + :literal:`c60.tile[1-6].nc` |br| + |br| -#. Create the regridding weights for the regridding transformation using - :code:`ESMF_RegridWeightGen`. +#. Create the regridding weights for the regridding transformation + using :code:`ESMF_RegridWeightGen`. .. code-block:: console @@ -308,10 +344,12 @@ C60 to demonstrate the standard cubed-sphere regridding process: --method conserve \ --weight c48_to_c60_weights.nc - This will produce a log file, :literal:`PET0.RegridWeightGen.Log`, and our - regridding weights, :literal:`c48_to_c60_weights.nc` + This will produce a log file, :file:`PET0.RegridWeightGen.Log`, + and our regridding weights, :file:`c48_to_c60_weights.nc` |br| + |br| -#. Switch to your GCPy python environment. +#. Switch to your GCPy python environment. |br| + |br| #. Use the grid weights produced in earlier steps to complete the regridding. @@ -322,19 +360,22 @@ C60 to demonstrate the standard cubed-sphere regridding process: c48_to_c60_weights.nc \ GEOSChem.Restart.20190701_0000z.c48.nc4 - This will produce a single file, :literal:`new_restart_file.nc`, regridded - from C48 to C60, that you can rename and use as you please. + This will produce a single file, :file:`new_restart_file.nc`, + regridded from C48 to C60, that you can rename and use as you + please. Example 3: Standard to Stretched Cubed-Sphere Regridding ---------------------------------------------- +-------------------------------------------------------- -This example regrids the out-of-the-box c48 restart file -(:literal:`GEOSChem.Restart.20190701_0000z.c48.nc4`) from a standard cubed-sphere -grid to a stretched grid. The base resolution will remain the same at c48. The -regridded file will have a stretch factor of 4.0 over Bermuda which means a -regional grid resolution of c196 (4 times 48) in that area. +This example regrids the out-of-the-box c48 restart file +(:file:`GEOSChem.Restart.20190701_0000z.c48.nc4`) from a standard +cubed-sphere grid to a stretched grid. The base resolution will remain +the same at c48. The regridded file will have a stretch factor of 4.0 +over Bermuda which means a regional grid resolution of c196 (4 +times 48) in that area. -#. Load the :literal:`gchp_regridding` python environment. +#. Load the :literal:`gchp_regridding` python environment. |br| + |br| #. Create a source grid specification using :code:`gridspec-create`. @@ -342,26 +383,31 @@ regional grid resolution of c196 (4 times 48) in that area. $ gridspec-create gcs 48 - This will produce 7 files - :literal:`c48_gridspec.nc` and - :literal:`c48.tile[1-6].nc` + This will produce 7 files - :file:`c48_gridspec.nc` and + :file:`c48.tile[1-6].nc` |br| + |br| -#. Create a target grid specification using :code:`gridspec-create`. This will be for the stretched grid. +#. Create a target grid specification using :code:`gridspec-create`. + This will be for the stretched grid. .. code-block:: console $ gridspec-create sgcs 48 -s 4.0 -t 32.0 -64.0 - Here, the :code:`-s` option denotes the stretch factor and the :code:`-t` - option denotes the latitude / longitude of the centre point of the grid - stretch. + Here, the :code:`-s` option denotes the stretch factor and the + :code:`-t` option denotes the latitude / longitude of the centre + point of the grid stretch. - Again, this will produce 7 files - :literal:`c48_..._gridspec.nc` and - :literal:`c48_..._tile[1-6].nc`, where :literal:`...` denotes randomly - generated characters. Be sure to look for these since you will need them in the next step. + Again, this will produce 7 files - :file:`c48_..._gridspec.nc` and + :file:`c48_..._tile[1-6].nc`, where :file:`...` denotes randomly + generated characters. Be sure to look for these since you will need + them in the next step. |br| + |br| -#. Create the regridding weights for the regridding transformation using - :code:`ESMF_RegridWeightGen`, replacing :literal:`c48_..._gridspec.nc` - with the actual name of the file created in the previous step. An example is shown below. +#. Create the regridding weights for the regridding transformation + using :code:`ESMF_RegridWeightGen`, replacing + :file:`c48_..._gridspec.nc` with the actual name of the file + created in the previous step. An example is shown below. .. code-block:: console @@ -371,12 +417,15 @@ regional grid resolution of c196 (4 times 48) in that area. --method conserve \ --weight c48_to_c48_stretched_weights.nc - This will produce a log file, :literal:`PET0.RegridWeightGen.Log`, and our - regridding weights, :literal:`c48_to_c48_stretched_weights.nc` + This will produce a log file, :file:`PET0.RegridWeightGen.Log`, and our + regridding weights, :file:`c48_to_c48_stretched_weights.nc` |br| + |br| -#. Switch to your GCPy python environment. +#. Switch to your GCPy python environment. |br| + |br| -#. Use the grid weights produced in earlier steps to complete the regridding. +#. Use the grid weights produced in earlier steps to complete the + regridding. .. code-block:: console @@ -389,22 +438,23 @@ regional grid resolution of c196 (4 times 48) in that area. c48_to_c48_stretched_weights.nc \ GEOSChem.Restart.20190701_0000z.c48.nc4 - This will produce a single file, :literal:`new_restart_file.nc`, regridded - from C48 standard to C48 stretched with a stretch factor of 4.0 over 32.0N, -64.0E, that you - can rename and use as you please. It is generally a good idea to rename the - file to include the grid resolution, stretch factor, and target lat/lon for - easy reference. You can copy it somewhere to keep long-term and link to it from - the GCHP Restarts subdirectory in the run directory. + This will produce a single file, :literal:`new_restart_file.nc`, + regridded from C48 standard to C48 stretched with a stretch factor + of 4.0 over 32.0N, -64.0E, that you can rename and use as you + please. It is generally a good idea to rename the file to include + the grid resolution, stretch factor, and target lat/lon for easy + reference. You can copy it somewhere to keep long-term and link to + it from the GCHP Restarts subdirectory in the run directory. .. code-block:: console $ mv new_restart_file.nc GEOSChem.Restart.20190701_0000z.c120.s4_32N_64E.nc - You can also easily reference the file's stretch parameters by looking - at the global attributes in the file. When using the file as a restart - file in GCHP make sure that you use the exact same parameters in both - the file's global attributes and GCHP configuration file - :file:`setCommonRunSettings.sh`. + You can also easily reference the file's stretch parameters by + looking at the global attributes in the file. When using the + file as a restart file in GCHP make sure that you use the exact + same parameters in both the file's global attributes and GCHP + configuration file :file:`setCommonRunSettings.sh`. .. _regrid-plot: From 24acfe55d562030ab37159f1c7afab60e3a7e8e2 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Fri, 11 Aug 2023 17:18:47 -0400 Subject: [PATCH 02/11] Update GCPy installation instructions docs/source/Getting-Started-with-GCPy.rst - Updated for installation with MicroMamba Signed-off-by: Bob Yantosca --- docs/source/Getting-Started-with-GCPy.rst | 324 +++++++++------------- 1 file changed, 126 insertions(+), 198 deletions(-) diff --git a/docs/source/Getting-Started-with-GCPy.rst b/docs/source/Getting-Started-with-GCPy.rst index 71e57579..8c64523b 100644 --- a/docs/source/Getting-Started-with-GCPy.rst +++ b/docs/source/Getting-Started-with-GCPy.rst @@ -30,255 +30,172 @@ receive an error message if you attempt to use GCPy on Windows. computer. The only essential software you need before installing GCPy is a -distribution of the :program:`Conda` package manager. This is used to -create a Python environment for GCPy containing all of its software -dependences, including what version of Python you use. You must -using GCPy with Python version 3.9. +distribution of the :program:`Mamba` package manager. :program:`Mamba` +is a drop-in replacement for the widely-used :program:`Conda` +package manager. You will use :program:`Mamba` to create a Python +environment for GCPy, which contains a Python version (currently +3.9.6) plus specific versions of various dependent packages. -You can check if you already have Conda installed by running the -following command: +Check if you already have :program:`Mamba` by using this command: .. code-block:: console - $ conda --version + $ mamba --version -.. attention:: +If :program:`Mamba` has already been installed on your system, **skip +ahead to the** :ref:`gcpy_install` **section**. - You must use Conda 4.12.0 or earlier to install GCPy and its - dependencies. Newer versions of Conda than this will install - Python package versions that are incompatible with GCPy. See - :ref:`Installing Conda 4.12.0 with Miniconda ` - below. +.. _mamba_install: - In the future we hope to be able to resolve this installation issue - so that you can use the latest Conda version. +============================== +Installing Mamba as MicroMamba +============================== -If Conda is not already installed, you must use :program:`Miniconda` -to install Conda 4.12.0. Miniconda is a minimal installer for Conda -that generally includes many fewer packages in the base environment -than are available for download. This provides a lightweight Conda -installation from which you can create custom Python environments with -whatever Python packages you wish to use, including an environment -with GCPy dependencies. +:program:`MicroMamba` is a minimal :program:`Mamba` version that is +easier to install than the full :program:`MambaForge` distribution. +Follow the installation steps listed below: -.. _conda412_install: +#. Download and run the :program:`MicroMamba` executable. -============================================ -Steps to install Conda 4.12.0 with Miniconda -============================================ - -If you already have a Conda version prior to 4.12.0 installed on your -system, you may skip this step and proceed to the section entitled -:ref:`gcpy_install`. - -If you need to install Conda 4.12.0, follow these steps: - -#. Download the Miniconda installer script for your operating system - as shown below. The script will install Conda version 4.12.0 using - Python 3.9. - - **Linux (x86_64 CPUs)** - - .. code-block:: console - - $ wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-Linux-x86_64.sh - - **MacOS (M1 CPUs)** - - .. code-block:: console - - $ wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-MacOSX-arm64.sh - - **MacOS (x86_64 CPUs)** + Execute the command listed below that is relevant to your operating + system. (You can cut-and-paste the command into a terminal window.) .. code-block:: console - $ wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-MacOSX-x86_64.sh + # Linux Intel (x86_64), including Windows Subsystem for Linux + $ curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba - .. tip:: + # MacOS Intel (x86_64): + $ curl -Ls https://micro.mamba.pm/api/micromamba/osx-64/latest | tar -xvj bin/micromamba - If you do not have :program:`wget` installed on MacOS, you can - download it with the :program:`Homebrew` package manager: + # MacOS Silicon/M1, Silicon/M2 (ARM64): + $ curl -Ls https://micro.mamba.pm/api/micromamba/osx-arm64/latest | tar -xvj bin/micromamba - .. code-block:: - - $ brew install wget - - In the steps that follow, we will walk through installation using - the Linux installer script. The steps are the same for MacOS; just - substitute the appropriate MacOS script name for the Linux script - name in steps 2 and 3 below. |br| + This will download the :program:`MicroMamba` executable file into + following path: :file:`$HOME/bin/micromamba`. If you do not + already have a :file:`$HOME/bin` folder, it will be created for + you. |br| |br| +#. Tell your shell where it can find the :program:`MicroMamba` executable. -#. Change the permission of the Miniconda installer script so that it - is executable: + If you have not done so already, add the following + line to your :file:`$HOME/.bashrc` startup script: - .. code-block:: console + .. code-block:: bash - $ chmod 755 Miniconda3-py39_4.12.0-Linux-x86_64.sh + export PATH="$HOME/bin:$PATH" - |br| + .. note:: + + On some computer systems, it is preferred that users place + modifications not into the :file:`$HOME/.bashrc` file, but + instead to a different script (e.g. :file:`$HOME/.bash_aliases`) + that is executed by :file:`$HOME/.bashrc`. Ask your system + administrator for more information pertaining to your particular + setup. -#. Run the Miniconda installer script. + Apply the change with this command: .. code-block:: console - $ ./Miniconda3-py39_4.12.0-Linux-x86_64.sh + $ source $HOME/.bashrc + This will tell your shell to look for executable files in your + :file:`$HOME/bin` folder before it looks through the rest of your + search path. |br| |br| -#. Accept the license agreement. +#. Define the :literal:`mamba` convenience alias. - When the installer script starts, you will be prompted to accept - the Miniconda license agreement: + Add the following lines to your :file:`$HOME/.bashrc` file - .. code-block:: console - - Welcome to Miniconda3 py39_4.12.0 + .. code-block:: bash - In order to continue the installation process, please review the license - agreement. - Please, press ENTER to continue - >>> + # Invoke micromamba as "mamba" + alias mamba="micromamba" - When you press :literal:`ENTER`, you will see the license agreement - in all of its gory legalese detail. Press the space bar repeatedly - to scroll down ot the end. You will then see this prompt: + Apply the change with this command: .. code-block:: console - Do you accept the license terms? [yes|no] - [no] >>> + $ source ~/.bashrc - Type :literal:`yes` and hit :literal:`ENTER` to accept. |br| + This will allow you to invoke :program:`MicroMamba` by typing + :literal:`mamba`. |br| |br| +#. Specify where :program:`Mamba` will install packages and environments. -#. Specify the installation path. - - You will then be prompted to provide a directory path for the - installation: - - .. code-block:: console + The default installation paths are: - Miniconda3 will now be installed into this location: - /home/YOUR-USERNAME/miniconda3 - - - Press ENTER to confirm the location - - Press CTRL-C to abort the installation - - Or specify a different location below - - [/home/YOUR-USERNAME/miniconda3] >>> - - Press :literal:`ENTER` to continue, or specify a new path and then - press :literal:`ENTER`. - - .. tip:: - - If a previous Conda installation is already installed to the - default path, you may choose to delete the previous installation - folder, or install Conda 4.12.0 to a different path. - - The script will then start installing the Conda 4.12.0 package - manager. |br| - |br| + .. code-block:: bash + $HOME/micromamba/envs # Where environments will be created + $HOME/micromamba/pkgs # Where packages will be downloaded. -#. Specify post-installation options. + For most use cases, this should be sufficient. If you do not wish + to change these default installation paths, **skip ahead to the + next step**. - You will see this text at the bottom of the screen printout upon - successful installation: + However, you may need to install Python packages and environments + in a different location than your home directory. To change the + default installation paths, add these environment variables to your + :file:`~/.bashrc` file: - .. code-block:: console + .. code-block:: bash - Preparing transaction: done - Executing transaction: done - installation finished. - Do you wish the installer to initialize Miniconda3 - by running conda init? [yes|no] - [no] >>> + export MAMBA_PREFIX=/path/to/preferred/installation/folder + export MAMBA_PKGS_DIRS=${MAMBA_PREFIX}/pkgs + export MAMBA_ENVS_PATH=${MAMBA_PREFIX}/envs - Type :literal:`yes` and press :literal:`ENTER`. You will see - output similar to this: + Then appply the changes with this command. .. code-block:: console - no change /home/bob/miniconda3/condabin/conda - no change /home/bob/miniconda3/bin/conda - no change /home/bob/miniconda3/bin/conda-env - no change /home/bob/miniconda3/bin/activate - no change /home/bob/miniconda3/bin/deactivate - no change /home/bob/miniconda3/etc/profile.d/conda.sh - no change /home/bob/miniconda3/etc/fish/conf.d/conda.fish - no change /home/bob/miniconda3/shell/condabin/Conda.psm1 - no change /home/bob/miniconda3/shell/condabin/conda-hook.ps1 - no change /home/bob/miniconda3/lib/python3.9/site-packages/xontrib/conda.xsh - no change /home/bob/miniconda3/etc/profile.d/conda.csh - no change /home/bob/.bashrc - No action taken. - If you'd prefer that conda's base environment not be activated on startup, - set the auto_activate_base parameter to false: - - conda config --set auto_activate_base false - - Thank you for installing Miniconda3! + $ source ~/.bashrc |br| -#. Disable the base Conda environment from being activated at startup +#. Run first-time :program:`Mamba` initialization. - Close the terminal window that you used to install Conda 4.12.0 and - open a new terminal window. You will see this prompt: + Before installing any Python package, you must initialize + :program:`MicroMamba` with this command: .. code-block:: console - (base) $ + $ mamba shell init --shell=bash - By default, Conda will open the :literal:`base` environment each - time that you open a new terminal window. to disable this - behavior, type: + This will add some code into your :file:`.bashrc` startup script. + To apply the changes, use: .. code-block:: console - (base) $ conda config --set auto_activate_base false - - The next time you open a terminal window, you will just see the - regular prompt, such as; + $ source ~/.bashrc - .. code-block:: console - - $ - - (or whatever you have defined your prompt to be in your startup scripts). - -Now that you have installed Conda 4.12.0, you may proceed to creating -a new Conda environment for GCPy, as shown below. + :program:`Mamba` should now be ready for use! .. _gcpy_install: -========================================== -Steps to install GCPy and its dependencies -========================================== +================================= +Install GCPy and its dependencies +================================= -#. Install Conda if it is not already installed. - - If Conda 4.12.0 or prior is already installed on your system, you - may skip this step. Otherwise, please follow the instructions - listed in :ref:`conda412_install`. |br| - |br| +Once :program:`Mamba` has been installed, you may proceed use it to +create a Python environment for GCPy. (Please return to +:ref:`mamba_install` if you have not yet installed :program:`Mamba`.) #. Download the GCPy source code. Create and go to the directory in which you would like to store GCPy. In - this example we will store GCPy in a :file:`python/packages` - subdirectory in your home directory, but you can store it wherever - you wish. You can also name the GCPy download whatever you want. In - this example the GCPy directory is called :file:`GCPy`. + this example we will store GCPy in your :file:`$HOME/python/` + path, but you can store it wherever you wish. You can also name + the GCPy download whatever you want. In this example the GCPy + directory is called :file:`GCPy`. .. code-block:: console - $ cd $HOME/python/packages + $ cd $HOME/python $ git clone https://github.com/geoschem/gcpy.git GCPy $ cd GCPy @@ -292,12 +209,12 @@ Steps to install GCPy and its dependencies maintain a set of package dependencies compatible with GCPy without interfering with Python packages you use for other work. You can create a Python virtual environment from anywhere on your - system. It will be stored in your Conda installation rather than - the directory from which you create it. + system. It will be stored in your :program:`Mamba` installation + rather than the directory from which you create it. You can create a Python virtual environment using a file that lists all packages and their versions to be included in the environment. - GCPy includes such as file, environment.yml, located in the + GCPy includes such as file, :file:`environment.yml`, located in the top-level directory of the package. Run the following command at the command prompt to create a virtual @@ -306,20 +223,25 @@ Steps to install GCPy and its dependencies .. code-block:: console - $ conda env create -n gcpy_env --file=environment.yml + $ mamba env create -n gcpy_env --file=environment.yml + + A list of packages to be downloaded will be displayed. A + confirmation message will ask you if you really wish to install all + of the listed packages. Type :command:`Y` to proceed or + :command:`n` to abort. - Once successfully created you can load the environment by running the - following command, specifying the name of your environment. + Once successfully created you can activate the environment with + this command: .. code-block:: console - $ conda activate gcpy_env + $ mamba activate gcpy_env - To exit the environment do the following: + To exit the environment, use this command: .. code-block:: console - $ conda deactivate + $ mamba deactivate |br| @@ -328,18 +250,24 @@ Steps to install GCPy and its dependencies The environment variable :envvar:`PYTHONPATH` specifies the locations of Python libraries on your system that are not included in your conda environment. If GCPy is included in - :envvar:`PYTHONPATH` then Python will recognize its existence - when you try to use. Add the following line to your startup script, - e.g. :file:`.bashrc`, and edit the path to where you are storing - GCPy. + :envvar:`PYTHONPATH` then Python will recognize its existence. + + Add the path to your GCPy source code folder :file:`~/.bashrc` file: .. code-block:: bash - export PYTHONPATH=$PYTHONPATH:$HOME/python/packages/GCPy + export PYTHONPATH=$PYTHONPATH:$HOME/python/GCPy + + and then use + + .. code-block:: console + + $ source ~/.bashrc + to apply the change. |br| |br| -#. Perform a simple test. +#. Perform a simple test: Run the following commands in your terminal to check if the installation was succcesful. @@ -348,8 +276,8 @@ Steps to install GCPy and its dependencies $ source $HOME/.bashrc # Alternatively close and reopen your terminal $ echo $PYTHONPATH # Check it contains path to your GCPy clone - $ conda activate gcpy_env - $ conda list # Check it contains contents of gcpy env file + $ mamba activate gcpy_env + $ mamba list # Check it contains contents of gcpy env file $ python >>> import gcpy @@ -371,7 +299,7 @@ latest available. .. code-block:: console - $ cd $HOME/python/packages/GCPy + $ cd $HOME/python/GCPy $ git fetch -p $ git checkout main $ git pull @@ -380,7 +308,7 @@ You can also checkout an older version by doing the following: .. code-block:: console - $ cd $HOME/python/packages/GCPy + $ cd $HOME/python/GCPy $ git fetch -p $ git tag $ git checkout tags/version_you_want @@ -390,6 +318,6 @@ commands to then update your virtual environment: .. code-block:: console - $ source activate gcpy_env - $ cd $HOME/python/packages/GCPy - $ conda env update --file environment.yml --prune + $ mamba activate gcpy_env + $ cd $HOME/python/GCPy + $ mamba env update --file environment.yml --prune From 4901a0ab0f8494efef98866bfedca571e69bb902 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Fri, 11 Aug 2023 17:20:00 -0400 Subject: [PATCH 03/11] Remove note in Regridding.rst docs/source/Regridding.rst - Remove note that says gridspec & sparselt are both in the GCPy 1.4.0 development stream. Sparselt is, but gridspec currently is not. Signed-off-by: Bob Yantosca --- docs/source/Regridding.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/source/Regridding.rst b/docs/source/Regridding.rst index cb4b23ea..050de124 100644 --- a/docs/source/Regridding.rst +++ b/docs/source/Regridding.rst @@ -132,10 +132,6 @@ three stage process: Python environment for gridspec and sparselt -------------------------------------------- -**NOTE: Starting with GCPy 1.4.0 (currently under development),** -:code:`gridspec` **and** :code:`sparselt` **are included in the -standard environment file.** - Until GCPy contains a complete regridding implementation that works for all GEOS-Chem grid formats, we recommend that you create a small `mamba `_ environment in which to From 530a0002959809be243546f306e0556d6cb7a45b Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Fri, 11 Aug 2023 18:30:33 -0400 Subject: [PATCH 04/11] Use better installation instructions for MicroMamba docs/source/Getting-Started-with-GCPy.rst - Now use the preferred MicroMamba installation instructions Signed-off-by: Bob Yantosca --- docs/source/Getting-Started-with-GCPy.rst | 119 ++++++++++------------ 1 file changed, 55 insertions(+), 64 deletions(-) diff --git a/docs/source/Getting-Started-with-GCPy.rst b/docs/source/Getting-Started-with-GCPy.rst index 8c64523b..a990d3a9 100644 --- a/docs/source/Getting-Started-with-GCPy.rst +++ b/docs/source/Getting-Started-with-GCPy.rst @@ -55,26 +55,61 @@ Installing Mamba as MicroMamba easier to install than the full :program:`MambaForge` distribution. Follow the installation steps listed below: -#. Download and run the :program:`MicroMamba` executable. +#. Download and initialize :program:`MicroMamba`. - Execute the command listed below that is relevant to your operating - system. (You can cut-and-paste the command into a terminal window.) + Execute this command: .. code-block:: console - # Linux Intel (x86_64), including Windows Subsystem for Linux - $ curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba + $ "${SHELL}" <(curl -L micro.mamba.pm/install.sh) - # MacOS Intel (x86_64): - $ curl -Ls https://micro.mamba.pm/api/micromamba/osx-64/latest | tar -xvj bin/micromamba + This will download the :program:`MicroMamba` via the + :program:`curl` utility. You will be then asked several questions: - # MacOS Silicon/M1, Silicon/M2 (ARM64): - $ curl -Ls https://micro.mamba.pm/api/micromamba/osx-arm64/latest | tar -xvj bin/micromamba + .. code-block:: console + + Micromamba binary folder? [~/.local/bin] + + This prompt is asking where you prefer to install the + :program:`MicroMamba` executable. Press :command:`ENTER` to accept + the default or type a new location, then press :command:`ENTER`. + + .. code-block:: + + Prefix location? [~/micromamba] + + This prompt is asking where the Python packages and environments + created with Micromamba should be installed. The default location + is your :file:`$HOME/micromamba` folder. Press :command:`ENTER` to + accept the default or specify a new location and then press + :command:`ENTER`. + + .. code-block:: + + Init shell? [Y/n] + + This prompt is asking if you would like :program:`MicroMamba` to + add some code into your :file:`$HOME/.bashrc` startup script. + Press :command:`Y` and then :command:`ENTER`. + + .. code-block:: + + Configure conda-forge? [Y/n] + + This prompt is asking if you would like :program:`MicroMamba` to + have access to the :literal:`conda-forge` repository. Press + :command:`Y` then :command:`ENTER`. + + Then :program:`MicroMamba` installer will print out the following + information to the screen: + + .. code-block:: console + + Modifying RC file "/path/to/.bashrc" + Generating config for root prefix "/path/to/root/prefix" + Setting mamba executable to: "/path/to/mamba/executable-dir/micromamba + Adding (or replacing) the following in your "/path/to/.bashrc" file - This will download the :program:`MicroMamba` executable file into - following path: :file:`$HOME/bin/micromamba`. If you do not - already have a :file:`$HOME/bin` folder, it will be created for - you. |br| |br| #. Tell your shell where it can find the :program:`MicroMamba` executable. @@ -84,11 +119,14 @@ Follow the installation steps listed below: .. code-block:: bash - export PATH="$HOME/bin:$PATH" + export PATH="/path/to/mamba/executable-dir:$PATH" + where :file:`/path/to/mamba/executable-dir` is the same text as + displayed in the previous step. + .. note:: - On some computer systems, it is preferred that users place + Some shared computer systems prefer that users place modifications not into the :file:`$HOME/.bashrc` file, but instead to a different script (e.g. :file:`$HOME/.bash_aliases`) that is executed by :file:`$HOME/.bashrc`. Ask your system @@ -125,55 +163,8 @@ Follow the installation steps listed below: :literal:`mamba`. |br| |br| -#. Specify where :program:`Mamba` will install packages and environments. - - The default installation paths are: - - .. code-block:: bash - - $HOME/micromamba/envs # Where environments will be created - $HOME/micromamba/pkgs # Where packages will be downloaded. - - For most use cases, this should be sufficient. If you do not wish - to change these default installation paths, **skip ahead to the - next step**. - - However, you may need to install Python packages and environments - in a different location than your home directory. To change the - default installation paths, add these environment variables to your - :file:`~/.bashrc` file: - - .. code-block:: bash - - export MAMBA_PREFIX=/path/to/preferred/installation/folder - export MAMBA_PKGS_DIRS=${MAMBA_PREFIX}/pkgs - export MAMBA_ENVS_PATH=${MAMBA_PREFIX}/envs - - Then appply the changes with this command. - - .. code-block:: console - - $ source ~/.bashrc - - |br| - -#. Run first-time :program:`Mamba` initialization. - - Before installing any Python package, you must initialize - :program:`MicroMamba` with this command: - - .. code-block:: console - - $ mamba shell init --shell=bash - - This will add some code into your :file:`.bashrc` startup script. - To apply the changes, use: - - .. code-block:: console - - $ source ~/.bashrc - - :program:`Mamba` should now be ready for use! + You are now ready to use :program:`Mamba` (installed as + :program:`MicroMamba`)! .. _gcpy_install: From f5100d8c946c00f37452308283f3943dfb8193eb Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 12:10:00 -0400 Subject: [PATCH 05/11] Update README.md and About-GCPy.rst for ReadTheDocs README.md docs/source/About-GCPy.rst - Synchronize content - Updated formatting in bullet lists - Added links to license documents - Added Request Support section - Added Submitting New Features section Signed-off-by: Bob Yantosca --- README.md | 49 ++++++++++++++------- docs/source/About-GCPy.rst | 88 +++++++++++++++++++++++++++++--------- 2 files changed, 100 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 9c514daf..2ed0c75d 100644 --- a/README.md +++ b/README.md @@ -13,38 +13,55 @@ **GCPy** is a Python-based toolkit containing useful functions for working specifically with the GEOS-Chem model of atmospheric chemistry and composition. -GCPy aims to build on the well-established scientific Python technical stack, leveraging tools like cartopy and xarray to simplify the task of working with model output and performing atmospheric chemistry analyses. +**GCPy** aims to build on the well-established scientific Python technical stack, leveraging tools like **cartopy**, **numpy**, and **xarray** to simplify the task of working with GEOS-Chem model output and performing atmospheric chemistry analyses. ## What GCPy was intended to do: -1. Produce plots and tables from GEOS-Chem output using simple function calls. -2. Generate the standard evaluation plots and tables from GEOS-Chem benchmark output. -3. Obtain GEOS-Chem's horizontal/vertical grid information. -4. Implement GCHP-specific regridding functionalities (e.g. cubed-sphere to lat-lon regridding) -5. Provide example scripts for creating specific types of plots or analysis from GEOS-Chem output. +1. Produce plots and tables from [GEOS-Chem](https://geos-chem.readthedocs.io) output using simple function calls. +2. Generate the standard evaluation plots and tables for GEOS-Chem benchmark simulations. +3. Obtain GEOS-Chem's horizontal and vertical grid information. +4. Implement [GCHP](https://gchp.readthedocs.io)-specific regridding functionalities (e.g. cubed-sphere to lat-lon regridding) +5. Provide example scripts for creating specific types of plots or analysis from GEOS-Chem output. +6. Provide user-submitted scripts for specific applications related to GEOS-Chem and [HEMCO](https://hemco.readthedocs.io]. -## What GCPY was not intended to do: +## What GCPy was not intended to do: 1. General NetCDF file modification: (crop a domain, extract some variables): - * Use [xarray](http://xarray.pydata.org) instead. - * Also see [our *Working with netCDF data files* wiki page](http://wiki.geos-chem.org/Working_with_netCDF_data_files). + * Use standard netCDF tools, such as: + * [xarray](http://xarray.pydata.org) + * [netCDF operators (NCO)](https://nco.sourceforge.net) + * [Climate Data Operators](https://mpimet.mpg.de/cdo) instead. + * Also see our [*Work with netCDF files* guide](https://geos-chem.readthedocs.io/en/latest/geos-chem-shared-docs/supplemental-guides/netcdf-guide.html) at [geos-chem.readthedocs.io](https://geos-chem.readthedocs.io) + 2. Statistical analysis: - * Use [scipy](http://www.scipy.org)/[scikit-learn](https://scikit-learn.org) tools instead -3. Machine Learning: - * Use the standard machine learning utilities ([pytorch](https://pytorch.org), [tensorflow](https://www.tensorflow.org), [julia](https://julialang.org), etc.) + * Use standard statistical tools, such as: + * Use [scipy](http://www.scipy.org) + * [scikit-learn](https://scikit-learn.org) + * [R](https://r-project.org) + * etc +3. Machine Learning: + * Use the standard machine learning utilities: + * [pytorch](https://pytorch.org), + * [tensorflow](https://www.tensorflow.org) + * [julia](https://julialang.org) + * etc. ## Documentation: For more information on installing and using GCPy, visit the official documentation at [gcpy.readthedocs.io](https://gcpy.readthedocs.io/). - ## License -GCPy is distributed under the MIT license. Please read the license documents LICENSE.txt and AUTHORS.txt, which are located in the root folder. +GCPy is distributed under the MIT license. Please see the [GCPy license agreement](https://github.com/geoschem/gcpy/blob/dev/LICENSE.txt) and [List of GCPy developers](https://github.com/geoschem/gcpy/blob/dev/AUTHORS.txt) for more information. + +## Requesting support +To report a bug or suggest a new feature, please see our [Support +Guidelines](https://github.com/geoschem/gcpy/blob/dev/SUPPORT.md). -## Contact +## Submitting new features -To contact us, please [open a new issue on the issue tracker connected to this repository](https://github.com/geoschem/gcpy/issues/new/choose). You can ask a question, report a bug, or request a new feature. +If you are interested in submitting code to GCPy, please see our +[Contributing Guidelines](https://github.com/geoschem/gcpy/blob/dev/CONTRIBUTING.md). diff --git a/docs/source/About-GCPy.rst b/docs/source/About-GCPy.rst index c5a463ef..0d310757 100644 --- a/docs/source/About-GCPy.rst +++ b/docs/source/About-GCPy.rst @@ -1,16 +1,22 @@ +.. |br| raw:: html + +
+ .. _about: ########## About GCPy ########## -**GCPy** is a Python-based toolkit containing useful functions for -working specifically with the GEOS-Chem model of atmospheric chemistry -and composition. +:program:`GCPy` is a Python-based toolkit containing useful functions for +working specifically with the :program:`GEOS-Chem` model of +atmospheric chemistry and composition. -GCPy aims to build on the well-established scientific Python technical -stack, leveraging tools like cartopy and xarray to simplify the task of -working with model output and performing atmospheric chemistry analyses. +GCPy aims to build on the well-established scientific +Python technical stack, leveraging tools like :program:`cartopy`, +:program:`numpy`, and :program:`xarray` to simplify the task of +working with GEOS-Chem model output and performing atmospheric +chemistry analyses. .. _about-what-gcpy-does: @@ -18,37 +24,77 @@ working with model output and performing atmospheric chemistry analyses. What GCPy was intended to do ============================ -#. Produce plots and tables from GEOS-Chem output using simple function +#. Produce plots and tables from `GEOS-Chem + `_ output using simple function calls. #. Generate the standard evaluation plots and tables from GEOS-Chem - benchmark output. -#. Obtain GEOS-Chem's horizontal/vertical grid information. -#. Implement GCHP-specific regridding functionalities (e.g. cubed-sphere - to lat-lon regridding). + benchmark simulations. +#. Obtain GEOS-Chem's horizontal and vertical grid information. +#. Implement `GCHP `_-specific regridding + functionalities (e.g. cubed-sphere to lat-lon regridding). #. Provide example scripts for creating specific types of plots or analysis from GEOS-Chem output. +#. Provide user-submitted scripts for specific applications related to + GEOS-Chem and `HEMCO `_. .. _about-what-gcpy-doesnt-do: ================================ -What GCPY was not intended to do +What GCPy was not intended to do ================================ #. General NetCDF file modification: (crop a domain, extract some variables): - - Use `xarray `__ instead. - - Also see `Work with netCDF data files + - Use standard netCDF tools instead, such as: + + - `xarray `_ + - `netCDF Operators (NCO) `_ + - `Climate Data Operators (CDO) `_ + + - Also see our `Work with netCDF data `_ - at the GEOS-Chem ReadTheDocs site. + guide at `geos-chem.readthedocs.io + `_. #. Statistical analysis: - - Use `scipy `__ and `scikit-learn - `__ tools instead. + - Use standard statistical tools, such as: + + - `scipy `_ + - `scikit-learn `_ + - `R `_ + - etc. #. Machine Learning: - - Use the standard machine learning utilities - (`pytorch `__, - `tensorflow `__, - `julia `__, etc.). + - Use standard machine learning tools, such as + + - `pytorch `_ + - `tensorflow `_ + - `julia `_ + - etc. + +======= +License +======= + +GCPy is distributed under the `MIT license +`_. Please see the `GCPy license +agreement `_ +and `List of GCPy developers +`_ for more +information. + +================== +Requesting support +================== + +To report a bug or suggest a new feature, please see our `Support +Guidelines `_. + +======================= +Submitting new features +======================= + +If you are interested in submitting code to GCPy, please see our +`Contributing Guidelines `_. From 13e71db85a31aa4ccbea89249a26261ac21c2892 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 12:12:05 -0400 Subject: [PATCH 06/11] Update Getting-Started-with-GCPy.rst w/ MambaForge instructions docs/source/Getting-Started-with-GCPy.rst - Replaced MicroMamba installation instructions with MambaForge installation instructions - Add separate instructions for installing MambaForge on MacOS and Linux/WSL - Updated formatting, trimmed whitespace Signed-off-by: Bob Yantosca --- docs/source/Getting-Started-with-GCPy.rst | 266 +++++++++++++--------- 1 file changed, 159 insertions(+), 107 deletions(-) diff --git a/docs/source/Getting-Started-with-GCPy.rst b/docs/source/Getting-Started-with-GCPy.rst index a990d3a9..ad93963e 100644 --- a/docs/source/Getting-Started-with-GCPy.rst +++ b/docs/source/Getting-Started-with-GCPy.rst @@ -14,157 +14,210 @@ Installing GCPy Requirements ============ -:program:`GCPy` is currently supported for Linux and MacOS operating -systems. Due to a reliance on several packages without Windows -support, **GCPy is not currently supported for Windows**. You will -receive an error message if you attempt to use GCPy on Windows. - -.. tip:: - - Windows 11 (and some later builds of Windows 10) support the - `Windows Subsystem for Linux (WSL) - `_. If your - Windows version is WSL-compatible, you can install GCPy into a - Linux instance (such as Ubuntu 22.04) running under Windows. At - present, this is the only way to use GCPy locally on a Windows - computer. - -The only essential software you need before installing GCPy is a -distribution of the :program:`Mamba` package manager. :program:`Mamba` -is a drop-in replacement for the widely-used :program:`Conda` -package manager. You will use :program:`Mamba` to create a Python -environment for GCPy, which contains a Python version (currently -3.9.6) plus specific versions of various dependent packages. - -Check if you already have :program:`Mamba` by using this command: +:program:`GCPy` is currently supported on the following platforms: + +#. Linux (x86_64) +#. Windows Subsystem for Linux (running in Microsoft Windows 11) +#. MacOS + +To install GCPy, you will need a distribution of the :program:`Mamba` +package manager. :program:`Mamba` is a drop-in replacement for the +widely-used :program:`Conda` package manager. You will use +:program:`Mamba` to create a Python environment for GCPy, which +contains the Python interpreter plus other Python packages that GCPY +uses. + +Check if you already have :program:`Mamba` on your system: .. code-block:: console $ mamba --version -If :program:`Mamba` has already been installed on your system, **skip -ahead to the** :ref:`gcpy_install` **section**. +If :program:`Mamba` has been installed, you will see output similar to this: + +.. code-block:: + + mamba version X.Y.Z + conda version A.B.C + +If you see this output, you may skip ahead to the :ref:`gcpy_install` +section. Otherwise, read further for instructions on how to install +:program:`Mamba` on your system. .. _mamba_install: -============================== -Installing Mamba as MicroMamba -============================== +================== +Install MambaForge +================== -:program:`MicroMamba` is a minimal :program:`Mamba` version that is -easier to install than the full :program:`MambaForge` distribution. -Follow the installation steps listed below: +We recommend installing the :program:`MambaForge`, distribution, which +is a full implementation of :program:`Mamba` (as opposed to the +minimal :program:`MicroMamba` distribution). -#. Download and initialize :program:`MicroMamba`. +Follow the instructions below to install :program:`MambaForge`: - Execute this command: +MacOS +----- + +#. Install :program:`MambaForge` with `Homebrew `_: .. code-block:: console - $ "${SHELL}" <(curl -L micro.mamba.pm/install.sh) + $ brew install mambaforge + + |br| - This will download the :program:`MicroMamba` via the - :program:`curl` utility. You will be then asked several questions: +#. Initialize :program:`Mamba` for your shell. Type one of the + following commands: .. code-block:: console - Micromamba binary folder? [~/.local/bin] + $ mamba init bash # If you use the bash shell (recommended!) + $ mamba init zsh # If you use the zsh shell + $ mamba init fish # If you use the fish shell - This prompt is asking where you prefer to install the - :program:`MicroMamba` executable. Press :command:`ENTER` to accept - the default or type a new location, then press :command:`ENTER`. + :program:`Mamba` will add some code to your :file:`~/.bash_profile` + startup script that will tell your shell where to look for + Python environments. - .. code-block:: - - Prefix location? [~/micromamba] + |br| - This prompt is asking where the Python packages and environments - created with Micromamba should be installed. The default location - is your :file:`$HOME/micromamba` folder. Press :command:`ENTER` to - accept the default or specify a new location and then press - :command:`ENTER`. +#. Exit your current terminal session and open a new terminal + session. This will apply the changes. - .. code-block:: - - Init shell? [Y/n] +You may now skip ahead to the :ref:`gcpy_install` section. - This prompt is asking if you would like :program:`MicroMamba` to - add some code into your :file:`$HOME/.bashrc` startup script. - Press :command:`Y` and then :command:`ENTER`. - .. code-block:: +Linux and Windows Subsystem for Linux +-------------------------------------- - Configure conda-forge? [Y/n] +#. Download the :program:`MambaForge` installer script from the + `conda-forge GitHub releases page + `_: - This prompt is asking if you would like :program:`MicroMamba` to - have access to the :literal:`conda-forge` repository. Press - :command:`Y` then :command:`ENTER`. + .. code-block:: console + + $ wget https://github.com/conda-forge/miniforge/releases/download/23.3.1-0/Mambaforge-23.3.1-0-Linux-x86_64.sh + + This will download the :program:`MambaForge` installer script + :file:`Mambaforge-23.3.1-0-Linux-x86_64.sh` to your computer. + + .. note:: + + As of this writing (August 2023), the latest + :program:`MambaForge` version is :literal:`23.1.0-0`. If you + find that the version has since been updated, simply replace the + version number :literal:`23.3.1-0` in the above command with the + most recent version number. + + |br| + +#. Change the permission of the :program:`MambaForge` installer script + so that it is executable. - Then :program:`MicroMamba` installer will print out the following - information to the screen: - .. code-block:: console - Modifying RC file "/path/to/.bashrc" - Generating config for root prefix "/path/to/root/prefix" - Setting mamba executable to: "/path/to/mamba/executable-dir/micromamba - Adding (or replacing) the following in your "/path/to/.bashrc" file + $ chmod 755 Mambaforge-23.3.1-0-Linux-x86_64.sh |br| -#. Tell your shell where it can find the :program:`MicroMamba` executable. +#. Execute the :program:`Mambaforge` installer script. - If you have not done so already, add the following - line to your :file:`$HOME/.bashrc` startup script: + .. code-block:: - .. code-block:: bash + $ ./Mambaforge-23.3.1-0-Linux-x86_64.sh - export PATH="/path/to/mamba/executable-dir:$PATH" + To update an older version of :program:`Mamba`, add the + :literal:`-u` option to the above command. |br| + |br| - where :file:`/path/to/mamba/executable-dir` is the same text as - displayed in the previous step. - - .. note:: +#. Review and accept the license agreement. + + .. code-block:: console - Some shared computer systems prefer that users place - modifications not into the :file:`$HOME/.bashrc` file, but - instead to a different script (e.g. :file:`$HOME/.bash_aliases`) - that is executed by :file:`$HOME/.bashrc`. Ask your system - administrator for more information pertaining to your particular - setup. + In order to continue the installation process, please review the license + agreement. + Please, press ENTER to continue + >>> - Apply the change with this command: + Press :literal:`ENTER` and then :literal:`SPACE` until you reach + the end of the license agreement. Then you will be asked: .. code-block:: console - $ source $HOME/.bashrc + Do you accept the license terms? [yes|no] + [no] >>> - This will tell your shell to look for executable files in your - :file:`$HOME/bin` folder before it looks through the rest of your - search path. |br| + Type :literal:`yes` and hit :literal:`ENTER`. |br| |br| -#. Define the :literal:`mamba` convenience alias. - Add the following lines to your :file:`$HOME/.bashrc` file +#. Specify the root installation path for :program:`MambaForge`. - .. code-block:: bash + .. code-block:: - # Invoke micromamba as "mamba" - alias mamba="micromamba" + Mambaforge will now be installed into this location: + /home/YOUR-USER-NAME/mambaforge - Apply the change with this command: + - Press ENTER to confirm the location + - Press CTRL-C to abort the installation + - Or specify a different location below + [/home/bob/mambaforge] >>> + + In most cases, it should be OK to accept the default installation + location. But on some systems, users may be encouraged to install + software into a different location (e.g. if there is a faster + filesystem available than the home directory filesystem). + Consult your sysadmin or IT staff if you are unsure where to + install :program:`MambaForge`. + + Press the :literal:`ENTER` key to accept the default installation + path or type a new path and then press :literal:`ENTER`. .. code-block:: console - $ source ~/.bashrc + :program:`MambaForge` will downlad and install Python software + packages into the :file:`pkgs` subfolder of the root + installation path. Similarly, when you :ref:`create Python + environments `, these will be installed to the + :file:`envs` subfolder of the root installation path. - This will allow you to invoke :program:`MicroMamba` by typing - :literal:`mamba`. |br| |br| - You are now ready to use :program:`Mamba` (installed as - :program:`MicroMamba`)! +#. You may see this warning: + + .. code-block:: console + + WARNING: + You currently have a PYTHONPATH environment variable set. This may cause + unexpected behavior when running the Python interpreter in Mambaforge. + For best results, please verify that your PYTHONPATH only points to + directories of packages that are compatible with the Python interpreter + in Mambaforge: /home/YOUR-USER-NAMEb/mambaforge + + As long as your :envvar:`PYTHONPATH` environment variable only + contains the path to the root-level GCPy folder, you may safely + ignore this. (More on :envvar:`PYTHONPATH` in the :ref:`next + section `.) |br| + |br| + +#. Tell the installer to initialize :program:`MambaForge`. + + .. code-block:: console + + Do you wish the installer to initialize Mambaforge + by running conda init? [yes|no] + [no] >>> + + Type :literal:`yes` and then :literal:`ENTER`. The installer + script will add some code to your :file:`~/.bashrc` system startup + file that will tell your shell where to find Python + environments. |br| + |br| + + +#. Exit your current terminal session. Start a new terminal session + to apply the updates. You are now ready to install GCPy. .. _gcpy_install: @@ -172,11 +225,11 @@ Follow the installation steps listed below: Install GCPy and its dependencies ================================= -Once :program:`Mamba` has been installed, you may proceed use it to -create a Python environment for GCPy. (Please return to -:ref:`mamba_install` if you have not yet installed :program:`Mamba`.) +Once :program:`Mamba` has been installed, use it to ccreate a Python +environment for GCPy. (Please return to :ref:`mamba_install` if you +have not yet installed :program:`Mamba`.) -#. Download the GCPy source code. +#. **Download the GCPy source code.** Create and go to the directory in which you would like to store GCPy. In this example we will store GCPy in your :file:`$HOME/python/` @@ -192,7 +245,7 @@ create a Python environment for GCPy. (Please return to |br| -#. Create a new Python virtual environment for GCPy. +#. **Create a new Python virtual environment for GCPy.** A Python virtual environment is a named set of Python installs, e.g. packages, that are independent of other virtual @@ -236,12 +289,11 @@ create a Python environment for GCPy. (Please return to |br| -#. Add GCPy to Python path. +#. **Add GCPy to** :envvar:`PYTHONPATH` The environment variable :envvar:`PYTHONPATH` specifies the - locations of Python libraries on your system that are not included - in your conda environment. If GCPy is included in - :envvar:`PYTHONPATH` then Python will recognize its existence. + locations of Python libraries on your system that were not + installed by :program:`Mamba`. Add the path to your GCPy source code folder :file:`~/.bashrc` file: @@ -258,7 +310,7 @@ create a Python environment for GCPy. (Please return to to apply the change. |br| |br| -#. Perform a simple test: +#. **Perform a simple test:** Run the following commands in your terminal to check if the installation was succcesful. From 164904244ad7f49ff31237911f5ced53d5d9e773 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 13:39:47 -0400 Subject: [PATCH 07/11] Synchronize wording in README.md and About-GCPy.rst README.md docs/source/About-GCPy.rst - Now use the same wording in bulleted lists. Signed-off-by: Bob Yantosca --- README.md | 6 +++--- docs/source/About-GCPy.rst | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 2ed0c75d..f609bcd3 100644 --- a/README.md +++ b/README.md @@ -28,21 +28,21 @@ ## What GCPy was not intended to do: 1. General NetCDF file modification: (crop a domain, extract some variables): - * Use standard netCDF tools, such as: + * Instead, use netCDF tools such as: * [xarray](http://xarray.pydata.org) * [netCDF operators (NCO)](https://nco.sourceforge.net) * [Climate Data Operators](https://mpimet.mpg.de/cdo) instead. * Also see our [*Work with netCDF files* guide](https://geos-chem.readthedocs.io/en/latest/geos-chem-shared-docs/supplemental-guides/netcdf-guide.html) at [geos-chem.readthedocs.io](https://geos-chem.readthedocs.io) 2. Statistical analysis: - * Use standard statistical tools, such as: + * Instead, use statistical tools such as: * Use [scipy](http://www.scipy.org) * [scikit-learn](https://scikit-learn.org) * [R](https://r-project.org) * etc 3. Machine Learning: - * Use the standard machine learning utilities: + * Instead, use machine learning tools such as: * [pytorch](https://pytorch.org), * [tensorflow](https://www.tensorflow.org) * [julia](https://julialang.org) diff --git a/docs/source/About-GCPy.rst b/docs/source/About-GCPy.rst index 0d310757..f10d8002 100644 --- a/docs/source/About-GCPy.rst +++ b/docs/source/About-GCPy.rst @@ -45,7 +45,7 @@ What GCPy was not intended to do #. General NetCDF file modification: (crop a domain, extract some variables): - - Use standard netCDF tools instead, such as: + - Instead, use netCDF tools such as: - `xarray `_ - `netCDF Operators (NCO) `_ @@ -58,7 +58,7 @@ What GCPy was not intended to do #. Statistical analysis: - - Use standard statistical tools, such as: + - Instead, use statistical tools such as: - `scipy `_ - `scikit-learn `_ @@ -67,7 +67,7 @@ What GCPy was not intended to do #. Machine Learning: - - Use standard machine learning tools, such as + - Instead, use machine learning tools such as: - `pytorch `_ - `tensorflow `_ From d29e10aa4c2bec40185b90b532cccf9de889624c Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 15:36:45 -0400 Subject: [PATCH 08/11] Update gcpy/file_regrid.py and Regridding.rst for consistency docs/source/Regridding.rst - Add documentation for more options to gcpy/file_regrid.py - Remove references to the "gchp_regridding" environment, this is no longer needed gcpy/file_regrid.py - Use None for arguments that are list-based, and assign them default values later. (Suggested by Pylint) - Check arguments with util.verify_variable_type() - Update formatting, trim trailing whitespace - Remove unused variables - Rename ds -> dset - Rename mi -> mindex - Rename f -> oface - Move argument parsing & call to file_regrid into argument(main), which is called from within a 'if __name__ == "__main__"' block. We will test to make sure the examples listed in the documentation are actually correct. Further updates may be necessary. Signed-off-by: Bob Yantosca --- docs/source/Regridding.rst | 226 +++++++++++++++++++++------------- gcpy/file_regrid.py | 246 ++++++++++++++++++++++++++----------- 2 files changed, 309 insertions(+), 163 deletions(-) diff --git a/docs/source/Regridding.rst b/docs/source/Regridding.rst index 050de124..3a087732 100644 --- a/docs/source/Regridding.rst +++ b/docs/source/Regridding.rst @@ -8,13 +8,23 @@ Regridding ########## -This page describes the regridding capabilities of -:program:`GCPy`. GCPy currently supports regridding of data from -GEOS-Chem restarts and output NetCDF files. Regridding is supported -across any horizontal resolution and any grid type available in -GEOS-Chem, including lat/lon (global or non-global), global standard -cubed-sphere, and global stretched-grid. GCPy also supports arbitrary -vertical regridding across different vertical resolutions. +:program:`GCPy` currently supports regridding of data from: + +#. GEOS-Chem Classic restart files +#. GEOS-Chem Classic diagnostic files +#. GCHP restart files +#. GCHP diagnostic files +#. HEMCO restart files +#. HEMCO diagnostic files +#. As well as any netCDF file adhering to `COARDS + `_ + or `CF `_ conventions. + +Regridding is supported across any horizontal resolution and any grid +type available in GEOS-Chem, including lat/lon (global or non-global), +global standard cubed-sphere, and global stretched-grid. GCPy also +supports arbitrary vertical regridding across different vertical +resolutions. Regridding with GCPy is currently undergoing an overhaul. As of the current release, regridding is split into two different @@ -26,7 +36,7 @@ categories: computed as a preprocessing step. The latter method may be used for creating GCHP standard grid -and stretched grid restart files from either GCHP or GC-Classic +and stretched grid restart files from either GCHP or GEOS-Chem Classic restart files. .. _regrid-classic: @@ -35,20 +45,32 @@ restart files. Using Online Regridding Weights =============================== -You can regrid existing GEOS-Chem Classic restart or output diagnostic files -between lat/lon resolutions using GCPy function :code:`gcpy.file_regrid`. -:code:`gcpy.file_regrid` can either be called directly from the command line -using :code:`python -m gcpy.file_regrid` or as a function -(:code:`gcpy.file_regrid.file_regrid()`) from a Python script or interpreter. +You can regrid existing GEOS-Chem restart or diagnostic files using +GCPy function :code:`gcpy.file_regrid`. This function can called +directly from the command line (:ref:`see the example below +`) function +(:code:`gcpy.file_regrid.file_regrid())`) from a Python script or +interpreter. + The syntax of :code:`file_regrid` is as follows: .. code-block:: python - def file_regrid(fin, fout, dim_format_in, dim_format_out, ll_res_out='0x0'): - """ - Regrids an input file to a new horizontal grid specification and saves it - as a new file. - """ + def file_regrid( + fin, + fout, + dim_format_in, + dim_format_out, + cs_res_out=0, + ll_res_out='0x0', + sg_params_in=[1.0, 170.0, -90.0], + sg_params_out=[1.0, 170.0, -90.0], + vert_params_out=[[], []] + ): + """ + Regrids an input file to a new horizontal grid specification and saves it + as a new file. + """ gcpy.file_regrid required arguments: ------------------------------------ @@ -63,35 +85,71 @@ gcpy.file_regrid required arguments: .. option:: dim_format_in : str - Format of the input file's dimensions (set this to 'classic' - denoting - a GEOS-Chem Classic file with a lat/lon grid) + Format of the input file's dimensions. Accepted values are: + + - :literal:`classic`: For GEOS-Chem Classic restart & diagnostic files + - :literal:`checkpoint` : For GCHP checkpoint & restart files + - :literal:`diagnostic`: For GCHP diagnostic files .. option:: dim_format_out : str - Format of the output file's dimensions (set this to 'classic' - denoting - a GEOS-Chem Classic file with a lat/lon grid) + Format of the output file's dimensions. Accepted values are: + + - :literal:`classic`: For GEOS-Chem Classic restart & diagnostic files + - :literal:`checkpoint` : For GCHP checkpoint & restart files + - :literal:`diagnostic`: For GCHP diagnostic files gcpy.file_regrid optional arguments: ------------------------------------ +.. option:: sg_params_in : list of float + + Stretching parameters (:literal:`stretch-factor`, + :literal:`target-longitude`, :literal:`target-latitude`) for the + input grid. Only needed when the data contained in file + :option:`fin` is on a GCHP stretched grid. + + Default value: :literal:`[1.0, 170.0, -90.0]` + +.. option:: sg_params_out : list of float + + Stretching parameters (:literal:`stretch-factor`, + :literal:`target-longitude`, :literal:`target-latitude`) for the + output grid. Only needed when the data to be contained in file + :option:`fout` is to be placed on a GCHP stretched grid. + + Default value: :literal:`[1.0, 170.0, -90.0]` + +.. option:: cs_res_out : int + + Cubed-sphere resolution of the output dataset. Only needed when + the data in file :option:`fin` is on a GCHP cubed-sphere grid. + + Default value: :code:`0` + .. option:: ll_res_out : str - The lat/lon resolution of the output dataset. + The lat/lon resolution of the output dataset. Only needed when + the data to be contained in file :option:`fout` is to be placed + on a GEOS-Chem Classic lat-lon grid. + + Default value: :code:`"0x0"`. + +.. option:: vert_params_out : list of float + + Hybrid grid parameter :math:`A` (in :literal:`hPa` and :math:`B` + (:literal:`unitless`), returned in list format: :code:`[A, B]` - Default value: '0x0' + Default value: :code:`None` + +.. _regrid-classic-example: Example: -------- -There is now only one grid format supported for regridding files using -the :code:`gcpy.file_regrid` method: :literal:`classic`. You must -specify :literal:`classic` as the value of both :code:`dim_format_in` -and :code:`dim_format_out`, as well as specifying a resolution as the -value of :code:`ll_res_out`. - As stated previously, you can either call -:code:`file_regrid.file_regrid()` directly or call it from the command -line using :code:`python -m gcpy.file_regrid ARGS`. An example +:code:`gcpy.file_regrid.file_regrid()` directly or call it from the +command line using :code:`python -m gcpy.file_regrid ARGS`. An example command line call (separated by line for readability) for regridding a 2x2.5 lat/lon restart file to a 4x5 lat/lon grid looks like: @@ -127,47 +185,13 @@ three stage process: #. Run the regridding operation using the :code:`regrid_restart_file` submodule of GCPy -.. _regrid-gchp-firsttime: - -Python environment for gridspec and sparselt --------------------------------------------- - -Until GCPy contains a complete regridding implementation that works -for all GEOS-Chem grid formats, we recommend that you create a small -`mamba `_ environment in which to -generate regridding weights. - -The following `environment file -`_ -will get you set up with an environment for regridding with -:literal:`gridspec` and :literal:`sparselt`: - -.. code-block:: yaml - - name: gchp_regridding - channels: - - conda-forge - dependencies: - - python=3.9 - - esmf - - gridspec - - numpy - - requests - - sparselt - - xarray - - xesmf - -.. tip:: - - A copy of the above environment file is maintained at the path - :file:`docs/environment/gchp_regridding.yml`. You can create the - environment using command :code:`mamba env create -f - /path/to/gchp_regridding.yml`. - -After installing and switching to this new environment, you -should have the :literal:`gridspec` commands available to you at the -command line. +.. note:: + As of GCPy 1.4.0, the :ref:`default GCPy environment + ` (aka :literal:`gcpy_env`) now contains + :literal:`gridspec` and :literal:`sparselt` packages. You no + longer need to use the separate :literal:`gchp_regridding` + environment as in prior versions. gcpy.regrid_restart_file required arguments: -------------------------------------------- @@ -239,7 +263,12 @@ Example 1: Standard Lat-Lon to Cubed-Sphere Regridding This example will show regridding a GC-Classic 4x5 restart file to a GCHP c24 restart file. -#. Load the :literal:`gchp_regridding` python environment. |br| +#. Activate your GCPy environment. + + .. code-block:: console + + $ mamba activate gcpy_env # Or whatever your environment's name is + |br| #. Create a lat-lon source grid specification using @@ -278,9 +307,6 @@ GCHP c24 restart file. regridding weights, :file:`46x72_to_c24_weights.nc` |br| |br| -#. Switch to your GCPy python environment. |br| - |br| - #. Use the grid weights produced in previous steps to complete the regridding. The first file listed in the command contains the data you wish to regrid and so is a GC-Classic restart file. The second @@ -297,7 +323,14 @@ GCHP c24 restart file. This will produce a single file, :file:`new_restart_file.nc`, regridded from 4x5 to c24, that you can rename and use as you - please. + please. |br| + |br| + +#. Deactivate your GCPy environment when you have finished. + + .. code-block:: console + + $ mamba deactivate Example 2: Standard Cubed-Sphere to Cubed-Sphere Regridding ----------------------------------------------------------- @@ -306,7 +339,12 @@ We will use the example of regridding the out-of-the-box :file:`GEOSChem.Restart.20190701_0000z.c48.nc4` restart file from C48 to C60 to demonstrate the standard cubed-sphere regridding process: -#. Load the :literal:`gchp_regridding` python environment. |br| +#. Activate your GCPy environment. + + .. code-block:: console + + $ mamba activate gcpy_env # Or whatever your environment's name is + |br| #. Create a source grid specification using :code:`gridspec-create`. @@ -344,9 +382,6 @@ C48 to C60 to demonstrate the standard cubed-sphere regridding process: and our regridding weights, :file:`c48_to_c60_weights.nc` |br| |br| -#. Switch to your GCPy python environment. |br| - |br| - #. Use the grid weights produced in earlier steps to complete the regridding. .. code-block:: console @@ -358,7 +393,14 @@ C48 to C60 to demonstrate the standard cubed-sphere regridding process: This will produce a single file, :file:`new_restart_file.nc`, regridded from C48 to C60, that you can rename and use as you - please. + please. |br| + |br| + +#. Deactivate your GCPy environment when you have finished. + + .. code-block:: console + + $ mamba deactivate Example 3: Standard to Stretched Cubed-Sphere Regridding -------------------------------------------------------- @@ -370,7 +412,12 @@ the same at c48. The regridded file will have a stretch factor of 4.0 over Bermuda which means a regional grid resolution of c196 (4 times 48) in that area. -#. Load the :literal:`gchp_regridding` python environment. |br| +#. Activate your GCPy environment: + + .. code-block:: console + + $ mamba activate gcpy_env # Or whatever your environment's name is + |br| #. Create a source grid specification using :code:`gridspec-create`. @@ -403,7 +450,7 @@ times 48) in that area. #. Create the regridding weights for the regridding transformation using :code:`ESMF_RegridWeightGen`, replacing :file:`c48_..._gridspec.nc` with the actual name of the file - created in the previous step. An example is shown below. + created in the previous step. An example is shown below. .. code-block:: console @@ -417,9 +464,6 @@ times 48) in that area. regridding weights, :file:`c48_to_c48_stretched_weights.nc` |br| |br| -#. Switch to your GCPy python environment. |br| - |br| - #. Use the grid weights produced in earlier steps to complete the regridding. @@ -452,6 +496,12 @@ times 48) in that area. same parameters in both the file's global attributes and GCHP configuration file :file:`setCommonRunSettings.sh`. +#. Deactivate your GCPy environment when you have finished. + + .. code-block:: console + + $ mamba deactivate + .. _regrid-plot: =============================== @@ -461,7 +511,7 @@ Regridding for Plotting in GCPy When plotting in GCPy (e.g. through :code:`compare_single_level()` or :code:`compare_zonal_mean()`), the vast majority of regridding is handled internally. You can optionally request a specific -horizontal comparison resolution in :code:`compare_single_level()`` +horizontal comparison resolution in :code:`compare_single_level()` and :code:`compare_zonal_mean()`. Note that all regridding in these plotting functions only applies to the comparison panels (not the top two panels which show data directly from each dataset). There are only diff --git a/gcpy/file_regrid.py b/gcpy/file_regrid.py index e27a5c17..a97dd9df 100644 --- a/gcpy/file_regrid.py +++ b/gcpy/file_regrid.py @@ -1,3 +1,7 @@ +""" +GCPy regridding utility: Regrids data between lat/lon, cubed-sphere, +and cubed-sphere stretched grids. +""" import argparse import os import numpy as np @@ -8,7 +12,7 @@ if LooseVersion(xe.__version__) < LooseVersion("0.2.1"): raise ImportError( "file_regrid.py requires xESMF version 0.2.1 or higher.") -except ImportError as e: +except ImportError as exc: print('file_regrid.py requires xESMF version 0.2.1 or higher!\n\nSee the installation ' + \ 'instructions here: https://xesmf.readthedocs.io/en/latest/installation.html\n') import pandas as pd @@ -16,12 +20,19 @@ from gcpy.grid import get_input_res, get_vert_grid, get_grid_extents from gcpy.regrid import make_regridder_S2S, reformat_dims, make_regridder_L2S, \ make_regridder_C2L, make_regridder_L2L -from gcpy.util import reshape_MAPL_CS +from gcpy.util import reshape_MAPL_CS, verify_variable_type def file_regrid( - fin, fout, dim_format_in, dim_format_out, cs_res_out=0, - ll_res_out='0x0', sg_params_in=[1.0, 170.0, -90.0], - sg_params_out=[1.0, 170.0, -90.0], vert_params_out=[[], []]): + fin, + fout, + dim_format_in, + dim_format_out, + cs_res_out=0, + ll_res_out='0x0', + sg_params_in=None, + sg_params_out=None, + vert_params_out=None +): """ Regrids an input file to a new horizontal grid specification and saves it as a new file. @@ -32,8 +43,8 @@ def file_regrid( fout: str The output filename (file will be overwritten if it already exists) dim_format_in: str - Format of the input file's dimensions (choose from: classic, - checkpoint, diagnostic), where classic denotes lat/lon and + Format of the input file's dimensions (choose from: classic, + checkpoint, diagnostic), where classic denotes lat/lon and checkpoint / diagnostic are cubed-sphere formats dim_format_out: str Format of the output file's dimensions (choose from: classic, @@ -42,20 +53,20 @@ def file_regrid( Keyword Args (optional): cs_res_out: int - The cubed-sphere resolution of the output dataset. + The cubed-sphere resolution of the output dataset. Not used if dim_format_out is classic Default value: 0 ll_res_out: str - The lat/lon resolution of the output dataset. + The lat/lon resolution of the output dataset. Not used if dim_format_out is not classic Default value: '0x0' sg_params_in: list[float, float, float] - Input grid stretching parameters + Input grid stretching parameters [stretch-factor, target longitude, target latitude]. Not used if dim_format_in is classic Default value: [1.0, 170.0, -90.0] (No stretching) sg_params_out: list[float, float, float] - Output grid stretching parameters + Output grid stretching parameters [stretch-factor, target longitude, target latitude]. Not used if dim_format_out is classic Default value: [1.0, 170.0, -90.0] (No stretching) @@ -65,6 +76,18 @@ def file_regrid( Default value: [[], []] """ + verify_variable_type(fin, str) + verify_variable_type(fout, str) + verify_variable_type(dim_format_in, str) + verify_variable_type(dim_format_out, str) + + # Assign default values for optional keywords + if sg_params_in is None: + sg_params_in = [1.0, 170.0, -90.0] + if sg_params_out is None: + sg_params_out = [1.0, 170.0, -90.0] + if vert_params_out is None: + vert_params_out = [[], []] # Load dataset ds_in = xr.open_dataset(fin, decode_cf=False) @@ -111,11 +134,11 @@ def file_regrid( tlat_in=sg_params_in[2], sf_out=sg_params_out[0], tlon_out=sg_params_out[1], - tlat_out=sg_params_out[2]) + tlat_out=sg_params_out[2] + ) # Save temporary output face files to minimize RAM usage oface_files = [os.path.join('.',fout+str(x)) for x in range(6)] # For each output face, sum regridded input faces - oface_datasets = [] for oface in range(6): oface_regridded = [] for iface, regridder in regridders[oface].items(): @@ -132,7 +155,12 @@ def file_regrid( oface_files[oface], format='NETCDF4_CLASSIC' ) - ds_out=xr.open_mfdataset(oface_files, combine='by_coords', concat_dim='F',engine='netcdf4') + ds_out=xr.open_mfdataset( + oface_files, + combine='by_coords', + concat_dim='F', + engine='netcdf4' + ) # Put regridded dataset back into a familiar format ds_out = ds_out.rename({ 'y': 'Y', @@ -161,11 +189,11 @@ def file_regrid( if dim_format_out == 'checkpoint': # convert to checkpoint format ds_out = reshape_MAPL_CS(ds_out) - mi = pd.MultiIndex.from_product([ + mindex = pd.MultiIndex.from_product([ np.linspace(1, 6, 6), np.linspace(1, cs_res_out, cs_res_out) ]) - ds_out = ds_out.assign_coords({'lat': mi}) + ds_out = ds_out.assign_coords({'lat': mindex}) ds_out = ds_out.unstack('lat') ds_out = ds_out.stack(lat=['lat_level_0', 'lat_level_1']) @@ -268,15 +296,16 @@ def file_regrid( # Print the resulting dataset print(ds_out) # Remove any temporary files - for f in oface_files: os.remove(f) + for oface in oface_files: + os.remove(oface) -def rename_restart_variables(ds, towards_gchp=True): +def rename_restart_variables(dset, towards_gchp=True): """ Renames restart variables according to GEOS-Chem Classic and GCHP conventions. Args: - ds: xarray.Dataset + dset: xarray.Dataset The input dataset Keyword Args (optional): @@ -295,12 +324,12 @@ def rename_restart_variables(ds, towards_gchp=True): else: old_str = 'SPC' new_str = 'SpeciesRst' - return ds.rename({name: name.replace(old_str, new_str, 1) - for name in list(ds.data_vars) + return dset.rename({name: name.replace(old_str, new_str, 1) + for name in list(dset.data_vars) if name.startswith(old_str)}) -def drop_and_rename_classic_vars(ds, towards_gchp=True): +def drop_and_rename_classic_vars(dset, towards_gchp=True): """ Renames and drops certain restart variables according to GEOS-Chem Classic and GCHP conventions. @@ -320,75 +349,133 @@ def drop_and_rename_classic_vars(ds, towards_gchp=True): """ if towards_gchp: - ds = ds.rename( + dset = dset.rename( {name: name.replace('Met_', '', 1).replace('Chem_', '', 1) - for name in list(ds.data_vars) + for name in list(dset.data_vars) if name.startswith('Met_') or name.startswith('Chem_')}) - if 'DELPDRY' in list(ds.data_vars): ds = ds.rename({'DELPDRY': 'DELP_DRY'}) - ds = ds.drop_vars(['P0', - 'hyam', - 'hybm', - 'hyai', - 'hybi', - 'AREA', - 'ilev', - 'PS1DRY', - 'PS1WET', - 'TMPU1', - 'SPHU1', - 'StatePSC'], - errors='ignore') + if 'DELPDRY' in list(dset.data_vars): + dset = dset.rename({'DELPDRY': 'DELP_DRY'}) + dset = dset.drop_vars( + ['P0', + 'hyam', + 'hybm', + 'hyai', + 'hybi', + 'AREA', + 'ilev', + 'PS1DRY', + 'PS1WET', + 'TMPU1', + 'SPHU1', + 'StatePSC'], + errors='ignore' + ) else: - renames = {'DELP_DRY': 'Met_DELPDRY', - 'BXHEIGHT': 'Met_BXHEIGHT', - 'TropLev': 'Met_TropLev', - 'DryDepNitrogen': 'Chem_DryDepNitrogen', - 'WetDepNitrogen': 'Chem_WetDepNitrogen', - 'H2O2AfterChem': 'Chem_H2O2AfterChem', - 'SO2AfterChem': 'Chem_SO2AfterChem', - 'KPPHvalue': 'Chem_KPPHvalue'} - data_vars = list(ds.data_vars) + renames = { + 'DELP_DRY': 'Met_DELPDRY', + 'BXHEIGHT': 'Met_BXHEIGHT', + 'TropLev': 'Met_TropLev', + 'DryDepNitrogen': 'Chem_DryDepNitrogen', + 'WetDepNitrogen': 'Chem_WetDepNitrogen', + 'H2O2AfterChem': 'Chem_H2O2AfterChem', + 'SO2AfterChem': 'Chem_SO2AfterChem', + 'KPPHvalue': 'Chem_KPPHvalue' + } + data_vars = list(dset.data_vars) new_renames = renames.copy() for key in renames.keys(): if key not in data_vars: del(new_renames[key]) - ds = ds.rename(new_renames) + dset = dset.rename(new_renames) - return rename_restart_variables(ds, towards_gchp=towards_gchp) + return rename_restart_variables(dset, towards_gchp=towards_gchp) -if __name__ == '__main__': +def main(): + """ + Main program for file_regrid. Parses command-line arguments and + calls the file_regrid routine. + + Command-line arguments: + ----------------------- + -i, --filein + Input file, contains original data. + + -o --fileout + Output file, contains regridded data. + + --sg-params-in + Input grid stretching parameters (GCHP only). + + --sg-params-out + Output grid stretching parameters (GCHP only). + + --dim-format-in + Format of the input file's dimensions: + ("checkpoint", "diagnostics". "classic") + + --dim-format-out + Format of the output file's dimensions: + ("checkpoint", "diagnostics", "classic") + + --cs_res_out + Cubed-sphere resolution for the output file (e.g 24, 48, 360) + + --ll_res_out + Resolution for the output file in 'latxlon` format + + --vert_params_out + Hybrid grid parameter A in hPa and B (unitless) in [AP, BP] format + """ + + # Parse arguments from the command line parser = argparse.ArgumentParser( - description='General cubed-sphere to cubed-sphere regridder.') - parser.add_argument('-i', '--filein', - metavar='FIN', - type=str, - required=True, - help='input NetCDF file') - parser.add_argument('-o', '--fileout', - metavar='FOUT', - type=str, - required=True, - help='name of output file') + description='General cubed-sphere to cubed-sphere regridder.' + ) + parser.add_argument( + '-i', '--filein', + metavar='FIN', + type=str, + required=True, + help='input NetCDF file' + ) parser.add_argument( - '--sg_params_in', metavar='P', type=float, nargs=3, + '-o', '--fileout', + metavar='FOUT', + type=str, + required=True, + help='name of output file' + ) + parser.add_argument( + '--sg_params_in', + metavar='P', + type=float, + nargs=3, default=[1.0, 170.0, -90.0], - help='input grid stretching parameters (stretch-factor, target longitude, target latitude)') + help='input grid stretching parameters (stretch-factor, target longitude, target latitude)' + ) parser.add_argument( - '--sg_params_out', metavar='P', type=float, nargs=3, + '--sg_params_out', + metavar='P', + type=float, + nargs=3, default=[1.0, 170.0, -90.0], - help='output grid stretching parameters (stretch-factor, target longitude, target latitude)') - parser.add_argument('--cs_res_out', - metavar='RES', - type=int, - required=False, - help='output grid\'s cubed-sphere resolution') + help='output grid stretching parameters (stretch-factor, target longitude, target latitude)' + ) + parser.add_argument( + '--cs_res_out', + metavar='RES', + type=int, + required=False, + help='output grid\'s cubed-sphere resolution' + ) parser.add_argument( '--ll_res_out', metavar='RES', type=str, required=False, - help='output grid\'s lat/lon resolution in \'latxlon\' format') + help='output grid\'s lat/lon resolution in \'latxlon\' format' + ) parser.add_argument( '--dim_format_in', metavar='WHICH', @@ -398,7 +485,8 @@ def drop_and_rename_classic_vars(ds, towards_gchp=True): 'diagnostic', 'classic'], required=True, - help='format of the input file\'s dimensions (choose from: checkpoint, diagnostic)') + help='format of the input file\'s dimensions (choose from: checkpoint, diagnostic)' + ) parser.add_argument( '--dim_format_out', metavar='WHICH', @@ -408,15 +496,18 @@ def drop_and_rename_classic_vars(ds, towards_gchp=True): 'diagnostic', 'classic'], required=True, - help='format of the output file\'s dimensions (choose from: checkpoint, diagnostic)') + help='format of the output file\'s dimensions (choose from: checkpoint, diagnostic)' + ) parser.add_argument( '--vert_params_out', metavar='VERT', type=list, required=False, - help='Hybrid grid parameter A in hPa and B (unitless) in [AP, BP] format') - + help='Hybrid grid parameter A in hPa and B (unitless) in [AP, BP] format' + ) args = parser.parse_args() + + # Regrid the file file_regrid( args.filein, args.fileout, @@ -427,3 +518,8 @@ def drop_and_rename_classic_vars(ds, towards_gchp=True): args.sg_params_in, args.sg_params_out, args.vert_params_out) + + +# Only call when run as standalone +if __name__ == '__main__': + main() From 5d51aadc9d18b74901445036ea85332eef7c2835 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 15:44:04 -0400 Subject: [PATCH 09/11] Fixed broken link in README.md The link to HEMCO ended with ] intead of ). Now fixed. Signed-off-by: Bob Yantosca --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f609bcd3..010e3098 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ 3. Obtain GEOS-Chem's horizontal and vertical grid information. 4. Implement [GCHP](https://gchp.readthedocs.io)-specific regridding functionalities (e.g. cubed-sphere to lat-lon regridding) 5. Provide example scripts for creating specific types of plots or analysis from GEOS-Chem output. -6. Provide user-submitted scripts for specific applications related to GEOS-Chem and [HEMCO](https://hemco.readthedocs.io]. +6. Provide user-submitted scripts for specific applications related to GEOS-Chem and [HEMCO](https://hemco.readthedocs.io). ## What GCPy was not intended to do: From b065cdb97f73fd750315f552875c56035fb308b6 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 15:46:16 -0400 Subject: [PATCH 10/11] Update CHANGELOG.md for updates to gcpy/file_regrid.py gcpy/file_regrid.py - Added blurb about new main() function and updates suggested by Pylint Signed-off-by: Bob Yantosca --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9153e068..44669a00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - Construct ops budget table filename without using the `label` argument - Updated species_database.yml for consistency with GEOS-Chem 14.2.0 - Renamed TransportTracers species in `benchmark_categories.yml` and in documentation +- Added `main()` routine to `gcpy/file_regrid.py`; Also added updates suggested by Pylint ### Removed - Removed `gchp_is_pre_13_1` arguments & code from benchmarking routines From 21bc00a2cde48219fbf49bb266da7a4491139ae5 Mon Sep 17 00:00:00 2001 From: Bob Yantosca Date: Mon, 21 Aug 2023 16:04:00 -0400 Subject: [PATCH 11/11] Update gcpy.file_regrid example in Regridding.rst docs/source/Regridding.rst - Update example to show 4x5 to c24 regridding Signed-off-by: Bob Yantosca --- docs/source/Regridding.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/Regridding.rst b/docs/source/Regridding.rst index 3a087732..e964acbd 100644 --- a/docs/source/Regridding.rst +++ b/docs/source/Regridding.rst @@ -155,12 +155,12 @@ command line call (separated by line for readability) for regridding a .. code-block:: - python -m gcpy.file_regrid \ - --filein initial_GEOSChem_rst.2x2.5.nc \ - --dim_format_in classic \ - --fileout GEOSChem_rst.4x5.nc \ - --ll_res_out 4x5 \ - --dim_format_out classic + $ python -m gcpy.file_regrid \ + --filein GEOSChem.Restart.20190701_0000z.nc4 \ + --dim_format_in classic \ + --fileout GEOSChem.Restart.20190701_0000z.c24.nc4 \ + --cs_res_out 24 \ + --dim_format_out checkpoint .. _regrid-gchp: