Feature #2769 tc_diag_driver_v0.11.0

.github/jobs/set_job_controls.sh

@@ -6,7 +6,7 @@ run_unit_tests=false
 run_diff=false
 run_update_truth=false
 met_base_repo=met-base
-met_base_tag=v3.1
+met_base_tag=v3.2
 input_data_version=develop
 truth_data_version=develop
 

.github/workflows/build_docker_and_trigger_metplus.yml

@@ -29,7 +29,7 @@ jobs:
         env:
           SOURCE_BRANCH: ${{ steps.get_branch_name.outputs.branch_name }}-lite
           MET_BASE_REPO: met-base
-          MET_BASE_TAG: v3.1
+          MET_BASE_TAG: v3.2
 
       - name: Push Docker Image
         run: .github/jobs/push_docker_image.sh

data/config/TCDiagConfig_default

@@ -85,8 +85,8 @@ domain_info = [
 // May be set separately in each data "field" array entry
 //
 regrid = {
-   method     = NEAREST;
-   width      = 1;
+   method     = BILIN;
+   width      = 2;
    vld_thresh = 0.5;
    shape      = SQUARE;
 }

docs/Users_Guide/appendixF.rst

@@ -22,15 +22,19 @@ In order to use Python embedding, a local Python installation must be available
 
 3. **NumPy** Python package
 
-4. **netCDF4** Python package
+4. **Pandas** Python package
 
-5. **Pandas** Python package
+5. **Xarray** Python package
 
-6. **Xarray** Python package
+6. **YAML** Python package
 
-7. **YAML** Python package
+7. **SciPy** Python package
 
-Users should be aware that in some cases, the C-language Python header files and libraries may be deleted at the end of the Python installation process, and they may need to confirm their availability prior to compiling MET. Once the user has confirmed the above requirements are satisfied, they can compile the MET software for Python embedding by passing the **\-\-enable-python** option to the **configure** script on the command line. This will link the MET C++ code directly to the Python libraries. The **NumPy** and **netCDF4** Python packages are required by the Python scripts included with the MET software that facilitate the passing of data in memory and the reading and writing of temporary files when Python embedding is used. The **YAML** Python package is required by the tropical cyclone diagnostics Python scripts called by the TC-Diag tool.
+8. **netCDF4** Python package
+
+Users should be aware that in some cases, the C-language Python header files and libraries may be deleted at the end of the Python installation process, and they may need to confirm their availability prior to compiling MET. Once the user has confirmed the above requirements are satisfied, they can compile the MET software for Python embedding by passing the **\-\-enable-python** option to the **configure** script on the command line. This will link the MET C++ code directly to the Python libraries.
+
+The **NumPy**, **Xarray**, and **Pandas** Python packages are required by the Python scripts included with the MET software that facilitate the passing of data in memory. The *SciPy** and **YAML** Python packages are required by the tropical cyclone diagnostics Python scripts called by the TC-Diag tool. The **netCDF4** package is used for reading and writing temporary files for Python embedding, but only when the **MET_PYTHON_TMP_FORMAT** environment variable is set to `netcdf` at runtime. 
 
 In addition to using **\-\-enable-python** with **configure** as mentioned above, the following environment variables must also be set prior to executing **configure**: **MET_PYTHON_BIN_EXE**, **MET_PYTHON_CC**, and **MET_PYTHON_LD**. These may either be set as environment variables or as command line options to **configure**. These environment variables are used when building MET to enable the compiler to find the requisite Python executable, header files, and libraries in the user's local filesystem. Fortunately, Python provides a way to set these variables properly. This frees the user from the necessity of having any expert knowledge of the compiling and linking process. Along with the **Python** executable in the users local Python installation, there should be another executable called **python3-config**, whose output can be used to set these environment variables as follows:
 

docs/Users_Guide/release-notes.rst

@@ -9,11 +9,12 @@ When applicable, release notes are followed by the GitHub issue number which des
 enhancement, or new feature (`MET GitHub issues <https://github.com/dtcenter/MET/issues>`_).
 Important issues are listed **in bold** for emphasis.
 
-MET Version 12.0.0-beta3 Release Notes (20240206)
+MET Version 12.0.0-beta3 Release Notes (20240207)
 -------------------------------------------------
 
   .. dropdown:: Repository, build, and test
 
+     * Enhance METbaseimage to install SciPy Python package needed by the MET TC-Diag tool (`METbaseimage#20 <https://github.com/dtcenter/METbaseimage/issues/20>`_).
      * Remove the SonarQube token from the properties file (`#2757 <https://github.com/dtcenter/MET/issues/2757>`_).
      * Repository cleanup of stale code and configuration consistency (`#2776 <https://github.com/dtcenter/MET/issues/2776>`_).
      * Add new example installation configuration files for Intel compiler users (`#2785 <https://github.com/dtcenter/MET/issues/2785>`_).

docs/Users_Guide/tc-diag.rst

@@ -63,6 +63,8 @@ Optional Arguments for tc_diag
 
 6. The **-v level** option indicates the desired level of verbosity. The contents of "level" will override the default setting of 2. Setting the verbosity to 0 will make the tool run with no log messages, while increasing the verbosity above 1 will increase the amount of logging.
 
+.. note:: Setting the **MET_KEEP_TEMP_FILE** (:numref:`met_keep_temp_file`) environment variable retains the temporary NetCDF cylindrical coordinate files for development, testing, and debugging purposes.
+
 tc_diag Configuration File
 --------------------------
 
@@ -155,13 +157,13 @@ Configuring regridding options
 .. code-block:: none
 
   regrid = {
-     method     = NEAREST;
-     width      = 1;
+     method     = BILIN;
+     width      = 2;
      vld_thresh = 0.5;
      shape      = SQUARE;
   }
 
-The **regrid** dictionary is common to multiple MET tools and is described in :numref:`config_options`. It specifies how the input data should be regridded to cylindrical coordinates prior to compute diagnostics. It can be specified separately in each **data.field** array entry, described below. The default setting uses nearest neighbor interpolation for all fields.
+The **regrid** dictionary is common to multiple MET tools and is described in :numref:`config_options`. It specifies how the input data should be regridded to cylindrical coordinates prior to compute diagnostics. It can be specified separately in each **data.field** array entry, described below. The default setting uses bilinear interpolation for all fields.
 
 Configuring Fields, Levels, and Domains
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

docs/conf.py

@@ -24,7 +24,7 @@
 verinfo = version
 release = f'{version}'
 release_year = '2024'
-release_date = f'{release_year}-02-06'
+release_date = f'{release_year}-02-07'
 copyright = f'{release_year}, {author}'
 
 # -- General configuration ---------------------------------------------------

internal/scripts/docker/Dockerfile

@@ -1,5 +1,5 @@
 ARG MET_BASE_REPO=met-base
-ARG MET_BASE_TAG=v3.1
+ARG MET_BASE_TAG=v3.2
 
 FROM dtcenter/${MET_BASE_REPO}:${MET_BASE_TAG}
 MAINTAINER John Halley Gotway <johnhg@ucar.edu>

internal/scripts/docker/Dockerfile.copy

@@ -1,5 +1,5 @@
 ARG MET_BASE_REPO=met-base-unit-test
-ARG MET_BASE_TAG=v3.1
+ARG MET_BASE_TAG=v3.2
 
 FROM dtcenter/${MET_BASE_REPO}:${MET_BASE_TAG}
 MAINTAINER John Halley Gotway <johnhg@ucar.edu>

internal/test_unit/config/TCDiagConfig_ian

@@ -77,8 +77,8 @@ domain_info = [
 // May be set separately in each data "field" array entry
 //
 regrid = {
-   method     = NEAREST;
-   width      = 1;
+   method     = BILIN;
+   width      = 2;
    vld_thresh = 0.5;
    shape      = SQUARE;
 }

scripts/python/tc_diag/compute_tc_diag.py

@@ -85,6 +85,19 @@ def main():
     land_filename   = os.path.expandvars(args.land_file)
     data_filename   = os.path.expandvars(args.data_file)
 
+    # Validate inputs
+    if not os.path.isfile(config_filename):
+        print("Error: Input Config File (" + config_filename + ") does not exist!")
+        sys.exit(1)
+
+    if not os.path.isfile(land_filename):
+        print("Error: Input Land File (" + land_filename + ")does not exist!")
+        sys.exit(1)
+
+    if not os.path.isfile(data_filename):
+        print("Error: Input Data File (" + data_filename + ") does not exist!")
+        sys.exit(1)
+
     # Print verbose arguments
     if args.verbose:
         print("Python Script:\t" + os.path.expandvars(sys.argv[0]))

scripts/python/tc_diag/config/post_resample.yml

@@ -100,23 +100,25 @@ pressure_independent_computation_specs:
     callable: *div_vort_func 
     output_vars: [850DVRG, 850VORT] 
     kwargs: {u_name: *u_vert_name, v_name: *v_vert_name, level_hPa: 850, radius_km: *div_vort_max_km}
+    units: [/S, /S]
   div_vort_200: 
     callable: *div_vort_func
     output_vars: [200DVRG, 200VORT] 
     kwargs: {u_name: *u_vert_name, v_name: *v_vert_name, level_hPa: 200, radius_km: *div_vort_max_km}
+    units: [/S, /S]
 
 
   shear: 
     batch_order: 1
     callable: tc_diag_driver.met_diag_vars.shear 
     output_vars: [SHR_MAG, SHR_HDG] 
-    kwargs: {u_name: u, v_name: v, bottom_hPa: 850, top_hPa: 200}
+    kwargs: {u_name: U, v_name: V, bottom_hPa: 850, top_hPa: 200}
     unit_converters: [tc_diag_driver.met_post_process.mps_to_kt, pass]
     units: [KT, DEG]
   TGRD:
     batch_order: 1
     callable: tc_diag_driver.met_diag_vars.temperature_gradient
-    kwargs: {u_name: u, v_name: v, bottom_hPa: 850, top_hPa: 700}
+    kwargs: {u_name: U, v_name: V, bottom_hPa: 850, top_hPa: 700}
     units: "10^7C/M"
 
 sounding_computation_specs:
@@ -161,4 +163,4 @@ comment_format: |
   *   850VORT, 200DVRG averaged from {div_vort_min}-{div_vort_max} km around storm center [x10^7 /s, x10^7 /s]   *
   *   850TANG averaged from {rad_tan_min}-{rad_tan_max} km around storm center [x10 m/s]   *
   *   T, R, Z, P averaged from {therm_min}-{therm_max} km around storm center [x10 C, %, dm, mb]   *
-  *   TPW averaged from {tpw_min}-{tpw_max} km around storm center [mm]   *
+  *   TPW averaged from {tpw_min}-{tpw_max} km around storm center [mm]   *

scripts/python/tc_diag/config/post_resample_nest.yml

@@ -100,23 +100,25 @@ pressure_independent_computation_specs:
     callable: *div_vort_func 
     output_vars: [850DVRG, 850VORT] 
     kwargs: {u_name: *u_vert_name, v_name: *v_vert_name, level_hPa: 850, radius_km: *div_vort_max_km}
+    units: [/S, /S]
   div_vort_200: 
     callable: *div_vort_func
     output_vars: [200DVRG, 200VORT] 
     kwargs: {u_name: *u_vert_name, v_name: *v_vert_name, level_hPa: 200, radius_km: *div_vort_max_km}
+    units: [/S, /S]
 
 
   shear: 
     batch_order: 1
     callable: tc_diag_driver.met_diag_vars.shear 
     output_vars: [SHR_MAG, SHR_HDG] 
-    kwargs: {u_name: u, v_name: v, bottom_hPa: 850, top_hPa: 200}
+    kwargs: {u_name: U, v_name: V, bottom_hPa: 850, top_hPa: 200}
     unit_converters: [tc_diag_driver.met_post_process.mps_to_kt, pass]
     units: [KT, DEG]
   TGRD:
     batch_order: 1
     callable: tc_diag_driver.met_diag_vars.temperature_gradient
-    kwargs: {u_name: u, v_name: v, bottom_hPa: 850, top_hPa: 700}
+    kwargs: {u_name: U, v_name: V, bottom_hPa: 850, top_hPa: 700}
     units: "10^7C/M"
 
 sounding_computation_specs:
@@ -161,4 +163,4 @@ comment_format: |
   *   850VORT, 200DVRG averaged from {div_vort_min}-{div_vort_max} km around storm center [x10^7 /s, x10^7 /s]   *
   *   850TANG averaged from {rad_tan_min}-{rad_tan_max} km around storm center [x10 m/s]   *
   *   T, R, Z, P averaged from {therm_min}-{therm_max} km around storm center [x10 C, %, dm, mb]   *
-  *   TPW averaged from {tpw_min}-{tpw_max} km around storm center [mm]   *
+  *   TPW averaged from {tpw_min}-{tpw_max} km around storm center [mm]   *

scripts/python/tc_diag/tc_diag_driver/__init__.py

@@ -1 +1 @@
-__version__ = "0.9.0"
+__version__ = "0.11.0"

scripts/python/tc_diag/tc_diag_driver/computation_engine.py

@@ -251,6 +251,21 @@ def get_result_names(computations: List[DiagComputation]) -> List[str]:
     return names
 
 
+def get_all_result_units(
+    pressure_indedpendent: List[DiagComputation], sounding: List[DiagComputation]
+) -> Tuple[List[str], List[str]]:
+    pi_var_units = get_result_units(pressure_indedpendent)
+    snd_var_units = get_result_units(sounding)
+    return pi_var_units, snd_var_units
+
+
+def get_result_units(computations: List[DiagComputation]) -> List[str]:
+    units = []
+    for c in computations:
+        units.extend(c.units)
+    return units
+
+
 def get_computation_batches(
     pressure_indedpendent: List[DiagComputation], sounding: List[DiagComputation]
 ) -> List[ComputationBatch]: