Update `development.seneca` to compile MET after upgrading seneca to Debian Bookworm #2761

JohnHalleyGotway · 2023-12-11T20:52:36Z

Describe the Task

On Friday, 12/8/23, RAL-IT upgraded the RAL/JNT METplus development machine to Debian Bookworm. This issue is to update the contents of development.seneca as needed to enable MET to compile in new operating system. Note that changes should be made to both the develop, main_v11.1, and main_v11.0 branches since nightly build regression tests are run for all three versions on seneca.

Note, as part of this issue, please also recompile earlier MET releases in seneca:/d1/projects/MET/MET_releases. For releases already compiled in /nrit/ral, just provide a soft-link to them in that directory.

Time Estimate

1 day.

Sub-Issues

Consider breaking the task down into sub-issues.
None needed

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

Select engineer(s) or no engineer required
Select scientist(s) or no scientist required

Labels

Review default alert labels
Select component(s)
Select priority
Select requestor(s)

Milestone and Projects

Select Milestone as the next official version or Backlog of Development Ideas
For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
No impacts

Task Checklist

See the METplus Workflow for details.

The text was updated successfully, but these errors were encountered:

…n RAL-IT compilation settings described in /nrit/ral/src/met/README.ral-it.

…version used by Debian Bookworm on our devleopment machine, seneca.

….0 branch.

…eca to debian bookworm.

JohnHalleyGotway · 2023-12-12T20:02:08Z

Note that I somewhat inadvertently updated the main_v11.0 and main_v11.1 branches with the updated version of development.seneca. These should have technically gone through a pull request instead. However, I did test by manually triggering the nightly builds on seneca in:

/d1/projects/MET/MET_regression/main_v11.0/NB20231212
/d1/projects/MET/MET_regression/main_v11.1/NB20231212

And I will monitor their progress to confirm that they run as expected.

…g the default version which no longer exists in /usr/local

… ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion.

… ncdiff.

…2770) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: Stephen Herbener <32968781+srherbener@users.noreply.github.com> fix #2389 main_v11.0 flowchart (#2391) fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405) fix #2380 main_v11.0 override (#2381) fix #2408 main_v11.0 empty config (#2409) fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403) fix #2415 main_v11.0 modulefiles (#2416) fix #2412 main_v11.0 climo (#2420) fix #2426 main_v11.0 buoy (#2432) fix #2437 main_v11.0 convert (#2438) fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup. fix 2428 python from env main v11.0 (#2443) fix 2428 python csv input (#2450) fix #2452 main_v11.0 airnow (#2453) fix #2402 main_v11.0 sonarqube (First PR) (#2447) fix #2449 main_v11.0 pdf (#2465) fix 2428 python csv input (#2467) fix #2514 main_v11.0 clang (#2556) fix #2575 main_v11.0 python_convert (#2577) fix 2596 main v11.0 rpath compilation (#2600) fix 2596 main v11.0 rpath compilation (#2613) fix #2644 main_v11.0 percentile (#2645)

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

@anewman89

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

@anewman89

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2753 comp script config (#2771) * set dynamic library file extension to .dylib if running on MacOS and .so otherwise * Added disabling of jasper documentation for compiliation on Hera * Updated * remove extra export of compiler env vars * include full path to log file so it is easier to file the log file to examine when a command fails * send cmake output to a log file * remove redundant semi-colon * use full path to log file so it is easier to examine on failure * use run_cmd to catch if rm command fails * Modifications for compilation on hera, gaea, and orion * Updating * fixed variable name * clean up if/else statements * set TIFF_LIBRARY_RELEASE argument to use full path to dynamic library file to prevent failure installing proj library * set LDFLAGS so that LDFLAGS value set in the user's environment will also be used * Updated based on gaea, orion, and hera installs * Updated * change extension of dynamic library files only if architecture is arm64 because older Macs still use .so * added netcdf library to args to prevent error installing NetCDF-CXX when PROJ has been installed in the same run of the script -- PATH is set in the COMPILE_PROJ if block that causes this flag from being added automatically * clean up how rpath and -L are added to LDFLAGS so that each entry is separate -- prevents errors installing on Mac arm64 because multiple rpath values aren't read using :. Also use MET_PROJLIB * Updated * removed -ltiff from MET libs * only add path to rpath and -L arguments if they are not already included in LDFLAGS * changed from using LIB_TIFF (full path to tiff lib file) to use TIFF_LIB_DIR (dir containing tiff lib file). Added TIFF_INCLUDE_DIR to proj compilation and -DJAS_ENABLE_DOC to jasper compliation taken from @jprestop branch * update comments * ensure all MET_* and MET_*LIB variables are added to the rpath for consistency * remove unnecessary if block and only export LDFLAGS at the end of setting locally * Updated * Added section for adding <VALUE>/lib64 and rearranged placement of ADDTL_DIR * Commenting out the running of the Jasper lib tests * Updating and/or removing files * Updating and/or removing files * Latest udpates which include the addition of the tiff library for proj * Remove commented out line. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Feature #2547 Read WRF output files natively (#2790) * Per #2547, add the file_type = NETCDF_WRF configuration option. * Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf. * Per #2547, rename nccf files as nc_cf for consistency throughout * Per #2547, rename Pinterp classes as Wrf * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * ignore directory automatically created by CLion * rename pinterp variables and classes to Wrf * more rename and fix typos * per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension * per #2547, add DataPlane function to handle staggering of grid * per #2547, change file type returned from pinterp to wrf * per #2547, call function to stagger DataPlane if necessary * per #2547, keep store if variable is on pressure levels and only read pressure field if it is * ignore cmake directory * per #2547, support U_PL, U_ZL, U, and U<x> where <x> is an integer in is_u_wind and the same for is_v_wind * change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V * error and exit if var with unsupported subgrid dimension is processed * clean up logging * only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info * Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected * per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag) * per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat * merged develop and resolved conflicts * rename lib from pinterp to wrf * rename lib * fixed incorrect merge changes * fixed broken unit test format * Per #2547, restore the mention of NAK.poly to Appendix B. * Change model to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * change obtype to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * plot higher vertical level to get non-zero output Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * indentation Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

@anewman89

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * #2755 Added a header count and checking header count instead of using header id (hid) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2753 comp script config (#2771) * set dynamic library file extension to .dylib if running on MacOS and .so otherwise * Added disabling of jasper documentation for compiliation on Hera * Updated * remove extra export of compiler env vars * include full path to log file so it is easier to file the log file to examine when a command fails * send cmake output to a log file * remove redundant semi-colon * use full path to log file so it is easier to examine on failure * use run_cmd to catch if rm command fails * Modifications for compilation on hera, gaea, and orion * Updating * fixed variable name * clean up if/else statements * set TIFF_LIBRARY_RELEASE argument to use full path to dynamic library file to prevent failure installing proj library * set LDFLAGS so that LDFLAGS value set in the user's environment will also be used * Updated based on gaea, orion, and hera installs * Updated * change extension of dynamic library files only if architecture is arm64 because older Macs still use .so * added netcdf library to args to prevent error installing NetCDF-CXX when PROJ has been installed in the same run of the script -- PATH is set in the COMPILE_PROJ if block that causes this flag from being added automatically * clean up how rpath and -L are added to LDFLAGS so that each entry is separate -- prevents errors installing on Mac arm64 because multiple rpath values aren't read using :. Also use MET_PROJLIB * Updated * removed -ltiff from MET libs * only add path to rpath and -L arguments if they are not already included in LDFLAGS * changed from using LIB_TIFF (full path to tiff lib file) to use TIFF_LIB_DIR (dir containing tiff lib file). Added TIFF_INCLUDE_DIR to proj compilation and -DJAS_ENABLE_DOC to jasper compliation taken from @jprestop branch * update comments * ensure all MET_* and MET_*LIB variables are added to the rpath for consistency * remove unnecessary if block and only export LDFLAGS at the end of setting locally * Updated * Added section for adding <VALUE>/lib64 and rearranged placement of ADDTL_DIR * Commenting out the running of the Jasper lib tests * Updating and/or removing files * Updating and/or removing files * Latest udpates which include the addition of the tiff library for proj * Remove commented out line. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2697 Share the temporary file for blocking * #2697 Reduced the temporary files for pb2nc * Moved typedef unixtime from vx_cal.h to time_array.h * #2697 Changed static const to constexpr for SonarQube (code smell) * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * #2673 Removed unused variable grid_x and grid_y * #2673 Renamed the shadowed variable n to nd * #2673 Use nullptr instead of literal 0 * #2673 Moved down namespace declarations. Removed unused variables * #2673 Reduced the scope of variables * #2673 Use nullptr instead of literal 0 * #2673 Removed unused variables * #2673 Renamed ex to ex2 which becomes shadowed variable * #2673 Removed always true condition * #2673 Moved down namespace declarations. Use nullptr instead of literal 0 * #2673 Use nullptr instead of literal 0. Removed unused variables * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * #2673 Removed unsued variables. Check nullptr of gt * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * Feature #2547 Read WRF output files natively (#2790) * Per #2547, add the file_type = NETCDF_WRF configuration option. * Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf. * Per #2547, rename nccf files as nc_cf for consistency throughout * Per #2547, rename Pinterp classes as Wrf * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * ignore directory automatically created by CLion * rename pinterp variables and classes to Wrf * more rename and fix typos * per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension * per #2547, add DataPlane function to handle staggering of grid * per #2547, change file type returned from pinterp to wrf * per #2547, call function to stagger DataPlane if necessary * per #2547, keep store if variable is on pressure levels and only read pressure field if it is * ignore cmake directory * per #2547, support U_PL, U_ZL, U, and U<x> where <x> is an integer in is_u_wind and the same for is_v_wind * change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V * error and exit if var with unsupported subgrid dimension is processed * clean up logging * only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info * Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected * per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag) * per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat * merged develop and resolved conflicts * rename lib from pinterp to wrf * rename lib * fixed incorrect merge changes * fixed broken unit test format * Per #2547, restore the mention of NAK.poly to Appendix B. * Change model to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * change obtype to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * plot higher vertical level to get non-zero output Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * indentation Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Hotfix to the develop branch after PR #2790 merged the feature_2547_wrf branch into develop. Rerunning bootstrap on the project machine seneca produces 2 small differences that should also be incorporated into develop. * Feature 2588 install rewrite (#2791) * updated first 3 sections * updated spacing * adding the Environment Variables to Run Script section * adding in the remainder of the google doc changes * trying to fix duplicate explicit target name issue * fixing required_external_libraries_to_build_met links * HDF4 testing double underscore * adding double underscores for HDF5 and HDF-EOS2 and changing library back * fixing HDF5 web link name * Added formatting, made minor edits, removed manual build instructions sections * Removed reference to sample-test-case which no longer exists * Resolving errors * Minor formatting change * Updated apptainer instructions, new version references * Formatting updates and removal of first person language * fixed code blocks * Resolving error * Removed first person references * Removed reference to met_directory_structure * updated bolding, removed end section * Changing X.Y.X references to X.Y.Z * updating thru Using the compile_MET_all.sh Script * fixing spacing for ATLAS web address * Fixed broken internal reference to installation due to typo * adding version numbers * fixing Recommended-Components MET * 3.4.2 section updates * fixing spacing * adding a period and more dropdown menus * fixing spacing * loose ends * removing space * removing bold * External Library handling in compile_MET_all.sh section updates * installation directory view * installation directory after * attempting to add figures via the web updating Executing the compile_MET_all.sh script section * fixing bolds, italics etc. * updating last half of doc * loose ends * loose ends * adding more dropdown menus * fixing dropdown menu and testing automatic version * testing automatic version * confirmed automatic version cannot be added to italics or code blocks * another round of updates * fixing indents * creating a table * fixing table * fixing table 2 * more loose end changes * more changes * hopefully final updates * trying a fancy new csv table * troubleshooting csv table * correcting table name * adding a grid table * fixing grid table * removing csv table * Per #2588, update to the environment variables table. * Delete docs/Users_Guide/environment_variables_table.csv * Per #2785, add new example installation files * updating small changes in the first half of the documentation * new dropdown menu and fixing some links * trying to fix line spacing in dropdown * take 2 fixing spacing * take 3 fix spacing * env var order * removing "s" * small changes * un capitalizing S * A few minor changes --------- Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Minor hotfix to develop to fix a typo in the comments of the PB2NC config files. * Feature #2796 develop gha node20 (#2797) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Feature #2796 develop gha_node20, fix artifact names (#2799) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Per #2796, update testing workflow for update-artifact v4 version which requires unique artifact names. ci-run-unit * Per #2796, merge the logs into a single artifact. ci-run-unit * Per #2796, renaming rc_leads and rc as 1c and 2c, respectively for consistency purposes. Note that this commit should result in just a logs_compile artifact being created... and not doing the log merging step. * Per #2796, update the compilation workflow to make the log artifact names unique. * #2772 Change nan and inf to -9999 on reading ASCII input if failed to parse * Feature #2801 warnings (#2802) * Per #2801, add new time_offset_warning config option. * Per #2810, update MetConfig class to parse and check the time_offset_warning config option. * Per #2801, update Grid-Stat, MODE, PCP-Combine, Series-Analysis, Wavelet-Stat, and the WWMCA tool to call MetConfig::time_offset_warning() to decide whether to print a Warning or Debug(3) log message about initialization or valid time differences. * Per #2801, no change to content. Just removing unneeded whitespace. * Per #2801, remove extra empty line foromw m warning message. * Per #2801, remove extra empty line from MODE warning messages. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2745 mvmode enhancements (#2779) * Working mvmode with percentile thresholds and grid_type fixes, plus more unit tests * separated multivar and traditional mode frontend classes to simplify the mode executive class design * documentation changes to go with changes to percentiles and data_type * Cleanup and bugfix to doc * Bug fix to doc * Another bugfix attempt * synching up development.seneca with what is in the develop branch * added clone() method to all the vx_data2d var_info classes * Fix multivar_name/level definitions in UG * bug fix regarding accessing the config to create a verification grid * Update docs/Users_Guide/mode.rst * Update docs/Users_Guide/mode.rst * Per #2745, commit changes to Makefile.am/.in files after running bootstrap on this feature branch on seneca. * Per #2745, no real code changes. Just whitespace. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu>

@anewman89

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * #2755 Added a header count and checking header count instead of using header id (hid) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2753 comp script config (#2771) * set dynamic library file extension to .dylib if running on MacOS and .so otherwise * Added disabling of jasper documentation for compiliation on Hera * Updated * remove extra export of compiler env vars * include full path to log file so it is easier to file the log file to examine when a command fails * send cmake output to a log file * remove redundant semi-colon * use full path to log file so it is easier to examine on failure * use run_cmd to catch if rm command fails * Modifications for compilation on hera, gaea, and orion * Updating * fixed variable name * clean up if/else statements * set TIFF_LIBRARY_RELEASE argument to use full path to dynamic library file to prevent failure installing proj library * set LDFLAGS so that LDFLAGS value set in the user's environment will also be used * Updated based on gaea, orion, and hera installs * Updated * change extension of dynamic library files only if architecture is arm64 because older Macs still use .so * added netcdf library to args to prevent error installing NetCDF-CXX when PROJ has been installed in the same run of the script -- PATH is set in the COMPILE_PROJ if block that causes this flag from being added automatically * clean up how rpath and -L are added to LDFLAGS so that each entry is separate -- prevents errors installing on Mac arm64 because multiple rpath values aren't read using :. Also use MET_PROJLIB * Updated * removed -ltiff from MET libs * only add path to rpath and -L arguments if they are not already included in LDFLAGS * changed from using LIB_TIFF (full path to tiff lib file) to use TIFF_LIB_DIR (dir containing tiff lib file). Added TIFF_INCLUDE_DIR to proj compilation and -DJAS_ENABLE_DOC to jasper compliation taken from @jprestop branch * update comments * ensure all MET_* and MET_*LIB variables are added to the rpath for consistency * remove unnecessary if block and only export LDFLAGS at the end of setting locally * Updated * Added section for adding <VALUE>/lib64 and rearranged placement of ADDTL_DIR * Commenting out the running of the Jasper lib tests * Updating and/or removing files * Updating and/or removing files * Latest udpates which include the addition of the tiff library for proj * Remove commented out line. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2697 Share the temporary file for blocking * #2697 Reduced the temporary files for pb2nc * Moved typedef unixtime from vx_cal.h to time_array.h * #2697 Changed static const to constexpr for SonarQube (code smell) * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * #2673 Removed unused variable grid_x and grid_y * #2673 Renamed the shadowed variable n to nd * #2673 Use nullptr instead of literal 0 * #2673 Moved down namespace declarations. Removed unused variables * #2673 Reduced the scope of variables * #2673 Use nullptr instead of literal 0 * #2673 Removed unused variables * #2673 Renamed ex to ex2 which becomes shadowed variable * #2673 Removed always true condition * #2673 Moved down namespace declarations. Use nullptr instead of literal 0 * #2673 Use nullptr instead of literal 0. Removed unused variables * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * #2673 Removed unsued variables. Check nullptr of gt * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * #2772 Added quit_msg * Feature #2547 Read WRF output files natively (#2790) * Per #2547, add the file_type = NETCDF_WRF configuration option. * Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf. * Per #2547, rename nccf files as nc_cf for consistency throughout * Per #2547, rename Pinterp classes as Wrf * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * ignore directory automatically created by CLion * rename pinterp variables and classes to Wrf * more rename and fix typos * per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension * per #2547, add DataPlane function to handle staggering of grid * per #2547, change file type returned from pinterp to wrf * per #2547, call function to stagger DataPlane if necessary * per #2547, keep store if variable is on pressure levels and only read pressure field if it is * ignore cmake directory * per #2547, support U_PL, U_ZL, U, and U<x> where <x> is an integer in is_u_wind and the same for is_v_wind * change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V * error and exit if var with unsupported subgrid dimension is processed * clean up logging * only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info * Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected * per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag) * per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat * merged develop and resolved conflicts * rename lib from pinterp to wrf * rename lib * fixed incorrect merge changes * fixed broken unit test format * Per #2547, restore the mention of NAK.poly to Appendix B. * Change model to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * change obtype to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * plot higher vertical level to get non-zero output Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * indentation Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2772 Use JSON for attrubutes and numpy serialization dor 2D data instead of NetCDF * Hotfix to the develop branch after PR #2790 merged the feature_2547_wrf branch into develop. Rerunning bootstrap on the project machine seneca produces 2 small differences that should also be incorporated into develop. * Feature 2588 install rewrite (#2791) * updated first 3 sections * updated spacing * adding the Environment Variables to Run Script section * adding in the remainder of the google doc changes * trying to fix duplicate explicit target name issue * fixing required_external_libraries_to_build_met links * HDF4 testing double underscore * adding double underscores for HDF5 and HDF-EOS2 and changing library back * fixing HDF5 web link name * Added formatting, made minor edits, removed manual build instructions sections * Removed reference to sample-test-case which no longer exists * Resolving errors * Minor formatting change * Updated apptainer instructions, new version references * Formatting updates and removal of first person language * fixed code blocks * Resolving error * Removed first person references * Removed reference to met_directory_structure * updated bolding, removed end section * Changing X.Y.X references to X.Y.Z * updating thru Using the compile_MET_all.sh Script * fixing spacing for ATLAS web address * Fixed broken internal reference to installation due to typo * adding version numbers * fixing Recommended-Components MET * 3.4.2 section updates * fixing spacing * adding a period and more dropdown menus * fixing spacing * loose ends * removing space * removing bold * External Library handling in compile_MET_all.sh section updates * installation directory view * installation directory after * attempting to add figures via the web updating Executing the compile_MET_all.sh script section * fixing bolds, italics etc. * updating last half of doc * loose ends * loose ends * adding more dropdown menus * fixing dropdown menu and testing automatic version * testing automatic version * confirmed automatic version cannot be added to italics or code blocks * another round of updates * fixing indents * creating a table * fixing table * fixing table 2 * more loose end changes * more changes * hopefully final updates * trying a fancy new csv table * troubleshooting csv table * correcting table name * adding a grid table * fixing grid table * removing csv table * Per #2588, update to the environment variables table. * Delete docs/Users_Guide/environment_variables_table.csv * Per #2785, add new example installation files * updating small changes in the first half of the documentation * new dropdown menu and fixing some links * trying to fix line spacing in dropdown * take 2 fixing spacing * take 3 fix spacing * env var order * removing "s" * small changes * un capitalizing S * A few minor changes --------- Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Minor hotfix to develop to fix a typo in the comments of the PB2NC config files. * Feature #2796 develop gha node20 (#2797) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Feature #2796 develop gha_node20, fix artifact names (#2799) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Per #2796, update testing workflow for update-artifact v4 version which requires unique artifact names. ci-run-unit * Per #2796, merge the logs into a single artifact. ci-run-unit * Per #2796, renaming rc_leads and rc as 1c and 2c, respectively for consistency purposes. Note that this commit should result in just a logs_compile artifact being created... and not doing the log merging step. * Per #2796, update the compilation workflow to make the log artifact names unique. * #2772 Change nan and inf to -9999 on reading ASCII input if failed to parse * Feature #2801 warnings (#2802) * Per #2801, add new time_offset_warning config option. * Per #2810, update MetConfig class to parse and check the time_offset_warning config option. * Per #2801, update Grid-Stat, MODE, PCP-Combine, Series-Analysis, Wavelet-Stat, and the WWMCA tool to call MetConfig::time_offset_warning() to decide whether to print a Warning or Debug(3) log message about initialization or valid time differences. * Per #2801, no change to content. Just removing unneeded whitespace. * Per #2801, remove extra empty line foromw m warning message. * Per #2801, remove extra empty line from MODE warning messages. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * #2772 Initial release, Separated from point.py * #2772 Added point_nc.py * #2772 Changed write_tmp_nc and read_tmp_nc to write_tmp_py and read_tmp_py * #2772 Removed python_key_point_data & python_key_point_data_list and replaced them to tmp_point_var_name and tmp_point_data * #2772 Renamed tmp_nc_base_name, tmp_nc_file_var_name & tmp_nc_point_var_name to tmp_py_base_name, tmp_file_var_name, & tmp_point_var_name * #2772 More log messages for error * #2772 Changed API (log_msg to log_message) * #2772 Use met_point_nc_tools instead of met_point_tools * #2772 Changed APIs * #2772 Changed API * #2772 Changed default temp output format to JSON ande numpoy serialization * #2772 Allow to keep the temporary files by using the environment variable * Added log message if the temprary fiule was not deleted * Feature #2745 mvmode enhancements (#2779) * Working mvmode with percentile thresholds and grid_type fixes, plus more unit tests * separated multivar and traditional mode frontend classes to simplify the mode executive class design * documentation changes to go with changes to percentiles and data_type * Cleanup and bugfix to doc * Bug fix to doc * Another bugfix attempt * synching up development.seneca with what is in the develop branch * added clone() method to all the vx_data2d var_info classes * Fix multivar_name/level definitions in UG * bug fix regarding accessing the config to create a verification grid * Update docs/Users_Guide/mode.rst * Update docs/Users_Guide/mode.rst * Per #2745, commit changes to Makefile.am/.in files after running bootstrap on this feature branch on seneca. * Per #2745, no real code changes. Just whitespace. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Per #2772, added MET_PYTHON_EXE to various test cases and removed what would have been a duplicate after adding MET_PYTHON_EXE * Per #2772, add documentation about 3 new environment variables. * Per #2772, tweak the wording. * Fixing typo * Feature #2772 python_embedding_tmp_file (#2807) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature #2805 filter_set_hdr (#2806) * Per #2805, update STATAnalysisJob::dump_stat_line() to take an optional argument to indicate whether the set_hdr options should be applied. Update Stat-Analysis to only call dump_stat_line() with set_hdr for filter jobs. * Per #2805, update User's Guide with -set_hdr info. * Per #2805, the -set_hdr option doesn't apply to tc_stat * Per #2805, modify one Stat-Analysis filter job to demonstrate that the -set_hdr job command option is now applied to the -dump_row output. * Per #2805, enhance the R differencing logic to handle NA strings in the header columns. Previously, differences that include the NA string were ignored. This enhances the compareStatLty() function to handle NA strings properly. * Per #2805, no real code change. Just whitespace. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu>

@anewman89

* Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * #2755 Added a header count and checking header count instead of using header id (hid) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2753 comp script config (#2771) * set dynamic library file extension to .dylib if running on MacOS and .so otherwise * Added disabling of jasper documentation for compiliation on Hera * Updated * remove extra export of compiler env vars * include full path to log file so it is easier to file the log file to examine when a command fails * send cmake output to a log file * remove redundant semi-colon * use full path to log file so it is easier to examine on failure * use run_cmd to catch if rm command fails * Modifications for compilation on hera, gaea, and orion * Updating * fixed variable name * clean up if/else statements * set TIFF_LIBRARY_RELEASE argument to use full path to dynamic library file to prevent failure installing proj library * set LDFLAGS so that LDFLAGS value set in the user's environment will also be used * Updated based on gaea, orion, and hera installs * Updated * change extension of dynamic library files only if architecture is arm64 because older Macs still use .so * added netcdf library to args to prevent error installing NetCDF-CXX when PROJ has been installed in the same run of the script -- PATH is set in the COMPILE_PROJ if block that causes this flag from being added automatically * clean up how rpath and -L are added to LDFLAGS so that each entry is separate -- prevents errors installing on Mac arm64 because multiple rpath values aren't read using :. Also use MET_PROJLIB * Updated * removed -ltiff from MET libs * only add path to rpath and -L arguments if they are not already included in LDFLAGS * changed from using LIB_TIFF (full path to tiff lib file) to use TIFF_LIB_DIR (dir containing tiff lib file). Added TIFF_INCLUDE_DIR to proj compilation and -DJAS_ENABLE_DOC to jasper compliation taken from @jprestop branch * update comments * ensure all MET_* and MET_*LIB variables are added to the rpath for consistency * remove unnecessary if block and only export LDFLAGS at the end of setting locally * Updated * Added section for adding <VALUE>/lib64 and rearranged placement of ADDTL_DIR * Commenting out the running of the Jasper lib tests * Updating and/or removing files * Updating and/or removing files * Latest udpates which include the addition of the tiff library for proj * Remove commented out line. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Make indentation consistent. Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2697 Share the temporary file for blocking * #2697 Reduced the temporary files for pb2nc * Moved typedef unixtime from vx_cal.h to time_array.h * #2697 Changed static const to constexpr for SonarQube (code smell) * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * #2673 Removed unused variable grid_x and grid_y * #2673 Renamed the shadowed variable n to nd * #2673 Use nullptr instead of literal 0 * #2673 Moved down namespace declarations. Removed unused variables * #2673 Reduced the scope of variables * #2673 Use nullptr instead of literal 0 * #2673 Removed unused variables * #2673 Renamed ex to ex2 which becomes shadowed variable * #2673 Removed always true condition * #2673 Moved down namespace declarations. Use nullptr instead of literal 0 * #2673 Use nullptr instead of literal 0. Removed unused variables * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * #2673 Removed unsued variables. Check nullptr of gt * #2673 Moved down namespace declarations. * #2673 Added namespace std to string * #2772 Added quit_msg * Feature #2547 Read WRF output files natively (#2790) * Per #2547, add the file_type = NETCDF_WRF configuration option. * Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf. * Per #2547, rename nccf files as nc_cf for consistency throughout * Per #2547, rename Pinterp classes as Wrf * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * ignore directory automatically created by CLion * rename pinterp variables and classes to Wrf * more rename and fix typos * per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension * per #2547, add DataPlane function to handle staggering of grid * per #2547, change file type returned from pinterp to wrf * per #2547, call function to stagger DataPlane if necessary * per #2547, keep store if variable is on pressure levels and only read pressure field if it is * ignore cmake directory * per #2547, support U_PL, U_ZL, U, and U<x> where <x> is an integer in is_u_wind and the same for is_v_wind * change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V * error and exit if var with unsupported subgrid dimension is processed * clean up logging * only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info * Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected * per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag) * per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat * merged develop and resolved conflicts * rename lib from pinterp to wrf * rename lib * fixed incorrect merge changes * fixed broken unit test format * Per #2547, restore the mention of NAK.poly to Appendix B. * Change model to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * change obtype to WRF Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * plot higher vertical level to get non-zero output Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * indentation Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2772 Use JSON for attrubutes and numpy serialization dor 2D data instead of NetCDF * Hotfix to the develop branch after PR #2790 merged the feature_2547_wrf branch into develop. Rerunning bootstrap on the project machine seneca produces 2 small differences that should also be incorporated into develop. * Feature 2588 install rewrite (#2791) * updated first 3 sections * updated spacing * adding the Environment Variables to Run Script section * adding in the remainder of the google doc changes * trying to fix duplicate explicit target name issue * fixing required_external_libraries_to_build_met links * HDF4 testing double underscore * adding double underscores for HDF5 and HDF-EOS2 and changing library back * fixing HDF5 web link name * Added formatting, made minor edits, removed manual build instructions sections * Removed reference to sample-test-case which no longer exists * Resolving errors * Minor formatting change * Updated apptainer instructions, new version references * Formatting updates and removal of first person language * fixed code blocks * Resolving error * Removed first person references * Removed reference to met_directory_structure * updated bolding, removed end section * Changing X.Y.X references to X.Y.Z * updating thru Using the compile_MET_all.sh Script * fixing spacing for ATLAS web address * Fixed broken internal reference to installation due to typo * adding version numbers * fixing Recommended-Components MET * 3.4.2 section updates * fixing spacing * adding a period and more dropdown menus * fixing spacing * loose ends * removing space * removing bold * External Library handling in compile_MET_all.sh section updates * installation directory view * installation directory after * attempting to add figures via the web updating Executing the compile_MET_all.sh script section * fixing bolds, italics etc. * updating last half of doc * loose ends * loose ends * adding more dropdown menus * fixing dropdown menu and testing automatic version * testing automatic version * confirmed automatic version cannot be added to italics or code blocks * another round of updates * fixing indents * creating a table * fixing table * fixing table 2 * more loose end changes * more changes * hopefully final updates * trying a fancy new csv table * troubleshooting csv table * correcting table name * adding a grid table * fixing grid table * removing csv table * Per #2588, update to the environment variables table. * Delete docs/Users_Guide/environment_variables_table.csv * Per #2785, add new example installation files * updating small changes in the first half of the documentation * new dropdown menu and fixing some links * trying to fix line spacing in dropdown * take 2 fixing spacing * take 3 fix spacing * env var order * removing "s" * small changes * un capitalizing S * A few minor changes --------- Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Minor hotfix to develop to fix a typo in the comments of the PB2NC config files. * Feature #2796 develop gha node20 (#2797) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Feature #2796 develop gha_node20, fix artifact names (#2799) * Per #2796, update actions/checkout@v3 to actions/checkout@v4. * Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5. * Per #2796, update to download-artifact@v4 and actions/github-script@v7 * Per #2796, update testing workflow for update-artifact v4 version which requires unique artifact names. ci-run-unit * Per #2796, merge the logs into a single artifact. ci-run-unit * Per #2796, renaming rc_leads and rc as 1c and 2c, respectively for consistency purposes. Note that this commit should result in just a logs_compile artifact being created... and not doing the log merging step. * Per #2796, update the compilation workflow to make the log artifact names unique. * #2772 Change nan and inf to -9999 on reading ASCII input if failed to parse * Feature #2801 warnings (#2802) * Per #2801, add new time_offset_warning config option. * Per #2810, update MetConfig class to parse and check the time_offset_warning config option. * Per #2801, update Grid-Stat, MODE, PCP-Combine, Series-Analysis, Wavelet-Stat, and the WWMCA tool to call MetConfig::time_offset_warning() to decide whether to print a Warning or Debug(3) log message about initialization or valid time differences. * Per #2801, no change to content. Just removing unneeded whitespace. * Per #2801, remove extra empty line foromw m warning message. * Per #2801, remove extra empty line from MODE warning messages. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * #2772 Initial release, Separated from point.py * #2772 Added point_nc.py * #2772 Changed write_tmp_nc and read_tmp_nc to write_tmp_py and read_tmp_py * #2772 Removed python_key_point_data & python_key_point_data_list and replaced them to tmp_point_var_name and tmp_point_data * #2772 Renamed tmp_nc_base_name, tmp_nc_file_var_name & tmp_nc_point_var_name to tmp_py_base_name, tmp_file_var_name, & tmp_point_var_name * #2772 More log messages for error * #2772 Changed API (log_msg to log_message) * #2772 Use met_point_nc_tools instead of met_point_tools * #2772 Changed APIs * #2772 Changed API * #2772 Changed default temp output format to JSON ande numpoy serialization * #2772 Allow to keep the temporary files by using the environment variable * Added log message if the temprary fiule was not deleted * Feature #2745 mvmode enhancements (#2779) * Working mvmode with percentile thresholds and grid_type fixes, plus more unit tests * separated multivar and traditional mode frontend classes to simplify the mode executive class design * documentation changes to go with changes to percentiles and data_type * Cleanup and bugfix to doc * Bug fix to doc * Another bugfix attempt * synching up development.seneca with what is in the develop branch * added clone() method to all the vx_data2d var_info classes * Fix multivar_name/level definitions in UG * bug fix regarding accessing the config to create a verification grid * Update docs/Users_Guide/mode.rst * Update docs/Users_Guide/mode.rst * Per #2745, commit changes to Makefile.am/.in files after running bootstrap on this feature branch on seneca. * Per #2745, no real code changes. Just whitespace. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Per #2772, added MET_PYTHON_EXE to various test cases and removed what would have been a duplicate after adding MET_PYTHON_EXE * Per #2772, add documentation about 3 new environment variables. * Per #2772, tweak the wording. * Fixing typo * Feature #2772 python_embedding_tmp_file (#2807) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature #2805 filter_set_hdr (#2806) * Per #2805, update STATAnalysisJob::dump_stat_line() to take an optional argument to indicate whether the set_hdr options should be applied. Update Stat-Analysis to only call dump_stat_line() with set_hdr for filter jobs. * Per #2805, update User's Guide with -set_hdr info. * Per #2805, the -set_hdr option doesn't apply to tc_stat * Per #2805, modify one Stat-Analysis filter job to demonstrate that the -set_hdr job command option is now applied to the -dump_row output. * Per #2805, enhance the R differencing logic to handle NA strings in the header columns. Previously, differences that include the NA string were ignored. This enhances the compareStatLty() function to handle NA strings properly. * Per #2805, no real code change. Just whitespace. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2772 python embedding fill value (#2810) * #2772 Update masked value to -9999. Renamed error_messageg to error_message * #2772 Renamed error_messageg to error_message * #2772 Accept 4th argument as user defined fill value. * #2772 Added python_numpy_plot_data_plane_missing which processes user defined fill value --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature #2809 nc_laea (#2811) * Per #2809 work in progress parsing LAEA grids from MET NetCDF files. * Per #2809, update the LaeaData struct to store the spheroid_name as a character array. I tried very hard to store this as a string instead but ran into lots of problems with segfaults when deleting the allocated structs. Using a fixed length character array while making calls to the m_strncpy() and m_strlen() utility functions enables the code to work properly while hopefully minimizing SonarQube findings. * Per #2809, call m_strncpy() to set LaeaData::spheroid_name * Per #2809, add tests to pcp_combine and plot_data_plane to demonstrate the reading the LAEA data from MET NetCDF files. * Per #2809, correct the expected PostScript file output name in unit_plot_data_plane.xml * Feature #2809 v12.0.0_beta3 (#2814) * Per #2809, update copyright to 2024 * Per #2809, add release v12.0.0-beta3 release notes. * Feature #2769 tc_diag_driver_v0.11.0 (#2812) * Per #2769, update to tc_diag_driver version 0.11.0. * Per #2769, add tmp_nc_diag_flag config option to retain temporary files. * Per #2769, change default regrdding method from NEAREST to BILIN. * Per #2769, add checks to make sure input files exist. * Fix the handling of diagnostic name and units strings. Both are limited to 7 characters but the units are enclosed in parantheses, increasing the field width to 9. If the strings are too long, truncate them and print warnings like this: WARNING: OutFileInfo::write_cira_diag_vals() -> long diagnostic name "850TANGXXX" truncated to "850TANG"! WARNING: OutFileInfo::write_cira_diag_vals() -> long diagnostic units string "(10^7C/MXXX)" truncated to "(10^7C/M)"! Pad the first 2 columns out to widths of 7 and 9 and set the inter-column spacing between columns 2 and 3 to 0. This could technically introduce parsing problems when the units are 7 characters long and the values have 5 digits, but this is the logic needed to exactly replicate the existing output. * Just whitespace * Per #2769, remove the tmp_nc_diag_flag TC-Diag config file option in favor of the MET_KEEP_TEMP_FILE environment variable option that Howard is adding on his feature_2772_python_embedding_json branch. I did add documentation about that option here even though the functionality is coming from Howard's branch. * Per #2769, remove tmp_nc_diag_flag that has been removed. * Per #2769, remove mention of tmp_nc_diag_flag option that has been removed. * Per #2769, upgrade to METbaseimage version 3.2 to provide SciPy in the Python environment which is needed by TC-Diag. * Per #2769, add SciPy to the list of required Python packages. * Per #2769, tweak AppendixF details. * Per #2769, update release date for MET-12.0.0-beta3 to be 20240207 since we didn't get it out on the 6th. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: root <root@83062d57c5dd> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: Christina Kalb <kalb@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: j-opatz <jopatz@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: Tracy Hertneky <hertneky@ucar.edu>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2867 point2grid qc flag main v11.1 (#2878) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values * #2867 Added adjusted confidnece counts * #2867 Corretced indent * #2867 Corretced indent --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #2884 No filtering by QC flags without -qc option (#2885) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2659, making updates as proposed at the 20240516 MET Eng. Mtg. (#2896) * Bugfix #2897 main_v11.1 python_valid_time (#2898) * Per #2897, fix typos in 3 log messages. Also fix the bug in storing the valid time strings. The time string in vld_array should exactly correspond to the numeric unixtime values in vld_num_array. Therefore they need to be updated inside the same if block. The bug is that we were storing only the unique unixtime values but storing ALL of the valid time string, not just the unique ones. * Per #2897, don’t waste time searching, just set vld_odd to n-1 * #2687 Saved the PBL input into the vector (#2903) * #2904 Changed R path to R-4.4.0 (#2909) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2856 main_v11.1 ens_climo (#2917) * Per #2856, reinitialize the climo cdf info pointer. * Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined. * Revert "Per #2856, reinitialize the climo cdf info pointer." This reverts commit 99b6bb3. * Per #2856, reinitialize the climo cdf info pointer. * Revert "Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined." This reverts commit 0ae224f. * Per #2856, update the truth_data_version and input_data_version if main_vX.Y is not present in the branch name. * Update set_job_controls.sh * Update set_job_controls.sh * Per #2856, update logic for counting the maximum number of PCT, PRC, PJC, and PSTD lines. Need to multiply by the number of climo cdf bins. * Per #2856, add more error checking to avoid writing nan to .stat output files * Bugfix #2841 main_v11.1 tang_rad_winds (#2920) * Per #2841, unrelated, just removing a spurious character from a log message. * Per #2841, work in progress. Mostly just modifying whitespace and log messages so far * Per #2841, running tc_rmw with only UGRD input causes the tool to hang. Adding break statements to prevent the hang. * Per #2841, add new has_pressure_level() utility function and update tc_rmw to use it to deteremine whether or not the pressure dimension should be written the output rather than basing that off the number of levels being > 1. * Per #1849, fix to radial and tangential winds. * Per #2841, just changing a code comment about the longitude swap * Per #2841, update variable names for consistency and understanding. * Per #2841, the substantive bug fixed here is how we index into the U and V data in wind_ne_to_rt() using i_rev instead of i. Also, update the variable names for consistency and clarity.. * Per #2841, patch apparent copy/paste bug in tcrmw_grid.cc code. * Per #2841, only changing whitespace. * Per #2841, update logic in EarthRotation::set_tcrmw() to change the rotation of TCRMW grids from pointing north to pointing east. * Per #2841, fix warning message * Replace tab with spaces * Per #2841, correct the units for the azimuth netcdf output variable * Per #2841, reverse the x dimension of the rotated latlon grid to effectively switch from counterclockwise rotation to clockwise. * Feature #2855 v11.1.1 (#2923) * Per #2855, add v11.1.1 release notes. * Per #2841, fix bold formatting of release notes. * Update conf.py with actual 11.1.1 release date --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2867 point2grid qc flag main v11.1 (#2878) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values * #2867 Added adjusted confidnece counts * #2867 Corretced indent * #2867 Corretced indent --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #2884 No filtering by QC flags without -qc option (#2885) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2659, making updates as proposed at the 20240516 MET Eng. Mtg. (#2896) * Bugfix #2897 main_v11.1 python_valid_time (#2898) * Per #2897, fix typos in 3 log messages. Also fix the bug in storing the valid time strings. The time string in vld_array should exactly correspond to the numeric unixtime values in vld_num_array. Therefore they need to be updated inside the same if block. The bug is that we were storing only the unique unixtime values but storing ALL of the valid time string, not just the unique ones. * Per #2897, don’t waste time searching, just set vld_odd to n-1 * #2687 Saved the PBL input into the vector (#2903) * #2904 Changed R path to R-4.4.0 (#2909) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2856 main_v11.1 ens_climo (#2917) * Per #2856, reinitialize the climo cdf info pointer. * Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined. * Revert "Per #2856, reinitialize the climo cdf info pointer." This reverts commit 99b6bb3. * Per #2856, reinitialize the climo cdf info pointer. * Revert "Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined." This reverts commit 0ae224f. * Per #2856, update the truth_data_version and input_data_version if main_vX.Y is not present in the branch name. * Update set_job_controls.sh * Update set_job_controls.sh * Per #2856, update logic for counting the maximum number of PCT, PRC, PJC, and PSTD lines. Need to multiply by the number of climo cdf bins. * Per #2856, add more error checking to avoid writing nan to .stat output files * Bugfix #2841 main_v11.1 tang_rad_winds (#2920) * Per #2841, unrelated, just removing a spurious character from a log message. * Per #2841, work in progress. Mostly just modifying whitespace and log messages so far * Per #2841, running tc_rmw with only UGRD input causes the tool to hang. Adding break statements to prevent the hang. * Per #2841, add new has_pressure_level() utility function and update tc_rmw to use it to deteremine whether or not the pressure dimension should be written the output rather than basing that off the number of levels being > 1. * Per #1849, fix to radial and tangential winds. * Per #2841, just changing a code comment about the longitude swap * Per #2841, update variable names for consistency and understanding. * Per #2841, the substantive bug fixed here is how we index into the U and V data in wind_ne_to_rt() using i_rev instead of i. Also, update the variable names for consistency and clarity.. * Per #2841, patch apparent copy/paste bug in tcrmw_grid.cc code. * Per #2841, only changing whitespace. * Per #2841, update logic in EarthRotation::set_tcrmw() to change the rotation of TCRMW grids from pointing north to pointing east. * Per #2841, fix warning message * Replace tab with spaces * Per #2841, correct the units for the azimuth netcdf output variable * Per #2841, reverse the x dimension of the rotated latlon grid to effectively switch from counterclockwise rotation to clockwise. * Feature #2855 v11.1.1 (#2923) * Per #2855, add v11.1.1 release notes. * Per #2841, fix bold formatting of release notes. * Update conf.py with actual 11.1.1 release date * bugfix #2936 point2grid for tripolar GFS input (#2945) * #2936 Support 1D lat/lon values * #2936 Change obs_type to TYPE_NCCF if FileType_NcCF is given from the config file --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2958 main_v11.1 BAGSS SEDI CI (#2960) --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2867 point2grid qc flag main v11.1 (#2878) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values * #2867 Added adjusted confidnece counts * #2867 Corretced indent * #2867 Corretced indent --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #2884 No filtering by QC flags without -qc option (#2885) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2659, making updates as proposed at the 20240516 MET Eng. Mtg. (#2896) * Bugfix #2897 main_v11.1 python_valid_time (#2898) * Per #2897, fix typos in 3 log messages. Also fix the bug in storing the valid time strings. The time string in vld_array should exactly correspond to the numeric unixtime values in vld_num_array. Therefore they need to be updated inside the same if block. The bug is that we were storing only the unique unixtime values but storing ALL of the valid time string, not just the unique ones. * Per #2897, don’t waste time searching, just set vld_odd to n-1 * #2687 Saved the PBL input into the vector (#2903) * #2904 Changed R path to R-4.4.0 (#2909) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2856 main_v11.1 ens_climo (#2917) * Per #2856, reinitialize the climo cdf info pointer. * Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined. * Revert "Per #2856, reinitialize the climo cdf info pointer." This reverts commit 99b6bb3. * Per #2856, reinitialize the climo cdf info pointer. * Revert "Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined." This reverts commit 0ae224f. * Per #2856, update the truth_data_version and input_data_version if main_vX.Y is not present in the branch name. * Update set_job_controls.sh * Update set_job_controls.sh * Per #2856, update logic for counting the maximum number of PCT, PRC, PJC, and PSTD lines. Need to multiply by the number of climo cdf bins. * Per #2856, add more error checking to avoid writing nan to .stat output files * Bugfix #2841 main_v11.1 tang_rad_winds (#2920) * Per #2841, unrelated, just removing a spurious character from a log message. * Per #2841, work in progress. Mostly just modifying whitespace and log messages so far * Per #2841, running tc_rmw with only UGRD input causes the tool to hang. Adding break statements to prevent the hang. * Per #2841, add new has_pressure_level() utility function and update tc_rmw to use it to deteremine whether or not the pressure dimension should be written the output rather than basing that off the number of levels being > 1. * Per #1849, fix to radial and tangential winds. * Per #2841, just changing a code comment about the longitude swap * Per #2841, update variable names for consistency and understanding. * Per #2841, the substantive bug fixed here is how we index into the U and V data in wind_ne_to_rt() using i_rev instead of i. Also, update the variable names for consistency and clarity.. * Per #2841, patch apparent copy/paste bug in tcrmw_grid.cc code. * Per #2841, only changing whitespace. * Per #2841, update logic in EarthRotation::set_tcrmw() to change the rotation of TCRMW grids from pointing north to pointing east. * Per #2841, fix warning message * Replace tab with spaces * Per #2841, correct the units for the azimuth netcdf output variable * Per #2841, reverse the x dimension of the rotated latlon grid to effectively switch from counterclockwise rotation to clockwise. * Feature #2855 v11.1.1 (#2923) * Per #2855, add v11.1.1 release notes. * Per #2841, fix bold formatting of release notes. * Update conf.py with actual 11.1.1 release date * bugfix #2936 point2grid for tripolar GFS input (#2945) * #2936 Support 1D lat/lon values * #2936 Change obs_type to TYPE_NCCF if FileType_NcCF is given from the config file --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2958 main_v11.1 BAGSS SEDI CI (#2960) * Bugfix #2958 main_v11.1 ORSS (#2972) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2867 point2grid qc flag main v11.1 (#2878) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values * #2867 Added adjusted confidnece counts * #2867 Corretced indent * #2867 Corretced indent --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #2884 No filtering by QC flags without -qc option (#2885) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2659, making updates as proposed at the 20240516 MET Eng. Mtg. (#2896) * Bugfix #2897 main_v11.1 python_valid_time (#2898) * Per #2897, fix typos in 3 log messages. Also fix the bug in storing the valid time strings. The time string in vld_array should exactly correspond to the numeric unixtime values in vld_num_array. Therefore they need to be updated inside the same if block. The bug is that we were storing only the unique unixtime values but storing ALL of the valid time string, not just the unique ones. * Per #2897, don’t waste time searching, just set vld_odd to n-1 * #2687 Saved the PBL input into the vector (#2903) * #2904 Changed R path to R-4.4.0 (#2909) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2856 main_v11.1 ens_climo (#2917) * Per #2856, reinitialize the climo cdf info pointer. * Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined. * Revert "Per #2856, reinitialize the climo cdf info pointer." This reverts commit 99b6bb3. * Per #2856, reinitialize the climo cdf info pointer. * Revert "Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined." This reverts commit 0ae224f. * Per #2856, update the truth_data_version and input_data_version if main_vX.Y is not present in the branch name. * Update set_job_controls.sh * Update set_job_controls.sh * Per #2856, update logic for counting the maximum number of PCT, PRC, PJC, and PSTD lines. Need to multiply by the number of climo cdf bins. * Per #2856, add more error checking to avoid writing nan to .stat output files * Bugfix #2841 main_v11.1 tang_rad_winds (#2920) * Per #2841, unrelated, just removing a spurious character from a log message. * Per #2841, work in progress. Mostly just modifying whitespace and log messages so far * Per #2841, running tc_rmw with only UGRD input causes the tool to hang. Adding break statements to prevent the hang. * Per #2841, add new has_pressure_level() utility function and update tc_rmw to use it to deteremine whether or not the pressure dimension should be written the output rather than basing that off the number of levels being > 1. * Per #1849, fix to radial and tangential winds. * Per #2841, just changing a code comment about the longitude swap * Per #2841, update variable names for consistency and understanding. * Per #2841, the substantive bug fixed here is how we index into the U and V data in wind_ne_to_rt() using i_rev instead of i. Also, update the variable names for consistency and clarity.. * Per #2841, patch apparent copy/paste bug in tcrmw_grid.cc code. * Per #2841, only changing whitespace. * Per #2841, update logic in EarthRotation::set_tcrmw() to change the rotation of TCRMW grids from pointing north to pointing east. * Per #2841, fix warning message * Replace tab with spaces * Per #2841, correct the units for the azimuth netcdf output variable * Per #2841, reverse the x dimension of the rotated latlon grid to effectively switch from counterclockwise rotation to clockwise. * Feature #2855 v11.1.1 (#2923) * Per #2855, add v11.1.1 release notes. * Per #2841, fix bold formatting of release notes. * Update conf.py with actual 11.1.1 release date * bugfix #2936 point2grid for tripolar GFS input (#2945) * #2936 Support 1D lat/lon values * #2936 Change obs_type to TYPE_NCCF if FileType_NcCF is given from the config file --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2958 main_v11.1 BAGSS SEDI CI (#2960) * Bugfix #2958 main_v11.1 ORSS (#2972) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Bugfix #2979 main_v11.1 MTD Grid (#2980) * Per #2979, remove nc_grid.h/.cc and replace it with calls to the read_netcdf_grid(...) and write_netcdf_proj(...) library utility functions. Note that these changes do compile but I haven't tested whether they actually fix the underlying problem. Also note that nc_utils_local.h/.cc can also likely be replaced with calls to common library functions. * Per #2979, remove references to nc_grid.o from the MTD test code. * Per #2979, the write_netcdf_proj(...) utility function adds the lat and lon dimensions. Update mtd to NOT define those dimensions prior to calling write_netcdf_proj(...). * Per #2979, add this issue to the revision history. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

* Add user execute permissions to compile script - main_v11.1 (#2740) * Changing -j to "-j 5" as the recommended value for MAKE_ARGS * Per #2761, update the MET development environment after upgrading seneca to debian bookworm. * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST) * #2757 The SonarQube token and URL are replaced by using the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2760 main_v11.1 --enable-python (#2767) * #2755 Added a header count and checking header count instead of using header id (hid) * Bugfix #2782 main_v11.1 MASSDEN (#2784) * Per #2782, update the multiple matching records warning message to include the table number for each record. * Per #2782, update read_grib2_record_list() to parse the level values and aerosol information correctly for table 4.48. * Per #2782 tweak variable naming convention. * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Adding -lnetcdf -lm to configure_lib_args for NetCDF-CXX * Feature #2796 main_v11.1 gha node20 (#2798) * Per #2796, update versions of actions to fix the node 16 to 20 warning message. * Per #2796, port fixes for artifact name handling over from the develop branch to the main_v11.1 testing workflow. Also add the compilation_options.yml workflow since the workflows are being updated. * Create 11.1.0_casper * Recent changes to branch protection rules for the main_vX.Y branches have broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge main_vX.Y into main_vX.Y-ref directly, use an intermediate update_truth_for_main_vX.Y branch. * Update the pull request template to include a question about expected impacts to existing METplus Use Cases. * Bugfix #2833 main_v11.1 azimuth (#2834) * Per #2833, fix n-1 bug when defining the azimuth delta for range/azimuth grids. * Per #2833, port fixes over from the bugfix_2833_develop_azimuth branch over to the main_v11.1 branch. --------- Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2379 main_v11.1 sonarqube GHA (#2848) * Per #2379, migrating largely the same changes for #2379 into the main_v11.1 branch. The difference is that --enable-all is not used since that is not a valid configuration option for MET version 11.1.0. * Hotfix related to #2379. The sonar.newCode.referenceBranch and sonar.branch.name cannot be set to the same string! Only add the newCode definition when they differ. * Feature #2379 main_v11.1 sonarqube updates (#2851) * Feature #2379 main_v11.1 single_sq_project (#2866) * Bugfix 2867 point2grid qc flag main v11.1 (#2874) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix 2867 point2grid qc flag main v11.1 (#2878) * #2867 Added point2grid_GOES_16_ADP_Enterprise_high and changed qc flags for ADP * #2867 Added get_nc_att_values_ * #2867 Added get_nc_att_values * #2867 Get the ADP QC flag values from the varibale attributes (support GOES16 Enterprise allgorithm) and apply them for QC Flags. Adjusted the ADP QC flags based on the variable QC values * #2867 Added adjusted confidnece counts * #2867 Corretced indent * #2867 Corretced indent --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * #2884 No filtering by QC flags without -qc option (#2885) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2659, making updates as proposed at the 20240516 MET Eng. Mtg. (#2896) * Bugfix #2897 main_v11.1 python_valid_time (#2898) * Per #2897, fix typos in 3 log messages. Also fix the bug in storing the valid time strings. The time string in vld_array should exactly correspond to the numeric unixtime values in vld_num_array. Therefore they need to be updated inside the same if block. The bug is that we were storing only the unique unixtime values but storing ALL of the valid time string, not just the unique ones. * Per #2897, don’t waste time searching, just set vld_odd to n-1 * #2687 Saved the PBL input into the vector (#2903) * #2904 Changed R path to R-4.4.0 (#2909) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2856 main_v11.1 ens_climo (#2917) * Per #2856, reinitialize the climo cdf info pointer. * Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined. * Revert "Per #2856, reinitialize the climo cdf info pointer." This reverts commit 99b6bb3. * Per #2856, reinitialize the climo cdf info pointer. * Revert "Per #2856, update the testing.yml workflow dispatch option to let the comparison branch be manually defined." This reverts commit 0ae224f. * Per #2856, update the truth_data_version and input_data_version if main_vX.Y is not present in the branch name. * Update set_job_controls.sh * Update set_job_controls.sh * Per #2856, update logic for counting the maximum number of PCT, PRC, PJC, and PSTD lines. Need to multiply by the number of climo cdf bins. * Per #2856, add more error checking to avoid writing nan to .stat output files * Bugfix #2841 main_v11.1 tang_rad_winds (#2920) * Per #2841, unrelated, just removing a spurious character from a log message. * Per #2841, work in progress. Mostly just modifying whitespace and log messages so far * Per #2841, running tc_rmw with only UGRD input causes the tool to hang. Adding break statements to prevent the hang. * Per #2841, add new has_pressure_level() utility function and update tc_rmw to use it to deteremine whether or not the pressure dimension should be written the output rather than basing that off the number of levels being > 1. * Per #1849, fix to radial and tangential winds. * Per #2841, just changing a code comment about the longitude swap * Per #2841, update variable names for consistency and understanding. * Per #2841, the substantive bug fixed here is how we index into the U and V data in wind_ne_to_rt() using i_rev instead of i. Also, update the variable names for consistency and clarity.. * Per #2841, patch apparent copy/paste bug in tcrmw_grid.cc code. * Per #2841, only changing whitespace. * Per #2841, update logic in EarthRotation::set_tcrmw() to change the rotation of TCRMW grids from pointing north to pointing east. * Per #2841, fix warning message * Replace tab with spaces * Per #2841, correct the units for the azimuth netcdf output variable * Per #2841, reverse the x dimension of the rotated latlon grid to effectively switch from counterclockwise rotation to clockwise. * Feature #2855 v11.1.1 (#2923) * Per #2855, add v11.1.1 release notes. * Per #2841, fix bold formatting of release notes. * Update conf.py with actual 11.1.1 release date * bugfix #2936 point2grid for tripolar GFS input (#2945) * #2936 Support 1D lat/lon values * #2936 Change obs_type to TYPE_NCCF if FileType_NcCF is given from the config file --------- Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Bugfix #2958 main_v11.1 BAGSS SEDI CI (#2960) * Bugfix #2958 main_v11.1 ORSS (#2972) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Bugfix #2979 main_v11.1 MTD Grid (#2980) * Per #2979, remove nc_grid.h/.cc and replace it with calls to the read_netcdf_grid(...) and write_netcdf_proj(...) library utility functions. Note that these changes do compile but I haven't tested whether they actually fix the underlying problem. Also note that nc_utils_local.h/.cc can also likely be replaced with calls to common library functions. * Per #2979, remove references to nc_grid.o from the MTD test code. * Per #2979, the write_netcdf_proj(...) utility function adds the lat and lon dimensions. Update mtd to NOT define those dimensions prior to calling write_netcdf_proj(...). * Per #2979, add this issue to the revision history. * Update config_options.rst (#2986) Updating to fix a typo in which "mesage_type_group_map" is mis-spelled. * Hotfix to main_v11.1 to set the MET_TEST_INPUT path on seneca after the change to the SEEPS filename. --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Howard Soh <hsoh@ucar.edu> Co-authored-by: Jonathan Vigh <jvigh@ucar.edu> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>

JohnHalleyGotway added component: build process Build process issue type: task An actionable item of work requestor: METplus Team METplus Development Team priority: high High Priority labels Dec 11, 2023

JohnHalleyGotway added this to the MET 12.0.0 milestone Dec 11, 2023

JohnHalleyGotway self-assigned this Dec 11, 2023

JohnHalleyGotway changed the title ~~Update development.seneca with the compilation settings needed after upgrading seneca to Debian Bookworm~~ Update development.seneca to compile MET after upgrading seneca to Debian Bookworm Dec 11, 2023

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Per #2761, update the development enviornment file for seneca based o…

4bd968f

…n RAL-IT compilation settings described in /nrit/ral/src/met/README.ral-it.

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Per #2761, update output from Bison to GNU Bison 3.8.2, which is the …

eee4946

…version used by Debian Bookworm on our devleopment machine, seneca.

JohnHalleyGotway linked a pull request Dec 12, 2023 that will close this issue

Feature #2761 develop seneca #2762

Merged

15 tasks

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Per #2761, update the seneca development environment for the main_v11…

1be9be4

….0 branch.

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Per #2761, update the MET development environment after upgrading sen…

48f4f5f

…eca to debian bookworm.

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Per #2761, update the MET development environment after upgrading sen…

7be9cce

…eca to debian bookworm.

JohnHalleyGotway added a commit that referenced this issue Dec 12, 2023

Feature #2761 develop seneca (#2762)

4135e27

JohnHalleyGotway closed this as completed Dec 12, 2023

JohnHalleyGotway pushed a commit that referenced this issue Dec 12, 2023

Per #2761, define runtime python version for testing rather than usin…

c67c7c9

…g the default version which no longer exists in /usr/local

JohnHalleyGotway pushed a commit that referenced this issue Dec 12, 2023

Per #2761, define runtime python version for testing rather than usin…

54eaa0c

…g the default version which no longer exists in /usr/local

JohnHalleyGotway pushed a commit that referenced this issue Dec 12, 2023

Per #2761, define runtime python version for testing rather than usin…

8f99ef0

…g the default version which no longer exists in /usr/local

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, fix setting ci-skip-all

40ae2fb

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, fix setting ci-skip-all

0ae6c03

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, fix setting ci-skip-all

514038f

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, patching test_util.R to use the -C command line option for…

7c6df3c

… ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion.

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, patching test_util.R to use the -C command line option for…

7d33415

… ncdiff.

JohnHalleyGotway pushed a commit that referenced this issue Dec 13, 2023

Per #2761, patching test_util.R to use the -C command line option for…

9b663e9

… ncdiff.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update `development.seneca` to compile MET after upgrading seneca to Debian Bookworm #2761

Update `development.seneca` to compile MET after upgrading seneca to Debian Bookworm #2761

JohnHalleyGotway commented Dec 11, 2023 •

edited

Loading

JohnHalleyGotway commented Dec 12, 2023

Update development.seneca to compile MET after upgrading seneca to Debian Bookworm #2761

Update development.seneca to compile MET after upgrading seneca to Debian Bookworm #2761

Comments

JohnHalleyGotway commented Dec 11, 2023 • edited Loading

Describe the Task

Time Estimate

Sub-Issues

Relevant Deadlines

Funding Source

Define the Metadata

Assignee

Labels

Milestone and Projects

Define Related Issue(s)

Task Checklist

JohnHalleyGotway commented Dec 12, 2023

Update `development.seneca` to compile MET after upgrading seneca to Debian Bookworm #2761

Update `development.seneca` to compile MET after upgrading seneca to Debian Bookworm #2761

JohnHalleyGotway commented Dec 11, 2023 •

edited

Loading