Bug fix for calculation of number of time steps for each WE2E test #782
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gsketefian
requested review from
mkavulich,
JeffBeck-NOAA,
RatkoVasic-NOAA,
BenjaminBlake-NOAA,
ywangwof,
chan-hoo,
panll,
christinaholtNOAA,
christopherwharrop-noaa,
danielabdi-noaa,
mark-a-potts,
jkbk2004,
willmayfield,
dmwright526,
gspetro-NOAA,
natalie-perlin and
EdwardSnyder-NOAA
as code owners
willmayfield
approved these changes
Jun 1, 2022
danielabdi-noaa
approved these changes
Jun 1, 2022
EdwardSnyder-NOAA
added a commit
that referenced
this pull request
Jun 10, 2022
* update IC/LBC file paths to match new IC/LBC file structure (#766) * update IC/LBC file paths to match new IC/LBC file structure * change forecast length * Update config.community.sh These changes reflects the SRW team's preference for a 12-hours forecast. The gfs.t18z.pgrb2.0p25.f012 file will need to be added to all Level 1 systems. Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> * add a machine file for MacOS (#777) Co-authored-by: Natalie Perlin <perlin.natalie@gmail.com> * Add columns for relative test and time step to CSV file containing WE2E test info (#776) ## DESCRIPTION OF CHANGES: This PR modifies the script `get_WE2Etest_names_subdirs_descs.sh` that creates the CSV file containing test information the columns for relative cost and time step (`DT_ATMOS`). This allows users to have a better idea of what the cost of each test is (the relative cost is defined such that 1 unit represents the cost of running a single 6-hour forecast on the `RRFS_CONUS_25km` grid). Other related changes: * Add the flag `generate_csv_file` to the `run_WE2E_tests.sh` script to allow users to skip the CSV file generation step (since it can take a while). * Use arguments for the function `set_predef_grid_params()` so that it can be called not just from the experiment generation scripts but also others (in this case from `get_WE2Etest_names_subdirs_descs.sh`). Necessary for this PR. * Move some constants (`NH0`, `NH3`, `NH4`) from `config_defaults.sh` to `constants.sh` to make it easier to access them. * Place the contents of `constants.sh` at the start of each experiment's `var_defns.sh` so that these constants are available to the j-jobs and ex-scripts when `var_defns.sh` is sourced. * Use variable `valid_vals_BOOLEAN` in many more places (which allows many other `valid_vals_...` variables in `valid_param_vals.sh` and `setup.sh` to be removed). * Several other bug fixes. ## TESTS CONDUCTED: The following tests (with results) were conducted on Hera. There were only three (previously existing) failures. ``` * grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16: FAILURE * grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_HRRR: SUCCESS * grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_RRFS_v1beta: SUCCESS * grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta: SUCCESS * grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR: SUCCESS * grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta: SUCCESS * GST_release_public_v1: SUCCESS * MET_ensemble_verification: SUCCESS * MET_verification: SUCCESS * community_ensemble_008mems: SUCCESS * community_ensemble_2mems: SUCCESS * custom_ESGgrid: SUCCESS * custom_GFDLgrid: SUCCESS * custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_FALSE: SUCCESS * custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_TRUE: SUCCESS * deactivate_tasks: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019061200: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019101818: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2020022518: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2020022600: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2021010100: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019061200: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019101818: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2020022518: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2020022600: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2021010100: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2021062000: SUCCESS * get_from_HPSS_ics_GSMGFS_lbcs_GSMGFS: SUCCESS * get_from_HPSS_ics_HRRR_lbcs_RAP: SUCCESS * get_from_HPSS_ics_RAP_lbcs_RAP: SUCCESS * get_from_NOMADS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio: SUCCESS * inline_post: SUCCESS * nco_ensemble: SUCCESS * nco_inline_post: SUCCESS * pregen_grid_orog_sfc_climo: SUCCESS * specify_DOT_OR_USCORE: SUCCESS * specify_DT_ATMOS_LAYOUT_XY_BLOCKSIZE: SUCCESS * specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS: SUCCESS * specify_RESTART_INTERVAL: SUCCESS * specify_template_filenames: SUCCESS * subhourly_post: FAILURE * subhourly_post_ensemble_2mems: FAILURE ``` ## DOCUMENTATION: Documentation is updated via PR #[278](ufs-community/ufs-srweather-app#278) in the ``ufs-srweather-app`` repo. * Bug fix for calculation of number of time steps for a given WE2E test. (#782) ## DESCRIPTION OF CHANGES: This PR fixes a bug in the way the number of time steps (`num_time_steps` in `get_WE2Etest_names_subdirs_descs.sh`) is calculated for each WE2E test when creating the CSV file containing information about the available WE2E tests. ## TESTS CONDUCTED: Reran the `run_WE2E_tests.sh` script (which in turn calls `get_WE2Etest_names_subdirs_descs.sh`), and a CSV file containing the adjusted numbers was successfully created. ## DOCUMENTATION: None needed. * Bugfix: Pass domain name to python plotting scripts (#783) ## DESCRIPTION OF CHANGES: PR #[763](#763) introduced the experiment variable `POST_OUTPUT_DOMAIN_NAME` and used it to modify the names of the UPP output files. This PR adds `POST_OUTPUT_DOMAIN_NAME` as an argument to the python plotting scripts to enable those scripts to form the proper UPP output file names to read in. It also modifies the batch scripts (`qsub_job.sh`, `qsub_job_diff.sh`, `sq_job.sh`, and `sq_job_diff.sh`) to add the new argument to the calls to the python plotting scripts that are called. ## TESTS CONDUCTED: Both python plotting scripts `plot_allvars.py` and `plot_allvars_diff.py` were run on Hera on preexisting UPP output files and completed successfully. The batch scripts `sq_job.sh` and `sq_job_diff.sh` were also run successfully on Hera (`qsub_job.sh` and `qsub_job_diff.sh` were not run on e.g. Cheyenne). ## DEPENDENCIES: PR # in `ufs-srweather-app`. ## DOCUMENTATION: Modifications to the documentation are in PR #. ## ISSUE: Partially resolves issue #781 (still need to introduce tests in `regional_workflow` that run the plotting scripts). ## CONTRIBUTORS: @mark-a-potts pointed out the problem and first made the necessary fixes in the `NOAA-EPIC` fork of `regional_workflow`; @gsketefian introduced those changes to this PR and made edits to the comments and documentation. * created new non-compact grids * fixed typo * Update scripts to work with the latest hashes of UFS_UTILS and UPP (#775) * update input namelist of chgres_cube * update diag_table templates * update scripts * back to original * specify miniconda version on Jet Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Co-authored-by: Natalie Perlin <68030316+natalie-perlin@users.noreply.github.com> Co-authored-by: Natalie Perlin <perlin.natalie@gmail.com> Co-authored-by: gsketefian <31046882+gsketefian@users.noreply.github.com> Co-authored-by: Chan-Hoo.Jeon-NOAA <60152248+chan-hoo@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DESCRIPTION OF CHANGES:
This PR fixes a bug in the way the number of time steps (
num_time_stepsinget_WE2Etest_names_subdirs_descs.sh) is calculated for each WE2E test when creating the CSV file containing information about the available WE2E tests.TESTS CONDUCTED:
Reran the
run_WE2E_tests.shscript (which in turn callsget_WE2Etest_names_subdirs_descs.sh), and a CSV file containing the adjusted numbers was successfully created.DOCUMENTATION:
None needed.