Adding new E2E non-compact grids #786
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…unity#766) * update IC/LBC file paths to match new IC/LBC file structure * change forecast length * Update config.community.sh These changes reflects the SRW team's preference for a 12-hours forecast. The gfs.t18z.pgrb2.0p25.f012 file will need to be added to all Level 1 systems. Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Natalie Perlin <perlin.natalie@gmail.com>
…2E test info (ufs-community#776) ## DESCRIPTION OF CHANGES: This PR modifies the script `get_WE2Etest_names_subdirs_descs.sh` that creates the CSV file containing test information the columns for relative cost and time step (`DT_ATMOS`). This allows users to have a better idea of what the cost of each test is (the relative cost is defined such that 1 unit represents the cost of running a single 6-hour forecast on the `RRFS_CONUS_25km` grid). Other related changes: * Add the flag `generate_csv_file` to the `run_WE2E_tests.sh` script to allow users to skip the CSV file generation step (since it can take a while). * Use arguments for the function `set_predef_grid_params()` so that it can be called not just from the experiment generation scripts but also others (in this case from `get_WE2Etest_names_subdirs_descs.sh`). Necessary for this PR. * Move some constants (`NH0`, `NH3`, `NH4`) from `config_defaults.sh` to `constants.sh` to make it easier to access them. * Place the contents of `constants.sh` at the start of each experiment's `var_defns.sh` so that these constants are available to the j-jobs and ex-scripts when `var_defns.sh` is sourced. * Use variable `valid_vals_BOOLEAN` in many more places (which allows many other `valid_vals_...` variables in `valid_param_vals.sh` and `setup.sh` to be removed). * Several other bug fixes. ## TESTS CONDUCTED: The following tests (with results) were conducted on Hera. There were only three (previously existing) failures. ``` * grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2: SUCCESS * grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16: FAILURE * grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_HRRR: SUCCESS * grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_RRFS_v1beta: SUCCESS * grid_RRFS_CONUScompact_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1alpha: SUCCESS * grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta: SUCCESS * grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16: SUCCESS * grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_HRRR: SUCCESS * grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta: SUCCESS * GST_release_public_v1: SUCCESS * MET_ensemble_verification: SUCCESS * MET_verification: SUCCESS * community_ensemble_008mems: SUCCESS * community_ensemble_2mems: SUCCESS * custom_ESGgrid: SUCCESS * custom_GFDLgrid: SUCCESS * custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_FALSE: SUCCESS * custom_GFDLgrid__GFDLgrid_USE_GFDLgrid_RES_IN_FILENAMES_eq_TRUE: SUCCESS * deactivate_tasks: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019061200: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019101818: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2020022518: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2020022600: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2021010100: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019061200: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019101818: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2020022518: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2020022600: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2021010100: SUCCESS * get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_netcdf_2021062000: SUCCESS * get_from_HPSS_ics_GSMGFS_lbcs_GSMGFS: SUCCESS * get_from_HPSS_ics_HRRR_lbcs_RAP: SUCCESS * get_from_HPSS_ics_RAP_lbcs_RAP: SUCCESS * get_from_NOMADS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio: SUCCESS * inline_post: SUCCESS * nco_ensemble: SUCCESS * nco_inline_post: SUCCESS * pregen_grid_orog_sfc_climo: SUCCESS * specify_DOT_OR_USCORE: SUCCESS * specify_DT_ATMOS_LAYOUT_XY_BLOCKSIZE: SUCCESS * specify_EXTRN_MDL_SYSBASEDIR_ICS_LBCS: SUCCESS * specify_RESTART_INTERVAL: SUCCESS * specify_template_filenames: SUCCESS * subhourly_post: FAILURE * subhourly_post_ensemble_2mems: FAILURE ``` ## DOCUMENTATION: Documentation is updated via PR #[278](ufs-community/ufs-srweather-app#278) in the ``ufs-srweather-app`` repo.
ufs-community#782) ## DESCRIPTION OF CHANGES: This PR fixes a bug in the way the number of time steps (`num_time_steps` in `get_WE2Etest_names_subdirs_descs.sh`) is calculated for each WE2E test when creating the CSV file containing information about the available WE2E tests. ## TESTS CONDUCTED: Reran the `run_WE2E_tests.sh` script (which in turn calls `get_WE2Etest_names_subdirs_descs.sh`), and a CSV file containing the adjusted numbers was successfully created. ## DOCUMENTATION: None needed.
## DESCRIPTION OF CHANGES: PR #[763](ufs-community#763) introduced the experiment variable `POST_OUTPUT_DOMAIN_NAME` and used it to modify the names of the UPP output files. This PR adds `POST_OUTPUT_DOMAIN_NAME` as an argument to the python plotting scripts to enable those scripts to form the proper UPP output file names to read in. It also modifies the batch scripts (`qsub_job.sh`, `qsub_job_diff.sh`, `sq_job.sh`, and `sq_job_diff.sh`) to add the new argument to the calls to the python plotting scripts that are called. ## TESTS CONDUCTED: Both python plotting scripts `plot_allvars.py` and `plot_allvars_diff.py` were run on Hera on preexisting UPP output files and completed successfully. The batch scripts `sq_job.sh` and `sq_job_diff.sh` were also run successfully on Hera (`qsub_job.sh` and `qsub_job_diff.sh` were not run on e.g. Cheyenne). ## DEPENDENCIES: PR # in `ufs-srweather-app`. ## DOCUMENTATION: Modifications to the documentation are in PR #. ## ISSUE: Partially resolves issue ufs-community#781 (still need to introduce tests in `regional_workflow` that run the plotting scripts). ## CONTRIBUTORS: @mark-a-potts pointed out the problem and first made the necessary fixes in the `NOAA-EPIC` fork of `regional_workflow`; @gsketefian introduced those changes to this PR and made edits to the comments and documentation.
EdwardSnyder-NOAA
requested review from
mkavulich,
gsketefian,
JeffBeck-NOAA,
RatkoVasic-NOAA,
BenjaminBlake-NOAA,
ywangwof,
chan-hoo,
panll,
christinaholtNOAA,
christopherwharrop-noaa,
danielabdi-noaa,
mark-a-potts,
jkbk2004,
willmayfield,
dmwright526,
gspetro-NOAA and
natalie-perlin
as code owners
|
@JeffBeck-NOAA @BenjaminBlake-NOAA @RatkoVasic-NOAA I advised @EdwardSnyder-NOAA to add the six tests |
|
Agree with this. Thanks, @gsketefian, @EdwardSnyder-NOAA! |
JeffBeck-NOAA
approved these changes
Jun 8, 2022
|
@EdwardSnyder-NOAA I'm just waiting to hear back from EMC (@BenjaminBlake-NOAA or @RatkoVasic-NOAA) and if they're ok with it, we can approve and merge. |
RatkoVasic-NOAA
approved these changes
Jun 8, 2022
|
@EdwardSnyder-NOAA Go ahead and merge after you can confirm that the 3km tests passed. Thanks for creating this PR. |
|
@EdwardSnyder-NOAA And please fix up the documentation section of the PR message. Currently says "Documentation might need to be" and that's all. |
BenjaminBlake-NOAA
approved these changes
Jun 9, 2022
…fs-community#775) * update input namelist of chgres_cube * update diag_table templates * update scripts * back to original * specify miniconda version on Jet
|
@gsketefian 3- I've updated the documentation section. The 3km tests ran successfully on the Parallel Works AWS cluster. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DESCRIPTION OF CHANGES:
Added 6 new non-compact grids for testing. See the list below. These tests will be included in the upcoming SRW v2 Release. These changes will be merged into the develop branch as well.
TESTS CONDUCTED:
The first four tests listed above ran on Orion successfully. The last two, RRFS_CONUS_3km, grids ran successfully on Parallel Works AWS cluster.
DEPENDENCIES:
No Dependencies.
DOCUMENTATION:
No documentation changes are needed.
ISSUE (optional):
CONTRIBUTORS (optional):