Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: Updated exregional_get_obs scripts #833

Merged

Conversation

michelleharrold
Copy link
Collaborator

DESCRIPTION OF CHANGES:

This PR addressed Issue #823. For more recent dates (June 2022 and beyond), the exregional_get_obs scripts fail because the tar files names on the NOAA HPSS were updated but not updated accordingly in the pull scripts. A simple fix of modifying the logic to account for mapping the valid date to the tar file name have been updated in scripts/exregional_get_obs_ccpa.sh, scripts/exregional_get_obs_mrms.sh, and scripts/exregional_get_obs_ndas.sh.

As a note: To avoid having to continually update file name changes/location on the HPSS, we will be looking to leverage some of the capabilities being used for pulling DA observations and overhauling the verification observation pull scripts. @mkavulich will be looking into this soon.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

TESTS CONDUCTED:

The modifications were tested on Hera only. Several dates were tested (20220619, 20230515) for the FV3_HRRR suite over RRFS_CONUS_25km domain. 24-h simulations were run and included pulling obs from NOAA HPSS and running verification.

  • hera.intel
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • NOAA Cloud (indicate which platform)
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

None.

DOCUMENTATION:

N/A.

ISSUE:

Fixes issue mentioned in #823.

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • Needs Cheyenne test
  • Needs Jet test
  • Needs Hera test
  • Needs Orion test
  • help wanted

CONTRIBUTORS (optional):

@@ -79,7 +79,7 @@ while [[ ${current_fcst} -le ${fcst_length} ]]; do
vdd=`echo ${vdate} | cut -c7-8` # day (DD) of valid time
vhh=`echo ${vdate} | cut -c9-10` # forecast hour (HH)

vhh_noZero=$(expr ${vhh} + 0)
vhh_noZero=$( printf "%d" "${vhh#0}" )
Copy link
Collaborator

@gsketefian gsketefian Jun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michelleharrold Is the #0 at the end something I suggested in our meeting? If so, sorry, it's not correct! Scrap that and change to $((10#${vhh})) to force ${vhh} to be interpreted as a decimal number. And please run a test!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsketefian -- what you originally suggested worked...until I tested on forecast hour 08 and 09, and then I got the dreaded octet error. So I had to modify it. I will go ahead and change to what you suggested and run it!

Copy link
Collaborator

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michelleharrold Tests with different dates are passing without issue. I have also ran the fundamental WE2E tests on Hera Intel and they all passed. I will now approve this PR.

@MichaelLueken MichaelLueken added the run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests label Jun 22, 2023
@MichaelLueken
Copy link
Collaborator

@michelleharrold The automated Jenkins tests passed on Cheyenne, Jet, and Hera. The WE2E coverage tests were manually run on Orion and all passed. Moving forward with merging this PR now.

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
deactivate_tasks                                                   COMPLETE               1.20
get_from_AWS_ics_GEFS_lbcs_GEFS_fmt_grib2_2022040400_ensemble_2me  COMPLETE             755.87
grid_CONUS_3km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_RRFS_v1beta   COMPLETE             257.96
grid_RRFS_AK_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot        COMPLETE             141.27
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_RRFS_v1beta            COMPLETE              14.20
grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp  COMPLETE              10.24
grid_RRFS_CONUS_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR              COMPLETE             398.33
grid_RRFS_CONUScompact_13km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16   COMPLETE              28.01
grid_RRFS_CONUScompact_3km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16    COMPLETE             279.89
grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0         COMPLETE              12.59
nco                                                                COMPLETE               7.64
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE            1907.20

@MichaelLueken MichaelLueken merged commit 7edaff8 into ufs-community:develop Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exregional_get_obs scripts for verification do not work for recent dates
5 participants