Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[develop] Fixing the issue related to NDAS file name change on NOAA HPSS #626

Merged
merged 9 commits into from
Feb 24, 2023
Merged

[develop] Fixing the issue related to NDAS file name change on NOAA HPSS #626

merged 9 commits into from
Feb 24, 2023

Conversation

panll
Copy link
Collaborator

@panll panll commented Feb 22, 2023

DESCRIPTION OF CHANGES:

NDAS file names on NOAA HPSS changed after June 28, 2022. This pull request fixed that issue (#617)

Type of change

  • [ x] Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

TESTS CONDUCTED:

  • [x ] hera.intel
  • orion.intel
  • cheyenne.intel
  • cheyenne.gnu
  • gaea.intel
  • jet.intel
  • wcoss2.intel
  • NOAA Cloud (indicate which platform)
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

N/A

DOCUMENTATION:

N/A

ISSUE:

Fixes issue mentioned in #617

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

@MichaelLueken MichaelLueken changed the title Fixing the issue related to NDAS file name change on NOAA HPSS [develop] Fixing the issue related to NDAS file name change on NOAA HPSS Feb 22, 2023
@MichaelLueken MichaelLueken linked an issue Feb 22, 2023 that may be closed by this pull request
NDAS data file name changed #617
Closed
@JeffBeck-NOAA
Copy link
Collaborator

Thanks for this fix, @panll! After making the change, did you test a date between 20200226 and 20220627, and also a date after 20220627 to make sure all the logic works correctly? If so, I'll approve!

@panll
Copy link
Collaborator Author

panll commented Feb 22, 2023 •
edited
Loading

Thanks for this fix, @panll! After making the change, did you test a date between 20200226 and 20220627, and also a date after 20220627 to make sure all the logic works correctly? If so, I'll approve!

Yes, it was tested on Hera (e.g., 2022062618, 2022062718. Cycle 2022062718 needs the data of 20220628 for 12 hour forecast). Thanks! @JeffBeck-NOAA

@JeffBeck-NOAA
Copy link
Collaborator

Thanks for this fix, @panll! After making the change, did you test a date between 20200226 and 20220627, and also a date after 20220627 to make sure all the logic works correctly? If so, I'll approve!

Yes, it was tested on Hera (e.g., 2022062618, 2022062718. Cycle 2022062718 needs the data of 20220628 for 12 hour forecast). Thanks! @JeffBeck-NOAA

Thanks, @panll! Approving.

@MichaelLueken MichaelLueken added ci-hera-intel-WE Kicks off automated workflow test on hera with intel run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests labels Feb 23, 2023
@venitahagerty venitahagerty removed the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 23, 2023
@venitahagerty
Copy link
Collaborator

venitahagerty commented Feb 23, 2023 •
edited
Loading

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1250519477/20230223170516/ufs-srweather-app
Build was Successful
Rocoto jobs started
Long term tracking will be done on 10 experiments
If test failed, please make changes and add the following label back:
ci-hera-intel-WE
Experiment Failed on hera: pregen_grid_orog_sfc_climo
2023-02-23 17:40:19 +0000 :: hfe02 :: Task make_lbcs, jobid=42291579, in state DEAD (FAILED), ran for 16.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: MET_ensemble_verification
2023-02-23 17:44:14 +0000 :: hfe11 :: Task make_sfc_climo, jobid=42291646, in state DEAD (FAILED), ran for 20.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-02-23 17:44:13 +0000 :: hfe07 :: Task make_sfc_climo, jobid=42291636, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
2023-02-23 17:44:13 +0000 :: hfe04 :: Task make_sfc_climo, jobid=42291616, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
2023-02-23 17:44:10 +0000 :: hfe03 :: Task make_sfc_climo, jobid=42291619, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
2023-02-23 17:44:09 +0000 :: hfe10 :: Task make_sfc_climo, jobid=42291649, in state DEAD (FAILED), ran for 17.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
2023-02-23 17:44:11 +0000 :: hfe05 :: Task make_sfc_climo, jobid=42291626, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
2023-02-23 17:44:07 +0000 :: hfe05 :: Task make_sfc_climo, jobid=42291632, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
2023-02-23 17:44:06 +0000 :: hfe12 :: Task make_sfc_climo, jobid=42291639, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Succeeded on hera: community_ensemble_2mems_stoch
All experiments completed

@MichaelLueken MichaelLueken added the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 23, 2023
@venitahagerty venitahagerty removed the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 23, 2023
@venitahagerty
Copy link
Collaborator

venitahagerty commented Feb 23, 2023 •
edited
Loading

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1250519477/20230223193510/ufs-srweather-app
Build was Successful
Rocoto jobs started
Long term tracking will be done on 10 experiments
If test failed, please make changes and add the following label back:
ci-hera-intel-WE
Experiment Failed on hera: pregen_grid_orog_sfc_climo
2023-02-23 20:08:12 +0000 :: hfe08 :: Task make_ics, jobid=42297078, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: pregen_grid_orog_sfc_climo
2023-02-23 20:08:12 +0000 :: hfe08 :: Task make_lbcs, jobid=42297079, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
2023-02-23 20:12:06 +0000 :: hfe03 :: Task make_sfc_climo, jobid=42297152, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-02-23 20:12:11 +0000 :: hfe04 :: Task make_sfc_climo, jobid=42297159, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
2023-02-23 20:12:13 +0000 :: hfe06 :: Task make_sfc_climo, jobid=42297158, in state DEAD (FAILED), ran for 23.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: MET_ensemble_verification
2023-02-23 20:12:14 +0000 :: hfe11 :: Task make_sfc_climo, jobid=42297147, in state DEAD (FAILED), ran for 20.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
2023-02-23 20:12:10 +0000 :: hfe02 :: Task make_sfc_climo, jobid=42297161, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
2023-02-23 20:12:13 +0000 :: hfe08 :: Task make_sfc_climo, jobid=42297153, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
2023-02-23 20:12:07 +0000 :: hfe02 :: Task make_sfc_climo, jobid=42297160, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Succeeded on hera: community_ensemble_2mems_stoch
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
2023-02-23 20:12:15 +0000 :: hfe05 :: Task make_sfc_climo, jobid=42297146, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
All experiments completed

@MichaelLueken
Copy link
Collaborator

@panll The Jenkins tests have all passed (the Orion tests were aborted since the machine is still down for maintenance, which is why the report is showing that the build of this commit was aborted, the tests on Cheyenne, Jet, and Gaea all passed). However, there are issues with the GiHub Action ci-hera-intel-WE tests. Only the community_ensemble_2mems_stoch is passing. When you ran the Hera Intel tests, did you encounter issues? I want to get these changes in, but I'm feeling apprehensive moving forward while the Hera GitHub Action tests are failing. Thanks!

@panll
Copy link
Collaborator Author

panll commented Feb 23, 2023 •
edited
Loading

@MichaelLueken I have no issue on Hera. What kind of error message do you get? This pull request only change NDAS data downloading and will not impact other parts (e.g., make_sfc_climo).

@MichaelLueken
Copy link
Collaborator

@panll The ci-hera-intel-WE tests are returning:

Experiment Failed on hera: pregen_grid_orog_sfc_climo
2023-02-23 17:40:19 +0000 :: hfe02 :: Task make_lbcs, jobid=42291579, in state DEAD (FAILED), ran for 16.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: MET_ensemble_verification
2023-02-23 17:44:14 +0000 :: hfe11 :: Task make_sfc_climo, jobid=42291646, in state DEAD (FAILED), ran for 20.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-02-23 17:44:13 +0000 :: hfe07 :: Task make_sfc_climo, jobid=42291636, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
2023-02-23 17:44:13 +0000 :: hfe04 :: Task make_sfc_climo, jobid=42291616, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
2023-02-23 17:44:10 +0000 :: hfe03 :: Task make_sfc_climo, jobid=42291619, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
2023-02-23 17:44:09 +0000 :: hfe10 :: Task make_sfc_climo, jobid=42291649, in state DEAD (FAILED), ran for 17.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
2023-02-23 17:44:11 +0000 :: hfe05 :: Task make_sfc_climo, jobid=42291626, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
2023-02-23 17:44:07 +0000 :: hfe05 :: Task make_sfc_climo, jobid=42291632, in state DEAD (FAILED), ran for 19.0 seconds, exit status=256, try=2 (of 2)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
2023-02-23 17:44:06 +0000 :: hfe12 :: Task make_sfc_climo, jobid=42291639, in state DEAD (FAILED), ran for 18.0 seconds, exit status=256, try=2 (of 2)
Experiment Succeeded on hera: community_ensemble_2mems_stoch

It looks like it might be a space issue or something else. As noted, the Jenkins tests are fine, it's just the automated tests on Hera that are failing now. I'll try submitting them again and see what happens. Thanks!

@MichaelLueken MichaelLueken added the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 23, 2023
@venitahagerty venitahagerty removed the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 23, 2023
@venitahagerty
Copy link
Collaborator

venitahagerty commented Feb 23, 2023 •
edited
Loading

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1250519477/20230223212013/ufs-srweather-app
Build was Successful
Rocoto jobs started
Long term tracking will be done on 10 experiments
If test failed, please make changes and add the following label back:
ci-hera-intel-WE
Experiment Succeeded on hera: pregen_grid_orog_sfc_climo
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
2023-02-23 22:00:16 +0000 :: hfe01 :: Task make_ics, jobid=42300311, in state DEAD (FAILED), ran for 39.0 seconds, exit status=256, try=2 (of 2)
Experiment Succeeded on hera: community_ensemble_2mems_stoch
Experiment Failed on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
2023-02-23 22:00:13 +0000 :: hfe05 :: Task make_ics, jobid=42300319, in state DEAD (FAILED), ran for 45.0 seconds, exit status=256, try=2 (of 2)
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
Experiment Succeeded on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
Experiment Succeeded on hera: MET_ensemble_verification
All experiments completed

@MichaelLueken MichaelLueken added the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 24, 2023
@venitahagerty venitahagerty removed the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 24, 2023
@venitahagerty
Copy link
Collaborator

venitahagerty commented Feb 24, 2023 •
edited
Loading

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1250519477/20230224142011/ufs-srweather-app
Build was Successful
Rocoto jobs started
Long term tracking will be done on 10 experiments
If test failed, please make changes and add the following label back:
ci-hera-intel-WE
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
2023-02-24 15:04:05 +0000 :: hfe01 :: Task run_fcst, jobid=42326696, in state DEAD (FAILED), ran for 106.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
2023-02-24 15:08:14 +0000 :: hfe12 :: Task run_fcst, jobid=42326768, in state DEAD (FAILED), ran for 106.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-02-24 15:08:07 +0000 :: hfe01 :: Task run_fcst, jobid=42326769, in state DEAD (FAILED), ran for 139.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
2023-02-24 15:04:14 +0000 :: hfe07 :: Task run_fcst, jobid=42326679, in state DEAD (FAILED), ran for 120.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
2023-02-24 15:04:08 +0000 :: hfe10 :: Task run_fcst, jobid=42326666, in state DEAD (FAILED), ran for 104.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: MET_ensemble_verification
2023-02-24 15:04:08 +0000 :: hfe07 :: Task run_fcst_mem001, jobid=42326686, in state DEAD (FAILED), ran for 106.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: MET_ensemble_verification
2023-02-24 15:04:08 +0000 :: hfe07 :: Task run_fcst_mem002, jobid=42326687, in state DEAD (FAILED), ran for 104.0 seconds, exit status=256, try=1 (of 1)
Experiment Failed on hera: pregen_grid_orog_sfc_climo
2023-02-24 15:00:17 +0000 :: hfe10 :: Task run_fcst, jobid=42326630, in state DEAD (FAILED), ran for 107.0 seconds, exit status=256, try=1 (of 1)
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
Experiment Succeeded on hera: community_ensemble_2mems_stoch
Experiment Succeeded on hera: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_HRRR
All experiments completed

MichaelLueken
MichaelLueken approved these changes Feb 24, 2023
View reviewed changes
Copy link
Collaborator

@MichaelLueken MichaelLueken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@panll Thank you for fixing this issue! As noted yesterday, the Jenkins tests successfully passed and, ultimately, all of the fundamental tests from the ci-hera-intel-WE tests have passed. I will now give my approval to these changes.

@MichaelLueken MichaelLueken merged commit 967186f into ufs-community:develop Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MichaelLueken MichaelLueken MichaelLueken approved these changes

@JeffBeck-NOAA JeffBeck-NOAA JeffBeck-NOAA approved these changes

@mkavulich mkavulich Awaiting requested review from mkavulich mkavulich is a code owner

@gsketefian gsketefian Awaiting requested review from gsketefian gsketefian is a code owner

@RatkoVasic-NOAA RatkoVasic-NOAA Awaiting requested review from RatkoVasic-NOAA RatkoVasic-NOAA is a code owner

@BenjaminBlake-NOAA BenjaminBlake-NOAA Awaiting requested review from BenjaminBlake-NOAA BenjaminBlake-NOAA is a code owner

@ywangwof ywangwof Awaiting requested review from ywangwof ywangwof is a code owner

@chan-hoo chan-hoo Awaiting requested review from chan-hoo chan-hoo is a code owner

@christinaholtNOAA christinaholtNOAA Awaiting requested review from christinaholtNOAA christinaholtNOAA is a code owner

@christopherwharrop-noaa christopherwharrop-noaa Awaiting requested review from christopherwharrop-noaa christopherwharrop-noaa is a code owner

@danielabdi-noaa danielabdi-noaa Awaiting requested review from danielabdi-noaa danielabdi-noaa is a code owner

@mark-a-potts mark-a-potts Awaiting requested review from mark-a-potts mark-a-potts is a code owner

@jkbk2004 jkbk2004 Awaiting requested review from jkbk2004 jkbk2004 is a code owner

@willmayfield willmayfield Awaiting requested review from willmayfield willmayfield is a code owner

@dmwright526 dmwright526 Awaiting requested review from dmwright526 dmwright526 is a code owner

@gspetro-NOAA gspetro-NOAA Awaiting requested review from gspetro-NOAA gspetro-NOAA is a code owner

@natalie-perlin natalie-perlin Awaiting requested review from natalie-perlin natalie-perlin is a code owner

@EdwardSnyder-NOAA EdwardSnyder-NOAA Awaiting requested review from EdwardSnyder-NOAA EdwardSnyder-NOAA is a code owner

Assignees
No one assigned
Labels
run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

NDAS data file name changed
5 participants
@panll @JeffBeck-NOAA @venitahagerty @MichaelLueken @linlin-pan