Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port RadMon to Hera #3

Closed
EdwardSafford-NOAA opened this issue Jun 4, 2020 · 1 comment · Fixed by #10
Closed

Port RadMon to Hera #3

EdwardSafford-NOAA opened this issue Jun 4, 2020 · 1 comment · Fixed by #10
Assignees
@EdwardSafford-NOAA

Comments

@EdwardSafford-NOAA
Copy link
Contributor

This is a re-issue of Redmine ticket #76267 ( https://vlab.ncep.noaa.gov/redmine/issues/76267 ).

Original ticket description: I thought the conversion from theia to hera had been done long ago but it's not in ProdGSI/master. Perhaps I'm confusing that with the conversion to slurm job controls. Either way, the task is to port the RadMon to theia, making the appropriate /scratchX and module changes.
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 4, 2020
@EdwardSafford-NOAA
Ref NOAA-EMC#3
762c755 
Rm wcoss, phase 1 test driver script.
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 4, 2020
@EdwardSafford-NOAA
Ref NOAA-EMC#3
acd4e84 
Rm sendmsg statements in radmon_verf scripts.
@EdwardSafford-NOAA
Copy link
Contributor Author

Changes have been retested on wcoss_d and hera. The radmon_verf_* extraction files were cleaned up to remove the sendmsg calls, and the driver script for wcoss phase 1 was removed.

EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 8, 2020
@EdwardSafford-NOAA
Ref NOAA-EMC#3
264307a 
Add missing gdas_radmon_base.tar file.
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 8, 2020
@EdwardSafford-NOAA
Ref NOAA-EMC#3
feef7c8 
Update hera driver script, rm sendmsg
calls in J-job and ex script.
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 8, 2020
@EdwardSafford-NOAA
Ref NOAA-EMC#3
7f43901 
Missed some cleanup on hera's driver script.
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - initial add of manage_externals and needed Externa…
bfc7bb0 
…ls.cfg. Also added .gitignore file and removed scripts/files associated with no-longer-used prod_util and grid_util builds.
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - remove copy of manage_externals under util and add…
cf00863 
… README.md file
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - reduce hashes down to minimum 8 characters
d9ea1ac
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - sync merge with develop branch
e46b175
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - Updated README with new manic tag v1.1.7
f662fff
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - Merge branch 'develop' into feature/manage_externals
40084e6
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - update EMC_verif-global tag in Externals.cfg after…
830c73f 
… sync with develop
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - updated link_fv3gfs.sh to adjust wafs links
e602cd3
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - updated README with new manic version
8699b46
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - remove prod_util and grib_util sections from build…
e83b90d 
…_all.sh, removed elsewhere already
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Issue NOAA-EMC#3 - added explicit config flag example for checkout_ex…
622167d 
…ternals in README and blurb about this replacing checkout.sh
KarinaApodaca-NOAA pushed a commit to KarinaApodaca-NOAA/GSI that referenced this issue Jun 9, 2020
@KateFriedman-NOAA
Merge pull request NOAA-EMC#29 from NOAA-EMC/feature/manage_externals
057b2a8 
Issue NOAA-EMC#3 - Introduce manage_externals as replacement for checkout.sh
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 11, 2020
@EdwardSafford-NOAA
GitHub Issue NOAA-EMC/GSI NOAA-EMC#3. Port RadMon to Hera.
d1082a1
@MichaelLueken MichaelLueken linked a pull request Jun 11, 2020 that will close this issue
GitHub Issue NOAA-EMC/GSI#3. Port RadMon to Hera. #10
Merged
EdwardSafford-NOAA added a commit to EdwardSafford-NOAA/GSI that referenced this issue Jun 12, 2020
@EdwardSafford-NOAA
GitHub Issue NOAA-EMC#3. Port RadMon to Hera.
9541c04
MichaelLueken added a commit that referenced this issue Jun 12, 2020
@MichaelLueken
Merge pull request #10 from EdwardSafford-NOAA/master
e2a33cc 
GitHub Issue #3.  Port RadMon to Hera.
@emilyhcliu emilyhcliu mentioned this issue Jul 22, 2021
Closed
9 tasks
@ADCollard ADCollard mentioned this issue Oct 15, 2021
Closed
@ScottSieron-NOAA ScottSieron-NOAA mentioned this issue Dec 15, 2021
Closed
hu5970 referenced this issue in hu5970/GSI Dec 16, 2021
@guoqing-noaa
add an option to allow GSI to compile using specific kJet optimization (
fb2c427 
#3)
@XuLi-NOAA XuLi-NOAA mentioned this issue Feb 28, 2022
Closed
lbi2018 pushed a commit to lbi2018/GSI that referenced this issue Mar 4, 2022
@BinLiu-NOAA
Minor modifications for dual resolution regional ensembles (NOAA-EMC#3)
636b622 
A couple of modifications for fv3-lam dual resolution from @TingLei-NOAA
@HaidaoLin-NOAA HaidaoLin-NOAA mentioned this issue Sep 30, 2023
Merged
7 tasks
@ShunLiu-NOAA ShunLiu-NOAA mentioned this issue Oct 4, 2023
Merged
9 tasks
@JingCheng-NOAA JingCheng-NOAA mentioned this issue Oct 26, 2023
Merged
8 tasks
@hongli-wang hongli-wang mentioned this issue Oct 27, 2023
Merged
6 tasks
@XuLu-NOAA XuLu-NOAA mentioned this issue Nov 16, 2023
Merged
@jderber-NOAA jderber-NOAA mentioned this issue Dec 14, 2023
Merged
@xyzemc xyzemc mentioned this issue Dec 19, 2023
Merged
jderber-NOAA added a commit to jderber-NOAA/GSI that referenced this issue Jan 7, 2024
@jderber-NOAA
Changes to code based on shoyokota changes NOAA-EMC#3
e123548
@yonghuiweng yonghuiweng mentioned this issue Jan 12, 2024
Merged
@TingLei-NOAA TingLei-NOAA mentioned this issue Jan 22, 2024
Merged
6 tasks
@xyzemc xyzemc mentioned this issue Jan 29, 2024
Merged
@ClaraDraper-NOAA ClaraDraper-NOAA mentioned this issue Feb 6, 2024
Merged
3 tasks
@yonghuiweng yonghuiweng mentioned this issue Feb 17, 2024
Closed
@wx20jjung wx20jjung mentioned this issue Feb 26, 2024
Merged
1 task
@xincjin-NOAA xincjin-NOAA mentioned this issue Mar 1, 2024
Merged
7 tasks
@JingCheng-NOAA JingCheng-NOAA mentioned this issue Mar 4, 2024
Merged
ShunLiu-NOAA pushed a commit that referenced this issue Mar 27, 2024
@XuLu-NOAA
Fix HAFS GSI debug build and run issues (#679)
b53740a 
**DUE DATE for merger of this PR into `develop` is 2/19/2024 (six weeks
after PR creation).**
**DUE DATE for this PR is extended to 3/19/2024 because @XuLu-NOAA is on
leave.**
**Description**

Xu Lu (xu.lu@noaa.gov) and Biju Thomas (biju.thomas@noaa.gov) fixed bugs
regarding HAFS GSI debug build and run issues. This is in corresponding
to issue #661

Fixes #661

1. In read_radar.f90, uninitialized toff is making all the ground-based
radar observations be placed at -3h instead of 0h, which creates wrong
increments for FGAT and 4DEnVar.
2. In read_radar.f90, uninitialized zsges will crash the debug mode.
3. In read_radar.f90, t4dvo should be used instead of t4dv in the
read_radar_l2rw_novadqc subroutine.
4. In radinfo.90, maxscan should be increased to at least 252 to allow
more scans, or it will crash the debug mode.
5. In read_fl_hdob.f90, dlnpsob is replaced with 1000. since the SFMR
does not sample surface pressure, and the uninitialized dlnpsob creates
issues later in setupspd.f90 in the debug mode.
6. In mod_fv3_lola.f90, (i,j+1) should be used instead of (i+1,j) in
searching for V edges.
7. In stpcalc.f90, when tried to find the best stepsize from outpen
around L838-864, the minimum outstp(i) is stored in stp(ii), but the
istp_use is asigned with i instead of ii. Create inconsistency when
assigning stp(istp_use) to stpinout at L872. Should use istp_use=ii
instead.

**Type of change**
- [Yes] Bug fix (non-breaking change which fixes an issue)

**How Has This Been Tested?**
Regression test on Orion:
```
Test project /work/noaa/hwrf/save/xulu/mergeversions/GSI/build
CMake Warning (dev) at CTestTestfile.cmake:9 (subdirs):  Syntax Warning in cmake code at /work/noaa/hwrf/save/xulu/mergeversions/GSI/build/regression/CTestTestfile.cmake:7:10
1/7 Test #4: [=[netcdf_fv3_regional]=] ........   Passed  365.11 sec
2/7 Test #7: [=[global_enkf]=] ................   Passed  430.29 sec
3/7 Test #3: [=[rrfs_3denvar_glbens]=] ........   Passed  605.35 sec
4/7 Test #2: [=[rtma]=] .......................   Passed  969.78 sec
5/7 Test #6: [=[hafs_3denvar_hybens]=] ........***Failed  1455.47 sec
6/7 Test #1: [=[global_4denvar]=] .............   Passed  1682.40 sec
7/7 Test #5: [=[hafs_4denvar_glbens]=] ........***Failed  1758.90 sec
```

The failed hafs_3denvar and 4denvar are within expectation due to the
fix for toff. As demonstrated in the single observation tests in the
following figure, the uninitialized toff can result in increment
degradations due to wrongly assigned observation times:

![image](https://github.com/NOAA-EMC/GSI/assets/26603014/0de870e1-f8c8-4b6d-8039-57f417b76367)
@JingCheng-NOAA JingCheng-NOAA mentioned this issue Mar 28, 2024
Merged
9 tasks
ShunLiu-NOAA pushed a commit that referenced this issue Apr 5, 2024
@GangZhao-NOAA
Adding I/O for direct analysis of near-surface wind gust for RRFS-bas…
29d9d8f 
…ed 3DRTMA (#730)

<!-- PLEASE READ -->
<!--
Before opening a PR, please note these guidelines:

- Each PR should only address ONE topic and have an associated issue
- No hardcoded or paths to personal directories should be present
- No temporary or backup files should be committed
- Any code that was disabled by being commented out should be removed
-->

**Description**

<!-- Please include relevant motivation and context. -->
<!-- Please include a summary of the change and which issue is fixed.
-->
<!-- List any dependencies that are required for this change. -->
To improve the analysis of the near-surface wind gust in 3DRTMA, the
observations of near-surface wind gust would be analyzed directly in GSI
(3DVar and Hybrid 3DEnVar), instead of being a derived product from the
near-surface wind analysis.

Since the core subroutines for direct variational assimilation of wind
gust (e.g., setupgust.f90, intgust.f90, stpgust.f90, etc.) had already
been implemented in GSI for 2DRTMA, so in the work the development in
GSI mainly focus on adding I/O of 2-D wind gust firstguess and analysis
fields for RRFS-based 3DRTMA, and some minor modifications in
observation and background error for wind gust, options to control the
analysis of wind gust, etc.
<!-- Please provide reference to the issue this pull request is
addressing. -->
<!-- For e.g. Fixes #IssueNumber -->
This PR is to address the issue #726 : Adding I/O for direct analysis of
near-surface wind gust for RRFS-based 3DRTMA
Fixes #726
**Type of change**

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

**How Has This Been Tested?**

<!-- Please describe the tests that you ran to verify your changes and
on the platforms these tests were conducted. -->
<!-- Provide instructions so we can reproduce. -->
<!-- Please also list any relevant details for your test configuration
-->
  
**Checklist**

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] New and existing tests pass with my changes
      tested with a real case - 2024-02-20_12:00Z, 
1. dry-run (using my updated GSI code with wind gust analysis, but
actually no wind gust obs is analyzed, so-called dryrun) is compared to
control-run (original GSI code running without wind gust obs): the
results are identical. This indicates that if without analyzing
wind-gust obs, then the updated code generates the analysis identical to
the analysis of original/control code. Or say, the added code does not
have influence on the other part of code.
2. real case run with updated GSI code to analyze the obs of wind gust:
The following figure shows the used observations of near-surface wind
gust:

![var_obs_2024022012_gust_used_maprll_datll_reg_ncf](https://github.com/NOAA-EMC/GSI/assets/53267411/ecbe479a-03c6-490f-a179-9e0027291468)
the following figure shows the analysis increments:

![GUST_hyb_hwllp90_corptuned_inc_incrintrp_maprll_datrll_reg_grb2](https://github.com/NOAA-EMC/GSI/assets/53267411/a01fca0d-dc1f-438b-b8eb-e624de35a631)
- [x] Any dependent changes have been merged and published
- [x] Regression tests on WCOSS2 (Cactus) and Hera (Rocky-8) : my
updated GSI commit
[#f91f247d](GangZhao-NOAA@f91f247))
vs control/original GSI code (commit
[#6d9ebbb7](6d9ebbb))
Here is the reports of the regression tests on WCOSS2 (Cactus):
~~~
[gang.zhao@clogin02:build] (feature/windgust_in_3dvar_for_3drtma)$ ctest
-j 7
Test project
/lfs/h2/emc/da/save/gang.zhao/WorkDir/ProdGSI_Dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  483.15 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  486.74 sec
3/7 Test #7: global_enkf ......................   Passed  850.98 sec
4/7 Test #2: rtma .............................   Passed  970.28 sec
5/7 Test #6: hafs_3denvar_hybens ..............   Passed  1152.82 sec
6/7 Test #5: hafs_4denvar_glbens ..............   Passed  1213.93 sec
7/7 Test #1: global_4denvar ...................   Passed  1683.16 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 1683.19 sec
~~~
Here is the reports of the regression tests on Hera (Rocky8):
~~~
(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$
ctest -j 7
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  491.53 sec
2/7 Test #3: rrfs_3denvar_glbens ..............***Failed  495.27 sec
3/7 Test #2: rtma .............................   Passed  982.45 sec
4/7 Test #6: hafs_3denvar_hybens ..............   Passed  1168.99 sec
5/7 Test #7: global_enkf ......................   Passed  1239.77 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  1347.87 sec
7/7 Test #1: global_4denvar ...................   Passed  1974.45 sec

71% tests passed, 2 tests failed out of 7

Total Test time (real) = 1974.91 sec

The following tests FAILED:
          3 - rrfs_3denvar_glbens (Failed)
          5 - hafs_4denvar_glbens (Failed)
Errors while running CTest
Output from these tests are in:
/scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases
verbosely.
(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$
ctest -R rrfs_3denvar_glbens
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 3: rrfs_3denvar_glbens
1/1 Test #3: rrfs_3denvar_glbens ..............   Passed  430.52 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 430.55 sec
(base) [Gang.Zhao@hfe11:build] (feature/windgust_in_3dvar_for_3drtma)$
ctest -R hafs_4denvar_glbens
Test project /scratch1/NCEPDEV/da/Gang.Zhao/ProdGSI_dev/gsi_dev/build
    Start 5: hafs_4denvar_glbens
1/1 Test #5: hafs_4denvar_glbens ..............   Passed  1225.37 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 1225.39 sec
~~~
**Note**: 
_When I was running the regression tests, GSI code was just updated to
the latest commit
[#b53740a7](GangZhao-NOAA@f91f247).
Considering the frequent update in EMC GSI code recently and saving the
time, after this PR has been reviewed and approved by peer-reviewers, I
will update the code to latest EMC GSI commit, then re-run the
regression tests for final approval.
@ADCollard ADCollard mentioned this issue Apr 17, 2024
Merged
5 tasks
@ShunLiu-NOAA ShunLiu-NOAA mentioned this issue Apr 29, 2024
Merged
9 tasks
@wx20jjung wx20jjung mentioned this issue Apr 29, 2024
Merged
6 tasks
@wx20jjung wx20jjung mentioned this issue Jul 11, 2024
Merged
7 tasks
@wx20jjung wx20jjung mentioned this issue Jul 23, 2024
Closed
@wx20jjung wx20jjung mentioned this issue Sep 9, 2024
Merged
@ShunLiu-NOAA ShunLiu-NOAA mentioned this issue Sep 28, 2024
Merged
9 tasks
@wx20jjung wx20jjung mentioned this issue Nov 5, 2024
Merged
5 tasks
@wx20jjung wx20jjung mentioned this issue Dec 16, 2024
Merged
6 tasks
@XuanliLi-NOAA XuanliLi-NOAA mentioned this issue Jan 23, 2025
Open
@wx20jjung wx20jjung mentioned this issue Feb 1, 2025
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EdwardSafford-NOAA EdwardSafford-NOAA

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

GitHub Issue NOAA-EMC/GSI#3. Port RadMon to Hera. EdwardSafford-NOAA/GSI
1 participant
@EdwardSafford-NOAA