Porting UPP for gfs v16 on wcoss2 #420
Conversation
building upp lib.
|
@WenMeng-NOAA why are manage_externals changes required here? |
|
@aerorahul @KateFriedman-NOAA found a new version of manage_externals is needed for the global-workflow checking gtg code. |
@WenMeng-NOAA Can the GTG checkout happen without manage_externals? I use whatever method you provide on the workflow side to do the checkouts. We're trying to move away from manage_externals now for a few reasons. Can we clone the GTG with submodule commands instead? Thanks! |
There was a problem hiding this comment.
Non-manage-externals-related changes look good and cover the changes Wen and I discussed when I initially tested the post jobs on WCOSS2. I am rebuilding my copy of the UPP with the recent LUA updates and will do the following for this PR:
- retest post jobs on Dogwood with updated UPP copy and have Wen look at output
- update my copy of the UPP to use submodules instead of manage_externals; will provide needed updates for Wen to incorporate
Stay tuned...
@KateFriedman-NOAA Yes, @aerorahul has been helping me on configurating gtg code checking out as a submodule. |
|
@WenMeng-NOAA Please see the following log from rebuilding the UPP after recent updates, please confirm the build went as intended: /lfs/h2/emc/global/noscrub/Kate.Friedman/git/feature-ops-wcoss2/sorc/logs/build_ncep_post_test5.log Thanks! Note: this is before I make any changes for the GTG checkout |
@KateFriedman-NOAA Your build log look good to me. Thanks! |
Thanks for confirming @WenMeng-NOAA ! I'm working on the patch to retire manage_externals and use submodules instead for the GTG checkout. Will provide the patch to include in this PR when ready. |
|
@WenMeng-NOAA A quick update... @aerorahul and I are still working on the patch (ironing out a detail) but I have removed manage_externals in my fork copy of the WenMeng-NOAA/UPP@post_gfsv16_wcoss2...KateFriedman-NOAA:post_gfsv16_wcoss2 I hacked things on the workflow side a tad to get the right tag version of the GTG checked out (the detail to iron out) and tested the build with GTG changes in place. Please see the following log and confirm that my build with GTG worked: /lfs/h2/emc/global/noscrub/Kate.Friedman/git/feature-ops-wcoss2/sorc/logs/build_ncep_post_test6_gtg.log I will submit a PR into your fork |
Your build log looks good to me. Thanks! |
Thanks @WenMeng-NOAA ! I just reran all of my gdaspost* jobs and the gfspost023 job in my Dogwood test. Please see the following logs and output and let me know if it looks ok: gdaspost*: gfspost023 (just redid the f213 files): FYI, I will be running a cleaner fresh test next week for everyone to examine, so I'll send logs/output to look at again when I do that test. Thanks! |
@KateFriedman-NOAA The runtime logs and UPP datasets look good to me. Thanks! |
|
@WenMeng-NOAA I submitted PR #5 from my fork to your fork with updates for converting the GTG checkout from manage_externals to git submodules. I wasn't able to add reviewers or assignees or labels. |
|
@KateFriedman-NOAA Your PR looks good to me. Would you like to have someone else review it? |
Nope, since it works for you and both @aerorahul and myself tested it, it should be good to submit my PR into your branch. Thanks! |
* Retire manage_externals for GTG checkout - remove manage_externals copy from repo - remove associated Externals.cfg file - replacing GTG checkout with submodules Refs: NOAA-EMC#363 * Add UPP_GTG submodule - created .gitmodules file via git submodule add command: git submodule add https://github.com/NCAR/UPP_GTG sorc/post_gtg.fd - new post_gtg.fd subfolder added (empty before submodule checkout) Refs: NOAA-EMC#363 * Updating submodule pointer of GTG to ncep_post_gtg.v2.0.4
|
@KateFriedman-NOAA and @aerorahul It looks the branch is working for inline and offline post in Kate's GFS V16 end-to-end testing on WCOSS2. Do you have more comments on it? Thanks! |
|
@WenMeng-NOAA If you could look at the post output in my free-forecast test that ran overnight and then, assuming it looks good, I'm good for this to be submitted. See this g-w issue comment with output details: NOAA-EMC/global-workflow#399 (comment) . Thanks! |
The changes are added for porting UPP for gfs v16 on wcoss2.