Add capability to read IAU increments in cubed sphere format #274
Conversation
There was a problem hiding this comment.
Thank you for designing this very useful functionality for the IAU on the native cubed-sphere grid, which promises to make a lot of the DA process smoother and can leverages some of JEDI's new features. There are some concerns I have about the coding that could be improved by leveraging existing functionality in FMS or by following similar codes already within FV3.
model/fv_arrays.F90
Outdated
| @@ -516,7 +516,7 @@ module fv_arrays_mod | |||
| !----------------------------------------------------------------------------------------------- | |||
|
|
|||
| logical :: reset_eta = .false. | |||
| logical :: ignore_rst_cksum = .false. !< enfore (.false.) or override (.true.) data integrity restart checksums | |||
There was a problem hiding this comment.
I would beware changing this default unless absolutely necessary. This restart checksumming makes sure that the data isn't corrupted; it should be simple to set this namelist item when restarting from modified restart files.
| write(6,*) 'diff is ',abs(a - b),compvals | ||
|
|
||
| end function | ||
| logical function is_tracer(varname,tracer_idx) |
There was a problem hiding this comment.
I think it would be easier to use get_tracer_index() to see if a given field in the input data is defined as a tracer before reading it in, rather than using a list of hard-coded tracer names that would be difficult to extend to other model use cases that might have a different set of tracers or tracer names.
| @@ -0,0 +1,551 @@ | |||
| #define NC_ERR_STOP(status) \ | |||
There was a problem hiding this comment.
I believe most if not all of the I/O code in this file can be replaced with standard FMS calls. This could be done similar to that in get_nggps_ic which also reads in native-grid data.
There was a problem hiding this comment.
@lharris4 If I recall correctly, FMS calls can only be used to read FMS-formatted restart files. Our aim with JEDI in the GDASApp and Global Workflow is to read increments in cubed-sphere history files.
There was a problem hiding this comment.
Lucas is correct that FMS can read FMS and non-FMS created NetCDF files on the native grid or for any grid. Otherwise, our data_override capability to read in standardized forcing field datasets would not work.
For non-native grid NetCDF files and to keep each MPI-rank from reading in the full dataset, you can use:
- use fms2_io::read_data to ingest the lat-lon data describing the fields in the file
- determine indices that define a bounding box for the MPI-rank (with margins for interpolation)
- use fms2_io::read_data to ingest only the portion of the field(s) within the bounding box indices
- interpolate data to the native grid using a method defined in horiz_interp
For a native grid NetCDF defined on the cubed-sphere, one can use the fms2_io::read_data to read in the full field or if you know your particular indices you can read in a subset of the data. I believe you can easily use the restart functionality as well without any issues.
There was a problem hiding this comment.
@bensonr Thanks for the info, I will look into changes using this functionality
Thanks for take a look so quickly. I have been working with Cory Martin at EMC to test this out as part of a JEDI-based workflow, and it is still definitely in draft format. That said, this is exactly the kind of feedback we wanted to get before going too far down the rabbbit hole. I'll take a look at each of your suggestions and make the appropriate changes. |
|
Hi, Mark. Glad to help.
Lucas
…On Wed, Jun 7, 2023 at 3:14 PM Mark Potts ***@***.***> wrote:
Thank you for designing this very useful functionality for the IAU on the
native cubed-sphere grid, which promises to make a lot of the DA process
smoother and can leverages some of JEDI's new features. There are some
concerns I have about the coding that could be improved by leveraging
existing functionality in FMS or by following similar codes already within
FV3.
Thanks for take a look so quickly. I have been working with Cory Martin at
EMC to test this out as part of a JEDI-based workflow, and it is still
definitely in draft format. That said, this is exactly the kind of feedback
we wanted to get before going too far down the rabbbit hole. I'll take a
look at each of your suggestions and make the appropriate changes.
—
Reply to this email directly, view it on GitHub
<#274 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMUQRVCTUGLZV7I65WWDVCLXKDHHRANCNFSM6AAAAAAY6AGHE4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Hi, @mark-a-potts . Is this still a WIP or is it ready for review? |
|
It is still in progress. I updated it recently so that EMC can do some testing with a current version of the UFS-WM. Thanks for checking! |
|
Hi, Mark. Thank you for your update. Please let us know if you have any
questions or need help.
Lucas
…On Mon, Aug 7, 2023 at 3:27 PM Mark Potts ***@***.***> wrote:
It is still in progress. I updated it recently so that EMC can do some
testing with a current version of the UFS-WM. Thanks for checking!
—
Reply to this email directly, view it on GitHub
<#274 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMUQRVBLLKOCAHTZ5THYL3LXUE6RPANCNFSM6AAAAAAY6AGHE4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
@mark-a-potts - is this PR still being actively developed? |
mark-a-potts
force-pushed
the
feature/netcdf_read
branch
from
bb57105 to
b4d6609
Compare
|
I think EMC is about ready to start using this capability actively, so I just updated this again and have tested it with some accompanying changes to the fv3atm repo. |
laurenchilutti
requested review from
XiaqiongZhou-NOAA,
bensonr,
lharris4 and
DavidNew-NOAA
|
Hi @laurenchilutti I'm working on an updated version of this PR that adds the ability to read cubed sphere increments both inside and outside the IAU framework. I'm hoping to submit a PR today or Monday once testing is complete. |
|
@DavidNew-NOAA Thank you for the update! When that new PR comes in would you like me to close this one? |
|
@laurenchilutti Yes, that's you could probably close it. |
|
@laurenchilutti #342 and FV3 PR #837 are ready for review. cc @XiaqiongZhou-NOAA @bensonr @lharris4 @mark-a-potts |
|
Closing, as PR #342 replaces this PR. |
This PR adds the capability to read in NetCDF-based cubed-sphere increments generated by JEDI. It includes a new (optional) namelist option to specify that non-gaussian formatted increment files are to be read.
Fixes # (issue)
How Has This Been Tested?
Two unit tests have been added to the code and can be run via ctest after the build.
Checklist:
Please check all whether they apply or not