Allow handling of tiled data in ExtData2G

[bena-nasa]

This topic keeps coming up so it should be tackled. There are several bits of forcing data on the tile side that can only be done now through MAPL_ReadForcing (unlike the gridded forcings like SST's which now have an option to use ExtData, assuming the data is there of course, but that's another issue, outside of MAPL...).

So ExtData should be extended if possible so that MAPL_ReadForcing can be retired someday. I'll investigate.

[bena-nasa]

I tried a simple experiment first, to see if I can even get a variable on a tile grid to the cap (so it can go to ExtData). I tried adding a new import to catchment (VISDF_EXTDATA was the name), then let that bubble above land by replacing this terminate import in the Surface component:

call MAPL_TerminateImport    ( GC, CHILD = LAND, RC=STATUS  )

with

call MAPL_TerminateImport    ( GC, ["VISDF_EXTDATA"], [land],    RC=STATUS  )

However, I got this traceback which is here on Discover:
/discover/nobackup/bmauer/experiments/test_tileextdata/run.log

Note that this is with MAPL's develop branch as of the time this comment was made.

[amolod]

hi ben, forgive any ignorance here, but isn't the original line terminating all the imports to LAND? and the replacement only one? so it will bubble all the imports up? or does the square bracket tell it "everyone but" this one from land?

[bena-nasa]

The original line is terminating all imports from land.
The new line, says terminate all imports except VISDF_EXTDATA from land.
That is the MAPL_TerminateImportAllBut overload of MAPL_TerminateImport:
https://github.com/GEOS-ESM/MAPL/blob/main/generic/MAPL_Generic.F90#L294

The square bracket is an automatic array constructor in Fortran, just an alternative to the (/ ... /) way of constructing an array.

https://riptutorial.com/fortran/example/6858/array-constructors

[atrayano]

@bena-nasa I just committed a fix (PR #1660) which should fix this issue. It will be great if you could try to run you test case with it. Thanks!

[bena-nasa]

I did confirm that variables from catchment etc... can bubble up to ExtData.

@tclune
I think I should just implement something to make everyone happy.

I do not see the new geometry class as necessary or really going to help us much in the short term. The fact is, no regridding is necessary or even possible, now or in the near term or long term for the stuff ReadForcing does that we need to retire. And for the foreseeable future those components that under surface will be what they are (this arbitrarily distributed grid that serves no purpose other than the component needed a grid and we needed a place to attach the tile mask...) in MAP2 and it will be far too long for MAPL3 to be functional in the real model.

It is a simple matter of reading the variable on root, redistribute according the attached the mask since that defines how the tiles are distributed for the grid. That's it, nothing more. I just have to find a way to make this not be total spaghetti code.

[tclune]

Fine. Will you use the ESMF "background grid" for the distribution, or are you doing something custom with regard to the "mask"?

[bena-nasa]

@tclune

It is not for ExtData to decide how to distribute these tiles. That is already known by the time ExtData gets the field, it's been created however the component decided to distribute. There is this "tile mask" that is attached to this pseudo grid that the tiled variables are created on. When we read the restarts, that defines how we scatter the tiles (via MAPL's ArrayScatter which can take a mask) to the appropriate MPI rank. Absolutely no different here in what ExtData has to do.

Now of course in practice we distribute them via the ESMF background grid when constructing the MAPL's location stream.

I'll see how if there's a way I can do this somewhat "clean".

[tclune]

Right - I did not keep in mind that you'll still be using tile files instead of next-gen NetCDF