Couln't find metadata for variable

[lgloege]

I am trying to open custom model diagnostics with only *.data and *.meta files. I do not have available_diagnostics.log since I did not run this model.

Code
import xmitgcm data_dir = '/local/data/artemis/workspace/gloege/data/ECCO_DARWIN/ECCO_Darwin/PISVEL' grid_dir = '/local/data/artemis/workspace/gloege/data/ECCO_DARWIN/ECCO_Darwin/grid/LLC_270' ds = xmitgcm.open_mdsdataset(data_dir=data_dir, grid_dir=grid_dir, geometry='llc')

Errors
UserWarning: Couldn't find available_diagnostics.log in /local/data/artemis/workspace/gloege/data/ECCO_DARWIN/ECCO_Darwin/PISVEL or /local/data/artemis/workspace/gloege/data/ECCO_DARWIN/ECCO_Darwin/grid/LLC_270. Using default version. "in %s or %s. Using default version." % (data_dir, grid_dir))

and

"Couln't find metadata for variable PISVEL and ignore_unknown_vars==False."

is there a way to specify the diagnostic name without available_diagnostics.log?

Thanks!

[timothyas]

Hey @lgloege! I'm hoping this can be solved by simply renaming the file basename meta/data files from PISVEL to PsiVEL, so that it is recognized as one of the variables in the default available_diagnostics. You could test this out by

$ cp PISVEL.<iternum>.meta PsiVEL.<iternum>.meta
$ cp PISVEL.<iternum>.data PsiVEL.<iternum>.data

then providing the prefix='PsiVEL' and iters=<iternum> to open_mdsdataset, where <iternum> is some iteration number at which you have data.

For this to work, I'm assuming that (1) you're looking for the variable PsiVEL (2) the meta file does not have the fldList field which defines the variable(s) contained in the .data file, and (3) there is one meta/data pair per variable, per time step where the file basename (or "prefix") is mislabeled as PISVEL instead of PsiVEL.

If that doesn't help, let me know if any of those assumptions are wrong. And in that case could you show the contents of one of the *.meta files? That might be helpful. Thanks!

[rabernat]

@timothyas - I don't think PISVEL is the same as PSIVEL. These new variables would need to be manually added to https://github.com/MITgcm/xmitgcm/blob/master/xmitgcm/variables.py.

[timothyas]

Oops, I didn’t know about that one! Thanks @rabernat

[cspencerjones]

This is also an issue for opening files produced by the adjoint code (ADJtheta etc.). It would be great to have some alternate way of telling xmitgcm how to open these files.

[timothyas]

@cspencerjones I just want to mention that some adjoint variables can be printed through the diagnostics package, see here. Then like all other diagnostics variables, these can be read easily with xmitgcm with the corresponding available_diagnostics.log. I've been through the ringer on this and if you're interested in adding your own adjoint variables I can help out (probably best to discuss that one via email or perhaps the MITgcm support list or an MITgcm issue).

The problem still remains, though, for the adxx_<> fields printed by the control package, and for any ADJ field that haven't yet been added and one simply wants to read easily. It's also an issue with the shelfice package, which has a number of variables printed out but not through diagnostics. So yep, I agree this would be nice.

[rabernat]

There's a pretty easy path to fixing this.

First, allow the user to specify extra_variables={} in open_mdsdataset to supplement what's in https://github.com/MITgcm/xmitgcm/blob/master/xmitgcm/variables.py

Then merge this with the existing metadata dictionary here: https://github.com/MITgcm/xmitgcm/blob/master/xmitgcm/mds_store.py#L839

[cspencerjones]

OK - I am working on it! Thanks @rabernat and @timothyas