-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exclude (or flag) coordinates #63
Comments
It seems that there are variables that are listed as coordinates that still make it into the catalogue, and others that aren't classified as coordinates but we would also want to exclude. e.g.
In this catalogue:
that datafile contains the following variables:
Opening with
and not in
So not sure why It might be worth using cf_xarray to identify variables
to exclude bounds variables
or even extract information to augment grid identification (#112)
|
Thanks for reporting @aidanheerdegen. At the moment, only 1D coordinates are excluded since files are opened with We can obviously easily switch these flags to Thinking more about this, I'm not sure we want to exclude coordinates, since I think it is useful to be able to search on these. In ACCESS-OM2 output, for example, grid information is included in a separate
I don't think I understand how you're suggesting we flag these in a robust way. In fact, it's not even clear to me what's in this list. I think you're referring to properties of the grid (e.g. |
It just occurred to me that flagging coordinates is going to be difficult with the way Intake-ESM is currently set up to handle multi-variable assets (files). Multi-variable assets are included as a single row in an Intake-ESM datastore, with a column containing a list of the variables available in that file - see here. There's thus no easy way include per-variable attributes in the table. |
Ok, thanks for the informative response @dougiesquire. I agree, I think I was being a bit ... cavalier ... with my wish to filter out a large number of variables as "griddy". There are indeed good use cases for many of these. For my use case I think we'll do what the COSIMA Explorer does, use some heuristics to "hide" variables that are potentially of little use to expose at a high level, but allow for them to be shown if a user wishes. Regarding extracting grid information that actually has the potential to be a useful way to exclude variables: if they exist in a higher-quality grid file, then use that and "hide" the same variables that are present in the data file. I am thinking specifically of un-masked grid coordinates. That may require more contemplation, so I'll just leave that thought bubble there and back away slowly ... |
It would be good to make it possible to only show diagnostic variables in the catalogue, by excluding all coordinates.
This could be achieved by not including coordinate variables in the catalogue at all: they are accessible when variables are loaded that contain those coordinates.
Another option would be to include a column that with a flag that indicates if the variable is a coordinate, or not. In that way they could be filtered out by the user (I assume).
The text was updated successfully, but these errors were encountered: