Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DataAPI metadata as GeoInterface name agreements #77

Open
rafaqz opened this issue Nov 16, 2022 · 9 comments
Open

Support DataAPI metadata as GeoInterface name agreements #77

rafaqz opened this issue Nov 16, 2022 · 9 comments

Comments

@rafaqz
Copy link
Member

rafaqz commented Nov 16, 2022

It's very cool that storing geometry column name metadata like this works:

evetion/GeoDataFrames.jl#47

I just realised it should also work in Rasters.jl, and probably lots of other places. Maybe we can we state that "crs" and "geometrycolumns" are our blessed metadata keys somewhere in the GeoInterface.jl docs, then we can use them everywhere and GeoDataFrames.jl will already be compliant.

@evetion
Copy link
Member

evetion commented Nov 17, 2022

Yeah, I've thought about this when I implemented it in GeoDataFrames, but it's actually a bit complex.

  • With metadata, there's now two sources of crs. What if they don't agree? Ideally, the Geointerface API just takes it from the metadata, as defined in the custom geometry package.
  • What should the key and values be named and look like? The best example is actually GeoParquet, but that uses its own namespaced keys, and the rest under column metadata.

@rafaqz
Copy link
Member Author

rafaqz commented Nov 17, 2022

Yeah namespacing is an issue. @bkamins is there a standard way of namespacing metadata for DataAPI? We could prefix it with "GeoInterface_" or something?

@bkamins
Copy link

bkamins commented Nov 17, 2022

We thought about it, but decided we can add a standard later after some experience with it.

I understand this metadata will mostly not be accessed by the user directly, but rather programmatically via API. Is this right?
Then the length of the key is not important.

Apache Arrow has the following standard: https://arrow.apache.org/docs/format/Columnar.html#extension-types.
I would propose to use the same style, so it would be:

GEOINTERFACE:name

@rafaqz
Copy link
Member Author

rafaqz commented Nov 17, 2022

Yes its mostly for programatic use.

"GEOINTERFACE:crs" and "GEOINTERFACE:geometrycolumn" seem reasonable to me.

@asinghvi17
Copy link
Member

I started an implementation of this in evetion/GeoDataFrames.jl#70. That being said, it does seem like GeoInterface would need to take on Tables.jl and DataAPI.jl as dependencies - is this something that we can do here? Package extensions would be pretty annoying given the need to support older versions of Julia, even if we do use an include trick.

@asinghvi17
Copy link
Member

The next step here is to have GeoInterface natively figure this stuff out - a direct dependency on DataAPI shouldn't be too bad, it's 500 lines of function stubs and documentation.

@evetion
Copy link
Member

evetion commented Jul 6, 2024

A package extension can work, 1.10 will be LTS very soon from what I gathered.

@asinghvi17
Copy link
Member

How would that work, though? Just defining GI.crs(::Nothing, item) = GI.crs(nothing, DataAPI.hasmetadatasupport(item), item) and then defining various methods for that last function in the extension? I can't see another way to do this cleanly without resorting to piracy.

@evetion
Copy link
Member

evetion commented Jul 7, 2024

I was merely answering your

Package extensions would be pretty annoying given the need to support older versions of Julia, even if we do use an include trick.

I haven't thought out the design just yet, but it would be good to document our "reserved" namespace and keys already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

4 participants