Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DataAPI.jl metadata API #337

Open
bkamins opened this issue Sep 20, 2022 · 3 comments
Open

Support DataAPI.jl metadata API #337

bkamins opened this issue Sep 20, 2022 · 3 comments

Comments

@bkamins
Copy link
Contributor

bkamins commented Sep 20, 2022

Hi @quinnj - could you please add to the release plan of Arrow.jl support for JuliaData/DataAPI.jl#48 for the created Arrow tables.
Only read methods need to be implemented for Arrow tables:

  • DataAPI.metadata
  • DataAPI.metadatakeys
  • DataAPI.colmetadata
  • DataAPI.colmetadatakeys
@bkamins
Copy link
Contributor Author

bkamins commented Aug 3, 2023

@quinnj - we should get back to it as the issue got forgotten (at least by me).
In particular we should also add an automatic writing of metadata from objects that support metadata (instead of using metadata and colmetadata keywords in writer).

Do you have any comments on this (or I should just propose a PR)?

@quinnj
Copy link
Member

quinnj commented Aug 3, 2023

Ah yes, just make a PR; I think it should be pretty straightforward.

@bkamins
Copy link
Contributor Author

bkamins commented Aug 13, 2023

In #481 I have implemented it for Arrow.Table.

My follow-up questions are:

  • I was not sure how Arrow.Stream should be handled for metadata reading.
  • I was not sure for Arrow.write and Arrow.Writer and Arrow.append how to handle writing metadata. In particular, if one writes to an already existing Arrow file what should be allowed (i.e. if adding/updating metadata should be supported).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants