Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add metadata to UDFs in catalog #606

Merged
merged 16 commits into from
Mar 18, 2023

Conversation

suryatejreddy
Copy link
Collaborator

@suryatejreddy suryatejreddy commented Mar 13, 2023

This PR adds support for storing additional metadata with a UDF. It can be used in the following ways:

  • Used to store attributes of the UDF (cache=True, batch=True) in the catalog.
  • Used during the definition of UDFS for more advanced use cases (HuggingFace).

Design

  • Create a new table in the catalog with the columns key, value, and udf_id.
  • This PR adds support only for metadata defined using the CREATE statement.

New Syntax

 CREATE UDF FastRCNN
     INPUT  (Frame_Array NDARRAY UINT8(3, 256, 256))
     OUTPUT (Labels NDARRAY STR(10), Bbox NDARRAY UINT8(10, 4))
     TYPE  Classification
     IMPL  'data/fastrcnn.py'
     "KEY" "VALUE";

Testing

Adds test cases.

@suryatejreddy suryatejreddy changed the title feat: Add metadata to UDFs feat: Add metadata to UDFs in catalog Mar 13, 2023
super().__init__(UdfMetadataCatalog)

def insert_entries(self, entries: List[UdfMetadataCatalogEntry]):
for entry in entries:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to catch any catalog error here

@gaurav274 gaurav274 marked this pull request as ready for review March 18, 2023 19:45
@gaurav274 gaurav274 merged commit 8d17043 into georgia-tech-db:master Mar 18, 2023
@jarulraj jarulraj mentioned this pull request Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants