Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hive integration: row-wise input/output format for vineyard tables #1551

Merged
merged 8 commits into from
Sep 5, 2023
Merged

Hive integration: row-wise input/output format for vineyard tables #1551

merged 8 commits into from
Sep 5, 2023

Conversation

vegetableysm
Copy link
Collaborator

@vegetableysm vegetableysm commented Sep 4, 2023 •
edited
Loading

What do these changes do?

  • Support create vineyard table with HQL.
  • Support query data from vineyard table with HQL
  • Implement vineyard file system(Integrate jimfs) to support simulate file operation.
  • Support static partition/dynamic partition.

Related issue number

Fixes #1420

@netlify
Copy link

netlify bot commented Sep 4, 2023 •
edited
Loading

✅ Deploy Preview for v6d ready!

Name Link
🔨 Latest commit ce0cf73
🔍 Latest deploy log https://app.netlify.com/sites/v6d/deploys/64f69fbd8d93b70008f2aa0f
😎 Deploy Preview https://deploy-preview-1551--v6d.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@vegetableysm vegetableysm force-pushed the hive-integration branch 3 times, most recently from 9814e45 to ad917ff Compare September 4, 2023 11:52
vegetableysm and others added 3 commits September 4, 2023 19:52
@vegetableysm @sighingnow
Hive integration.
4f7618f 
Co-authored-by: Tao He <linzhu.ht@alibaba-inc.com>
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@vegetableysm
Clean code.
0e02cd5 
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@vegetableysm
Fix a reading table bug on spark.
e561064 
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@sighingnow sighingnow changed the title Hive integration. Hive integration: row-wise input/output format for vineyard tables Sep 4, 2023
sighingnow
sighingnow approved these changes Sep 4, 2023
View reviewed changes
java/hive/docker/README.md Outdated
@@ -0,0 +1,130 @@
### Introduction
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this readme.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

vegetableysm and others added 5 commits September 4, 2023 21:52
@vegetableysm
Bind table name with table object id.
c42bdcb 
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@vegetableysm
Delete old table objects after merge files.
152ee8b 
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@vegetableysm
Remove readme.
734b2bd 
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@sighingnow
Unify RowWritable and RecordWrapperWritable
5a36271 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
@sighingnow
Lint & format
ce0cf73 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
@sighingnow sighingnow merged commit 0ad13c5 into v6d-io:main Sep 5, 2023
@sighingnow sighingnow deleted the hive-integration branch September 5, 2023 03:30
@vegetableysm vegetableysm mentioned this pull request Sep 5, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sighingnow sighingnow sighingnow approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Hive integration: vineyard serve as the storage backend for hive
2 participants
@vegetableysm @sighingnow