Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat][Spark] Add examples to show how to load/dump data from/to GraphAr for Nebula #244

Merged
merged 11 commits into from
Oct 9, 2023

Conversation

liuxiaocs7
Copy link
Member

@liuxiaocs7 liuxiaocs7 commented Oct 4, 2023

Proposed changes

Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue.

close #34
example in gar-test: apache/incubator-graphar-testing#4

Types of changes

What types of changes does your code introduce to GraphAr?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

@liuxiaocs7 liuxiaocs7 changed the title feat: support NebulaGraph Integration feat: Support NebulaGraph Integration Oct 4, 2023
@liuxiaocs7 liuxiaocs7 changed the title feat: Support NebulaGraph Integration [Feat][Spark]: Support NebulaGraph Integration Oct 4, 2023
@wey-gu
Copy link

wey-gu commented Oct 4, 2023

Dear Great Creators of GraphAr Project!
This is wey from the NebulaGraph community, we are so very excited to have a chance to enable GraphAr - NebulaGraph connection in Hacktoberfest, I wonder if it's possible to request your support on:

  • Adding hactoberfest-accepted label to this PR?
  • Help review and hopfully collaborate to merge this change in October?

This will enable the contributor to be part of the Hactoberfest project and it means a lot!

Thanks and appreciate your help!

Best Regards, Wey

@lixueclaire
Copy link
Contributor

Dear Great Creators of GraphAr Project! This is wey from the NebulaGraph community, we are so very excited to have a chance to enable GraphAr - NebulaGraph connection in Hacktoberfest, I wonder if it's possible to request your support on:

  • Adding hactoberfest-accepted label to this PR?
  • Help review and hopfully collaborate to merge this change in October?

This will enable the contributor to be part of the Hactoberfest project and it means a lot!

Thanks and appreciate your help!

Best Regards, Wey

Dear Wey,

Thank you for reaching out to us and expressing your excitement about enabling the GraphAr - NebulaGraph connection in Hacktoberfest. We appreciate your interest in our project and we are happy to support you.
We would be delighted to review your PR and collaborate with you to merge the changes in October.
If you have any further questions or need assistance, please feel free to reach out to us.

Best regards,
Xue

@acezen
Copy link
Contributor

acezen commented Oct 7, 2023

The hactoberfest-accepted label has been added to the PR.

Thanks for the EXCITING work to enabling the feature we expect. As @lixueclaire said, we are delighted to review the PR and collaborate with the Hactoberfest project.

@liuxiaocs7
Copy link
Member Author

The hactoberfest-accepted label has been added to the PR.

Thanks for the EXCITING work to enabling the feature we expect. As @lixueclaire said, we are delighted to review the PR and collaborate with the Hactoberfest project.

Thanks for your help, but I confirmed that the label should be hacktoberfest-accepted rather than hactoberfest-accepted. Could you help modify it in your free time?

CC: @wey-gu

@wey-gu
Copy link

wey-gu commented Oct 7, 2023

Dear Great Creators of GraphAr Project! This is wey from the NebulaGraph community, we are so very excited to have a chance to enable GraphAr - NebulaGraph connection in Hacktoberfest, I wonder if it's possible to request your support on:

  • Adding hactoberfest-accepted label to this PR?
  • Help review and hopfully collaborate to merge this change in October?

This will enable the contributor to be part of the Hactoberfest project and it means a lot!
Thanks and appreciate your help!
Best Regards, Wey

Dear Wey,

Thank you for reaching out to us and expressing your excitement about enabling the GraphAr - NebulaGraph connection in Hacktoberfest. We appreciate your interest in our project and we are happy to support you. We would be delighted to review your PR and collaborate with you to merge the changes in October. If you have any further questions or need assistance, please feel free to reach out to us.

Best regards, Xue

Dear Xue,

Wow, thanks so very much for the prompt help, care and powerful support!

@wey-gu
Copy link

wey-gu commented Oct 7, 2023

The hactoberfest-accepted label has been added to the PR.

Thanks for the EXCITING work to enabling the feature we expect. As @lixueclaire said, we are delighted to review the PR and collaborate with the Hactoberfest project.

Dear @acezen

Wow, many thanks for the super-fast and warm support!!!

@wey-gu
Copy link

wey-gu commented Oct 7, 2023

The hactoberfest-accepted label has been added to the PR.
Thanks for the EXCITING work to enabling the feature we expect. As @lixueclaire said, we are delighted to review the PR and collaborate with the Hactoberfest project.

Thanks for your help, but I confirmed that the label should be hacktoberfest-accepted rather than hactoberfest-accepted. Could you help modify it in your free time?

CC: @wey-gu

OMG, I'm so very sorry/ashamed to provide a typo on the label... let's wait @acezen for another go 🙇

spark/README.md Show resolved Hide resolved
@liuxiaocs7 liuxiaocs7 changed the title [Feat][Spark]: Support NebulaGraph Integration [Feat][Spark] Support NebulaGraph Integration Oct 9, 2023
@liuxiaocs7 liuxiaocs7 requested a review from acezen October 9, 2023 07:16
Copy link
Contributor

@acezen acezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please rebase to main.
Thanks for the job.

@acezen
Copy link
Contributor

acezen commented Oct 9, 2023

hi, @liuxiaocs7, I not sure the PR can close the issue vesoft-inc/nebula-spark-connector#76 because it's just an example to exchange a specific data(basketballplayergraph graph) between nebula and GraphAr, the example can't use for other data.
Do you think this can implement as a general tool that fits for every graph?

BTW, @wey-gu @lixueclaire, do you have some suggestions for that?

@liuxiaocs7
Copy link
Member Author

hi, @liuxiaocs7, I not sure the PR can close the issue vesoft-inc/nebula-spark-connector#76 because it's just an example to exchange a specific data(basketballplayergraph graph) between nebula and GraphAr, the example can't use for other data. Do you think this can implement as a general tool that fits for every graph?

BTW, @wey-gu @lixueclaire, do you have some suggestions for that?

Sure, thank you for your suggestion, I will remove close it and consider how to implement a more general tool in other PRs!

lixueclaire

This comment was marked as duplicate.

Copy link
Contributor

@lixueclaire lixueclaire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~ Thanks for your contribution. It would be even better if you could update the documentation or create an issue about it.

@lixueclaire
Copy link
Contributor

hi, @liuxiaocs7, I not sure the PR can close the issue vesoft-inc/nebula-spark-connector#76 because it's just an example to exchange a specific data(basketballplayergraph graph) between nebula and GraphAr, the example can't use for other data. Do you think this can implement as a general tool that fits for every graph?
BTW, @wey-gu @lixueclaire, do you have some suggestions for that?

Sure, thank you for your suggestion, I will remove close it and consider how to implement a more general tool in other PRs!

Sounds good! Maybe you can refer to #243 for this.

@acezen
Copy link
Contributor

acezen commented Oct 9, 2023

@liuxiaocs7 the example data apache/incubator-graphar-testing#4 has been merge, you can update the testing submodule to latest commit.

@acezen acezen changed the title [Feat][Spark] Support NebulaGraph Integration [Feat][Spark] Add example Nebula2GraphAr and GraphAr2Nebula to show how to exchange data between Nebula and GraphAr Oct 9, 2023
@acezen acezen changed the title [Feat][Spark] Add example Nebula2GraphAr and GraphAr2Nebula to show how to exchange data between Nebula and GraphAr [Feat][Spark] Add examples to show how to exchange data between Nebula and GraphAr Oct 9, 2023
@liuxiaocs7
Copy link
Member Author

LGTM~ Thanks for your contribution. It would be even better if you could update the documentation or create an issue about it.

Issue created here: #248, i'll finish ASAP.

@acezen acezen changed the title [Feat][Spark] Add examples to show how to exchange data between Nebula and GraphAr [Feat][Spark] Add examples to show how to load/dump data from/to GraphAr for Nebula Oct 9, 2023
@acezen acezen merged commit 78f9226 into apache:main Oct 9, 2023
3 checks passed
@liuxiaocs7
Copy link
Member Author

@liuxiaocs7 the example data GraphScope/gar-test#4 has been merge, you can update the testing submodule to latest commit.

Hi, @acezen, should i follow this to update gar-test submodule: https://stackoverflow.com/questions/5828324/update-git-submodule-to-latest-commit-on-origin

@wey-gu
Copy link

wey-gu commented Oct 10, 2023

Amazing OpenSource collaboration! I would like to express my utmost respect, appreciation, and admiration!

@wey-gu
Copy link

wey-gu commented Oct 10, 2023

@liuxiaocs7 the example data GraphScope/gar-test#4 has been merge, you can update the testing submodule to latest commit.

Hi, @acezen, should i follow this to update gar-test submodule: https://stackoverflow.com/questions/5828324/update-git-submodule-to-latest-commit-on-origin

@liuxiaocs7 think of submodule a way to let the parent module track submoudle as a folder only on its git hash.

With that in mind, when upating a submodle, we could just do the needed pull(to ensure its hash is the one from the remote) and do git add as a commit on the parent module.

What git submodule update was just to pull submodule based on its tracked git hash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feat] Integrate NebulaGraph spark connector as input data source for GraphAr spark tool
4 participants