Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add coral-beam for converting Calcite logical plan into Apache Beam API Java code #21

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

khaitranq
Copy link

End-to-end implementation and tests for translating from Pig Latin scripts into Apache Beam Java code

@wmoustafa
Copy link
Contributor

Could you please remove Pig references and unit tests since Currently Coral supports reading from HiveQL only?

@Rep1AI
Copy link

Rep1AI commented Jan 1, 2022

Any update on Apache Beam integration? it will be very useful to have beam as well.

cc: @wmoustafa @khaitranq

@wmoustafa
Copy link
Contributor

wmoustafa commented Jan 5, 2022

Any update on Apache Beam integration? it will be very useful to have beam as well.

cc: @wmoustafa @khaitranq

Thanks @RohitR1 for reaching out! Do you have a use case for this? Before merging this PR, we need to validate that the two sides of the conversion in this PR (from Pig and to Beam) commute with other sides already implemented in Coral (e.g, from Hive QL and to Trino SQL), so one can convert Hive QL (or Spark SQL) to Beam for example. It might also be best to split the PRs to two halves to ensure there is no coupling between the two conversion sides. I think this is the major remaining step here. We are not planning to pick up this now, but please feel free to make suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants