New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use with AWS Glue #77

Open

WarFox opened this issue Jun 3, 2020 · 1 comment

WarFox commented Jun 3, 2020

Is it recommended to use sparkling with AWS Glue?

AWS Glue is a serverless stack on top of Spark provided by EMR. It has some additional features to Spark, but spark context is available as normal.

Has anyone tried integrating with Glue?

samrose commented Sep 6, 2020

@WarFox I feel like you might need to use the AWS Java SDK like https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/glue/AWSGlueClient.html to communicate with AWS Glue.

But I'd be interested to know more about what you have in mind with "use sparkling with AWS Glue" do you have an example?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment