Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUGFIX] Add a parameter to SparkDFDatasource to reuse the Spark Context #2733

Merged
merged 5 commits into from
Apr 30, 2021
Merged

[BUGFIX] Add a parameter to SparkDFDatasource to reuse the Spark Context #2733

merged 5 commits into from
Apr 30, 2021

Conversation

j-f-r
Copy link
Contributor

@j-f-r j-f-r commented Apr 27, 2021 •
edited
Loading

Changes proposed in this pull request:

  • Add a force_reuse_spark_context to SparkDFDatasource, which gets passed to get_or_create_spark_application so that it does not stop the current context, but reuse it.

Issues:
#2732

@jfr-merca
Add force_reuse_spark_context as a parameter to get_or_create_spark_a…
d939971 
…pplication and pass the new parameters from sparkdfdatasource
@j-f-r
Copy link
Contributor Author

j-f-r commented Apr 27, 2021

@cla-bot check

1 similar comment
@j-f-r
Copy link
Contributor Author

j-f-r commented Apr 27, 2021

@cla-bot check

eugmandel
eugmandel approved these changes Apr 30, 2021
View reviewed changes
Copy link
Contributor

@eugmandel eugmandel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@j-f-r Thank you! Will merge.

@eugmandel eugmandel merged commit a553fa8 into great-expectations:develop Apr 30, 2021
@pasmavie pasmavie mentioned this pull request Jun 29, 2021
Closed
6 tasks
@pasmavie pasmavie mentioned this pull request Jul 28, 2021
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eugmandel eugmandel eugmandel approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@j-f-r @eugmandel @jfr-merca