Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-4180] [Core] Prevent creation of multiple active SparkContexts #3121

Closed
wants to merge 18 commits into from
Closed

[SPARK-4180] [Core] Prevent creation of multiple active SparkContexts #3121

wants to merge 18 commits into from

Commits on Nov 5, 2014

  1. [SPARK-4180] Prevent creations of multiple active SparkContexts.

    @JoshRosen
    JoshRosen committed Nov 5, 2014
    Configuration menu
    Copy the full SHA
    afaa7e3 View commit details
    Browse the repository at this point in the history

  2. Document "one SparkContext per JVM" limitation.

    @JoshRosen
    JoshRosen committed Nov 5, 2014
    Configuration menu
    Copy the full SHA
    918e878 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2014

  1. Log long form of creation site to aid debugging.

    @JoshRosen
    JoshRosen committed Nov 6, 2014
    Configuration menu
    Copy the full SHA
    c4d35a2 View commit details
    Browse the repository at this point in the history

  2. StreamingContext.stop() should stop SparkContext even if StreamingCon… 

    …text has not been started yet.
    @JoshRosen
    JoshRosen committed Nov 6, 2014
    Configuration menu
    Copy the full SHA
    d0437eb View commit details
    Browse the repository at this point in the history

  3. Add / improve SparkContext cleanup in streaming BasicOperationsSuite

    @JoshRosen
    JoshRosen committed Nov 6, 2014
    Configuration menu
    Copy the full SHA
    06c5c54 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/master' into SPARK-4180

    @JoshRosen
    JoshRosen committed Nov 6, 2014
    Configuration menu
    Copy the full SHA
    1c66070 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2014

  1. Address review feedback; expose hack workaround for existing unit tests.

    @JoshRosen
    JoshRosen committed Nov 7, 2014
    Configuration menu
    Copy the full SHA
    ed17e14 View commit details
    Browse the repository at this point in the history

  2. Set spark.driver.allowMultipleContexts=true in tests.

    @JoshRosen
    JoshRosen committed Nov 7, 2014
    Configuration menu
    Copy the full SHA
    4629d5c View commit details
    Browse the repository at this point in the history

  3. Add utility to set system properties in tests.

    @JoshRosen
    JoshRosen committed Nov 7, 2014
    Configuration menu
    Copy the full SHA
    7ba6db8 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2014

  1. Merge remote-tracking branch 'origin/master' into SPARK-4180 

    Conflicts:
	streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
    @JoshRosen
    JoshRosen committed Nov 10, 2014
    Configuration menu
    Copy the full SHA
    a1cba65 View commit details
    Browse the repository at this point in the history

  2. Fix commented out test

    @JoshRosen
    JoshRosen committed Nov 10, 2014
    Configuration menu
    Copy the full SHA
    79a7e6f View commit details
    Browse the repository at this point in the history

  3. Improve handling of failed SparkContext creation attempts.

    @JoshRosen
    JoshRosen committed Nov 10, 2014
    Configuration menu
    Copy the full SHA
    d809cb4 View commit details
    Browse the repository at this point in the history

  4. Update mvn build, too.

    @JoshRosen
    JoshRosen committed Nov 10, 2014
    Configuration menu
    Copy the full SHA
    f5bb78c View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2014

  1. Merge remote-tracking branch 'origin/master' into SPARK-4180 

    Conflicts:
	pom.xml
    @JoshRosen
    JoshRosen committed Nov 14, 2014
    Configuration menu
    Copy the full SHA
    372d0d3 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2014

  1. Incorporate more review feedback.

    @JoshRosen
    JoshRosen committed Nov 15, 2014
    Configuration menu
    Copy the full SHA
    85a424a View commit details
    Browse the repository at this point in the history

  2. Accept boolean instead of SparkConf in methods.

    @JoshRosen
    JoshRosen committed Nov 15, 2014
    Configuration menu
    Copy the full SHA
    c0987d3 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2014

  1. Address latest round of feedback.

    @JoshRosen
    JoshRosen committed Nov 17, 2014
    Configuration menu
    Copy the full SHA
    d38251b View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into SPARK-4180 

    Conflicts:
	core/src/main/scala/org/apache/spark/SparkContext.scala
    @JoshRosen
    JoshRosen committed Nov 17, 2014
    Configuration menu
    Copy the full SHA
    23c7123 View commit details
    Browse the repository at this point in the history