Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-12517] add default RDD name for one created via sc.textFile #10456

Closed
wants to merge 10 commits into from
Closed

[SPARK-12517] add default RDD name for one created via sc.textFile #10456

wants to merge 10 commits into from

Commits on Dec 23, 2015

  1. add default RDD name for one create via sc.textFile

    The feature was first added at commit: 7b877b2 but was later removed (probably by mistake) at at commit: fc8b581.
    
    here is the symptom:
    
    using spark-1.5.2-bin-hadoop2.6 I get:
    =================================
    scala> sc.textFile("/home/root/.bashrc").name
    res5: String = null
    
    scala> sc.binaryFiles("/home/root/.bashrc").name
    res6: String = /home/root/.bashrc
    
    while using Spark 1.3.1:
    =================================
    
    scala> sc.textFile("/home/root/.bashrc").name
    res0: String = /home/root/.bashrc
    
    scala> sc.binaryFiles("/home/root/.bashrc").name
    res1: String = /home/root/.bashrc
    yaron committed Dec 23, 2015
    Configuration menu
    Copy the full SHA
    86efdad View commit details
    Browse the repository at this point in the history

Commits on Dec 24, 2015

  1. add unit test for setting default RDD name to its path (SPARK-12517)

    This change extends SparkContextSuite to verify that RDDs that are created using file paths have their
    default name set to the path.
    wyaron committed Dec 24, 2015
    Configuration menu
    Copy the full SHA
    081a723 View commit details
    Browse the repository at this point in the history
  2. add unit test for setting default RDD name to its path (SPARK-12517)

    This change extends SparkContextSuite to verify that RDDs that are created using file paths have their
    default name set to the path.
    wyaron committed Dec 24, 2015
    Configuration menu
    Copy the full SHA
    1bc5ff8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    43d54ca View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    fe3d87d View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a2f5f07 View commit details
    Browse the repository at this point in the history
  6. fix compile error in test unit

    wyaron committed Dec 24, 2015
    Configuration menu
    Copy the full SHA
    50cd119 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    825f3f3 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    99b5bde View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    08d3fe2 View commit details
    Browse the repository at this point in the history