Refactored Dataframe join comment to use correct parameter ordering #4847
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
March 1, 2015 22:41
The API signatire for join requires the JoinType to be the third parameter. The code examples provided for join show JoinType being provided as the 2nd parater resuling in errors (i.e. "df1.join(df2, "outer", $"df1Key" === $"df2Key") ). The correct example should be df1.join(df2, $"df1Key" === $"df2Key", "outer")
Updated comments use correct sample code for Dataframe joins
|
Can one of the admins verify this patch? |
|
LGTM since it's definitely the last arg that specifies the join type, as you can see there, but will give @rxin a day to look |
| * | ||
| * // Java: | ||
| * import static org.apache.spark.sql.functions.*; | ||
| * df1.join(df2, "outer", col("df1Key") === col("df2Key")); | ||
| * df1.join(df2, col("df1Key") === col("df2Key"), "outer"); |
There was a problem hiding this comment.
actually while you are at it, do you mind fixing the java example? it definitely wouldn't work with col("df1key") === col("df2key").
I think you need col("df1Key").equalTo(col("df2Key"))
There was a problem hiding this comment.
I'll make this change while merging.
asfgit
pushed a commit
that referenced
this pull request
Mar 2, 2015
…ering The API signatire for join requires the JoinType to be the third parameter. The code examples provided for join show JoinType being provided as the 2nd parater resuling in errors (i.e. "df1.join(df2, "outer", $"df1Key" === $"df2Key") ). The correct sample code is df1.join(df2, $"df1Key" === $"df2Key", "outer") Author: Paul Power <paul.power@peerside.com> Closes #4847 from peerside/master and squashes the following commits: ebc1efa [Paul Power] Merge pull request #1 from peerside/peerside-patch-1 e353340 [Paul Power] Updated comments use correct sample code for Dataframe joins (cherry picked from commit d9a8bae) Signed-off-by: Michael Armbrust <michael@databricks.com>
|
Thanks! Merged to master and 1.3 with a small change for java. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The API signatire for join requires the JoinType to be the third parameter. The code examples provided for join show JoinType being provided as the 2nd parater resuling in errors (i.e. "df1.join(df2, "outer",$"df1Key" === $ "df2Key") ). The correct sample code is df1.join(df2, $"df1Key" === $ "df2Key", "outer")