Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-1371] fix computePreferredLocations signature to not depend on underlying implementation #302

Closed
wants to merge 2 commits into from
Closed

[SPARK-1371] fix computePreferredLocations signature to not depend on underlying implementation #302

wants to merge 2 commits into from

Conversation

mridulm
Copy link
Contributor

@mridulm mridulm commented Apr 2, 2014

Change to Map and Set - not mutable HashMap and HashSet

@AmplabJenkins
Copy link

Merged build triggered.

@mridulm mridulm changed the title fix computePreferredLocations signature to not depend on underlying implementation [spark] [SPARK-1371] fix computePreferredLocations signature to not depend on underlying implementation Apr 2, 2014
@mridulm mridulm changed the title [spark] [SPARK-1371] fix computePreferredLocations signature to not depend on underlying implementation [SPARK-1371] fix computePreferredLocations signature to not depend on underlying implementation Apr 2, 2014
@mridulm
Copy link
Contributor Author

mridulm commented Apr 2, 2014

Jenkins, this is ok to test

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13682/

andrewor14
andrewor14 reviewed Apr 2, 2014
View reviewed changes
core/src/main/scala/org/apache/spark/scheduler/InputFormatInfo.scala
@@ -164,7 +164,7 @@ object InputFormatInfo {

PS: I know the wording here is weird, hopefully it makes some sense !
*/
def computePreferredLocations(formats: Seq[InputFormatInfo]): HashMap[String, HashSet[SplitInfo]]
def computePreferredLocations(formats: Seq[InputFormatInfo]): Map[String, Set[SplitInfo]]
= {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: now that there's space in L167, move = { up one line

@andrewor14
Copy link
Contributor

What is the motivation behind this change?

@mridulm
Copy link
Contributor Author

mridulm commented Apr 3, 2014

Will make the changes, thx for review @andrewor14
Motivation is Interface cleanup - when I initially wrote it, it was more for my local testing : and exposed it since someone found it useful internally.
I forgot to clean up the interface to make it compatible with what SparkContext accepts

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13719/

@mateiz
Copy link
Contributor

mateiz commented Apr 5, 2014

Looks good, thanks Mridul.

@asfgit asfgit closed this in 6e88583 Apr 5, 2014
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@pdeyhim
[SPARK-1371] fix computePreferredLocations signature to not depend on…
161e79a 
… underlying implementation

Change to Map and Set - not mutable HashMap and HashSet

Author: Mridul Muralidharan <mridulm80@apache.org>

Closes apache#302 from mridulm/master and squashes the following commits:

df747af [Mridul Muralidharan] Address review comments
17e2907 [Mridul Muralidharan] fix computePreferredLocations signature to not depend on underlying implementation
gatesn pushed a commit to gatesn/spark that referenced this pull request Mar 14, 2018
@robert3005
Fixup make distribution dockerfile copies (apache#302)
ab1b681
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
@liusheng
Merge pull request apache#302 from theopenlab/osb-checker-fix
c39e8b5 
Fix huaweicloud open service broker test error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mridulm @AmplabJenkins @andrewor14 @mateiz