Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-3623][GraphX] GraphX should support the checkpoint operation #2631

Closed
wants to merge 3 commits into from
Closed

[SPARK-3623][GraphX] GraphX should support the checkpoint operation #2631

wants to merge 3 commits into from

Conversation

witgo
Copy link
Contributor

@witgo witgo commented Oct 2, 2014

No description provided.

@witgo witgo changed the title [WIP][SPARK-3625][SPARK-3623]Graph should support the checkpoint operation [WIP][SPARK-3625][SPARK-3623] GraphX should support the checkpoint operation Oct 2, 2014
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21204/

@ankurdave
Copy link
Contributor

Jenkins, retest this please

@SparkQA
Copy link

SparkQA commented Oct 2, 2014

QA tests have started for PR 2631 at commit 48f2f5e.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 2, 2014

QA tests have finished for PR 2631 at commit 48f2f5e.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21210/

@witgo witgo changed the title [WIP][SPARK-3625][SPARK-3623] GraphX should support the checkpoint operation [WIP][SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation Oct 3, 2014
@witgo witgo mentioned this pull request Oct 8, 2014
Closed
9 tasks
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@witgo witgo changed the title [WIP][SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation [SPARK-3625][SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation Oct 22, 2014
@witgo witgo changed the title [SPARK-3625][SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation [SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation Oct 22, 2014
@witgo witgo mentioned this pull request Oct 22, 2014
Closed
@witgo witgo force-pushed the SPARK-3623 branch 2 times, most recently from 8d6f537 to 43a1fb4 Compare December 3, 2014 01:53
@ankurdave
Copy link
Contributor

Due to https://issues.apache.org/jira/browse/SPARK-4672, we now support checkpointing graphs (by checkpointing their constituent vertices and edges) with the same semantics as RDDs, meaning that you have to checkpoint an RDD before calling an action on it.

This PR additionally adds support for checkpointing after calling actions, but that is not strictly necessary and involves a behavior change to Spark core. So I think we should close this PR.

@witgo witgo changed the title [SPARK-3625][SPARK-3623][GraphX] GraphX should support the checkpoint operation [SPARK-3623][GraphX] GraphX should support the checkpoint operation Dec 6, 2014
@witgo
Copy link
Contributor Author

witgo commented Dec 6, 2014

@ankurdave
I have removed the Spark core related to modify

@ankurdave
Copy link
Contributor

Great, thanks! LGTM - I'll merge this after Jenkins runs.

ok to test

@witgo
Copy link
Contributor Author

witgo commented Dec 6, 2014

Jenkins, retest this please.

1 similar comment
@ankurdave
Copy link
Contributor

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Dec 6, 2014

Test build #24208 has started for PR 2631 at commit a70c500.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 6, 2014

Test build #24208 has finished for PR 2631 at commit a70c500.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24208/
Test PASSed.

asfgit pushed a commit that referenced this pull request Dec 6, 2014
@witgo @ankurdave
[SPARK-3623][GraphX] GraphX should support the checkpoint operation
27d9f13 
Author: GuoQiang Li <witgo@qq.com>

Closes #2631 from witgo/SPARK-3623 and squashes the following commits:

a70c500 [GuoQiang Li] Remove java related
4d1e249 [GuoQiang Li] Add comments
e682724 [GuoQiang Li] Graph should support the checkpoint operation

(cherry picked from commit e895e0c)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
@asfgit asfgit closed this in e895e0c Dec 6, 2014
@ankurdave
Copy link
Contributor

Thanks! Merged into master & branch-1.2.

@witgo witgo deleted the SPARK-3623 branch December 6, 2014 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@witgo @AmplabJenkins @ankurdave @SparkQA