[SPARK-31860][BUILD][2.4] only push release tags on success #28667

holdenk · 2020-05-28T20:56:55Z

What changes were proposed in this pull request?

Only push the release tag after the build has finished.

Why are the changes needed?

If the build fails we don't need a release tag.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Running locally with a fake user

holdenk · 2020-05-28T20:57:10Z

cc @dongjoon-hyun ?

holdenk · 2020-05-28T21:45:46Z

cc @shaneknapp

SparkQA · 2020-05-28T23:41:30Z

Test build #123245 has finished for PR 28667 at commit d26c735.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2020-05-29T00:14:43Z

cc @dbtsai , too

holdenk · 2020-06-01T17:36:52Z

cc @srowen & @cloud-fan

srowen · 2020-06-01T17:40:50Z

I don't know these scripts well, but yes we should only push on success. This looks sensible.

dongjoon-hyun · 2020-06-01T18:17:29Z

Sorry, but I'll close this first to prevent accidental merge on this.
Let's proceed this on master first.

holdenk · 2020-06-01T18:25:51Z

I've got this for the 2.4 branch since I RMd the current release, I'm annoyed you closed this @dongjoon-hyun.

dongjoon-hyun · 2020-06-01T18:39:01Z

This is reopened according to @holdenk 's request.

…rt because when I was doing the 2.4.6 release I found dry_run was still pushing tags so I think something else is broken there but that's a seperate thing to poke at)

dongjoon-hyun

@holdenk . In any way, I believe this help you complete 2.4.6 release. So, sorry for closing this if this made any delay to you or 2.4.6 release.

BTW, could you make a PR on master and branch-3.0 eventually to prevent any regression at 3.0 after 2.4.6?

holdenk · 2020-06-01T19:35:46Z

Yeah I'll do a PR to master & branch-3.0 once I've gotten sign off on this being what we want to do (I know you had some reservations when we chatted).

dbtsai · 2020-06-01T20:59:33Z

It does make sense to push the tag only on success, and let's merge it in 2.4, and then also send out PRs to both master / 3.0. Thanks for working on this!

holdenk · 2020-06-01T21:07:42Z

Ok cool. I'm going to double check running it locally to make sure it the release still runs ok (I did a run last last week but just to be safe since we don't have any automated testing for these scripts), and once that's done I'll merge this and open a master & 3 PR :)

SparkQA · 2020-06-01T21:12:36Z

Test build #123393 has finished for PR 28667 at commit d26c735.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

holdenk · 2020-06-01T21:48:26Z

jenkins retest this please

SparkQA · 2020-06-01T22:13:42Z

Test build #123395 has finished for PR 28667 at commit 9bd1dca.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

holdenk · 2020-06-01T23:12:58Z

Tested build script locally, works ok :)

holdenk · 2020-06-01T23:19:25Z

(looks like I forgot to push the commit I tested it with, pushed now)

SparkQA · 2020-06-02T00:13:06Z

Test build #123401 has finished for PR 28667 at commit 9bd1dca.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

holdenk · 2020-06-02T00:42:31Z

Only test failure is unit test, unrelated and different than the last test failures. I'm going to merge this to branch-2.4 now. There is a separate PR to master at #28700

### What changes were proposed in this pull request? Only push the release tag after the build has finished. ### Why are the changes needed? If the build fails we don't need a release tag. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Running locally with a fake user Closes #28667 from holdenk/SPARK-31860-only-push-release-tags-on-success. Authored-by: Holden Karau <hkarau@apple.com> Signed-off-by: Holden Karau <hkarau@apple.com>

cloud-fan · 2020-06-02T01:45:08Z

dev/create-release/release-tag.sh

@@ -24,7 +24,7 @@ function exit_with_usage {
  local NAME=$(basename $0)
  cat << EOF
 usage: $NAME
-Tags a Spark release on a particular branch.
+Tags a Spark release on a particular branch. Must push after


uncompleted sentence Must push after ...?

SparkQA · 2020-06-02T01:45:31Z

Test build #123403 has finished for PR 28667 at commit dd9e4a4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-06-02T01:46:10Z

previously I delete the tags if something goes wrong. This looks like a better solution

AmplabJenkins · 2020-06-02T01:46:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123403/
Test FAILed.

### What changes were proposed in this pull request? Only push the release tag after the build has finished. ### Why are the changes needed? If the build fails we don't need a release tag. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Running locally with a fake user in #28667 Closes #28700 from holdenk/SPARK-31860-build-master-only-push-tags-on-success. Authored-by: Holden Karau <hkarau@apple.com> Signed-off-by: Holden Karau <hkarau@apple.com>

### What changes were proposed in this pull request? Only push the release tag after the build has finished. ### Why are the changes needed? If the build fails we don't need a release tag. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Running locally with a fake user in #28667 Closes #28700 from holdenk/SPARK-31860-build-master-only-push-tags-on-success. Authored-by: Holden Karau <hkarau@apple.com> Signed-off-by: Holden Karau <hkarau@apple.com> (cherry picked from commit 69ba9b6) Signed-off-by: Holden Karau <hkarau@apple.com>

holdenk added 3 commits May 28, 2020 11:55

Change build to only push tags on success

826e3a4

Don't delete the checkout we just did in the previous step

afa5d90

Don't tell people to wait for a tag that won't show up :p

d26c735

probot-autolabeler bot added BUILD INFRA labels May 28, 2020

holdenk changed the title ~~[WIP][SPARK-31860][BUILD] only push release tags on success~~ [SPARK-31860][BUILD] only push release tags on success Jun 1, 2020

holdenk changed the title ~~[SPARK-31860][BUILD] only push release tags on success~~ [WIP][SPARK-31860][BUILD] only push release tags on success Jun 1, 2020

dongjoon-hyun closed this Jun 1, 2020

dongjoon-hyun reopened this Jun 1, 2020

Fix the is_dry_run logic (although I'm suspcious of our dry_run suppo…

9bd1dca

…rt because when I was doing the 2.4.6 release I found dry_run was still pushing tags so I think something else is broken there but that's a seperate thing to poke at)

holdenk changed the title ~~[WIP][SPARK-31860][BUILD] only push release tags on success~~ [WIP][SPARK-31860][BUILD][2.4] only push release tags on success Jun 1, 2020

dongjoon-hyun reviewed Jun 1, 2020

View reviewed changes

Fix the relase tag dry run spark dir rename

dd9e4a4

holdenk mentioned this pull request Jun 1, 2020

[SPARK-31860][BUILD] only push release tags on succes #28700

Closed

holdenk changed the title ~~[WIP][SPARK-31860][BUILD][2.4] only push release tags on success~~ [SPARK-31860][BUILD][2.4] only push release tags on success Jun 1, 2020

cloud-fan reviewed Jun 2, 2020

View reviewed changes

holdenk closed this Jun 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-31860][BUILD][2.4] only push release tags on success #28667

[SPARK-31860][BUILD][2.4] only push release tags on success #28667

holdenk commented May 28, 2020 •

edited

Loading

holdenk commented May 28, 2020

holdenk commented May 28, 2020

SparkQA commented May 28, 2020

dongjoon-hyun commented May 29, 2020

holdenk commented Jun 1, 2020

srowen commented Jun 1, 2020

dongjoon-hyun commented Jun 1, 2020 •

edited

Loading

holdenk commented Jun 1, 2020

dongjoon-hyun commented Jun 1, 2020

dongjoon-hyun left a comment

holdenk commented Jun 1, 2020

dbtsai commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 1, 2020

holdenk commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 2, 2020

holdenk commented Jun 2, 2020

cloud-fan Jun 2, 2020

SparkQA commented Jun 2, 2020

cloud-fan commented Jun 2, 2020

AmplabJenkins commented Jun 2, 2020

[SPARK-31860][BUILD][2.4] only push release tags on success #28667

[SPARK-31860][BUILD][2.4] only push release tags on success #28667

Conversation

holdenk commented May 28, 2020 • edited Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

holdenk commented May 28, 2020

holdenk commented May 28, 2020

SparkQA commented May 28, 2020

dongjoon-hyun commented May 29, 2020

holdenk commented Jun 1, 2020

srowen commented Jun 1, 2020

dongjoon-hyun commented Jun 1, 2020 • edited Loading

holdenk commented Jun 1, 2020

dongjoon-hyun commented Jun 1, 2020

dongjoon-hyun left a comment

Choose a reason for hiding this comment

holdenk commented Jun 1, 2020

dbtsai commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 1, 2020

holdenk commented Jun 1, 2020

holdenk commented Jun 1, 2020

SparkQA commented Jun 2, 2020

holdenk commented Jun 2, 2020

cloud-fan Jun 2, 2020

Choose a reason for hiding this comment

SparkQA commented Jun 2, 2020

cloud-fan commented Jun 2, 2020

AmplabJenkins commented Jun 2, 2020

holdenk commented May 28, 2020 •

edited

Loading

dongjoon-hyun commented Jun 1, 2020 •

edited

Loading