Skip to content

Commit

Permalink
Merge pull request #2701 from nikhita/replace-submit-queue-with-tide
Browse files Browse the repository at this point in the history
Replace submit queue with Tide in docs
  • Loading branch information
k8s-ci-robot authored Oct 8, 2018
2 parents ea9d309 + c2bd5cb commit f1e2ffa
Show file tree
Hide file tree
Showing 7 changed files with 42 additions and 93 deletions.
58 changes: 9 additions & 49 deletions contributors/devel/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,65 +6,25 @@ Kubernetes uses a variety of automated tools in an attempt to relieve developers
of repetitive, low brain power work. This document attempts to describe these
processes.

## Tide

## Submit Queue
This project formerly used a Submit Queue, it has since been replaced by
[Tide](https://git.k8s.io/test-infra/prow/cmd/tide).

In an effort to
* reduce load on core developers
* maintain end-to-end test stability
* load test github's label feature
#### Ready to merge status

We have added an automated [submit-queue](https://git.k8s.io/test-infra/mungegithub/submit-queue)
to the
[github "munger"](https://git.k8s.io/test-infra/mungegithub)
for kubernetes.

The submit-queue does the following:

```go
for _, pr := range readyToMergePRs() {
if testsAreStable() {
if retestPR(pr) == success {
mergePR(pr)
}
}
}
```

The status of the submit-queue is [online.](http://submit-queue.k8s.io/)

### Ready to merge status

A PR is considered "ready for merging" by the submit queue if it matches the set
of conditions listed in the [merge requirements tab](http://submit-queue.k8s.io/#/info)
of the info page.
A PR is considered "ready for merging" by Tide if it matches the set
of conditions listed in the [Tide dashboard](https://prow.k8s.io/tide).
Please visit that page for more details.

### Merge process

If the PR has the `retest-not-required` label, it is simply merged. If the PR does
not have this label, the aforementioned required tests are re-run.
If these tests pass a second time, the PR will be merged when this PR finishes retesting.

## Github Munger

We run [github "mungers"](https://git.k8s.io/test-infra/mungegithub).

This runs repeatedly over github pulls and issues and runs modular "mungers".
The mungers include the "submit-queue" referenced above along
with numerous other functions. See the README in the link above.

Please feel free to unleash your creativity on this tool, send us new mungers
that you think will help support the Kubernetes development process.

### Closing stale pull-requests

Github Munger will close pull-requests that don't have human activity in the
Prow will close pull-requests that don't have human activity in the
last 90 days. It will warn about this process 60 days before closing the
pull-request, and warn again 30 days later. One way to prevent this from
happening is to add the `keep-open` label on the pull-request.
happening is to add the `lifecycle/frozen` label on the pull-request.

Feel free to re-open and maybe add the `keep-open` label if this happens to a
Feel free to re-open and maybe add the `lifecycle/frozen` label if this happens to a
valid pull-request. It may also be a good opportunity to get more attention by
verifying that it is properly assigned and/or mention people that might be
interested. Commenting on the pull-request will also keep it open for another 90
Expand Down
19 changes: 7 additions & 12 deletions contributors/devel/e2e-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -576,7 +576,7 @@ suite, it receives a `[Feature:.+]` label, e.g. `[Feature:Performance]` or
`[Feature:Ingress]`. `[Feature:.+]` tests are not run in our core suites,
instead running in custom suites. If a feature is experimental or alpha and is
not enabled by default due to being incomplete or potentially subject to
breaking changes, it does *not* block the merge-queue, and thus should run in
breaking changes, it does *not* block PR merges, and thus should run in
some separate test suites owned by the feature owner(s)
(see [Continuous Integration](#continuous-integration) below).

Expand Down Expand Up @@ -611,12 +611,10 @@ A quick overview of how we run e2e CI on Kubernetes.
### What is CI?
We run a battery of `e2e` tests against `HEAD` of the master branch on a
continuous basis, and block merges via the [submit
queue](http://submit-queue.k8s.io/) on a subset of those tests if they fail (the
subset is defined in the [munger config](https://git.k8s.io/test-infra/mungegithub/mungers/submit-queue.go)
via the `jenkins-jobs` flag; note we also block on `kubernetes-build` and
`kubernetes-test-go` jobs for build and unit and integration tests).
We run a battery of [release-blocking jobs](https://k8s-testgrid.appspot.com/sig-release-master-blocking)
against `HEAD` of the master branch on a continuous basis, and block merges
via [Tide](https://git.k8s.io/test-infra/prow/cmd/tide) on a subset of those
tests if they fail.
CI results can be found at [ci-test.k8s.io](http://ci-test.k8s.io), e.g.
[ci-test.k8s.io/kubernetes-e2e-gce/10594](http://ci-test.k8s.io/kubernetes-e2e-gce/10594).
Expand Down Expand Up @@ -684,14 +682,11 @@ contend for resources; see above about [kinds of tests](#kinds_of_tests).
Generally, a feature starts as `experimental`, and will be run in some suite
owned by the team developing the feature. If a feature is in beta or GA, it
*should* block the merge-queue. In moving from experimental to beta or GA, tests
*should* block PR merges and releases. In moving from experimental to beta or GA, tests
that are expected to pass by default should simply remove the `[Feature:.+]`
label, and will be incorporated into our core suites. If tests are not expected
to pass by default, (e.g. they require a special environment such as added
quota,) they should remain with the `[Feature:.+]` label, and the suites that
run them should be incorporated into the
[munger config](https://git.k8s.io/test-infra/mungegithub/mungers/submit-queue.go)
via the `jenkins-jobs` flag.
quota,) they should remain with the `[Feature:.+]` label.
Occasionally, we'll want to add tests to better exercise features that are
already GA. These tests also shouldn't go straight to CI. They should begin by
Expand Down
8 changes: 3 additions & 5 deletions contributors/devel/flaky-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,18 +76,16 @@ we have the following guidelines:
3. If you can reproduce it (or it's obvious from the logs what happened), you
should then be able to fix it, or in the case where someone is clearly more
qualified to fix it, reassign it with very clear instructions.
4. PRs that fix or help debug flakes may have the P0 priority set to get them
through the merge queue as fast as possible.
5. Once you have made a change that you believe fixes a flake, it is conservative
4. Once you have made a change that you believe fixes a flake, it is conservative
to keep the issue for the flake open and see if it manifests again after the
change is merged.
6. If you can't reproduce a flake: __don't just close it!__ Every time a flake comes
5. If you can't reproduce a flake: __don't just close it!__ Every time a flake comes
back, at least 2 hours of merge time is wasted. So we need to make monotonic
progress towards narrowing it down every time a flake occurs. If you can't
figure it out from the logs, add log messages that would have help you figure
it out. If you make changes to make a flake more reproducible, please link
your pull request to the flake you're working on.
7. If a flake has been open, could not be reproduced, and has not manifested in
6. If a flake has been open, could not be reproduced, and has not manifested in
3 months, it is reasonable to close the flake issue with a note saying
why.

Expand Down
2 changes: 1 addition & 1 deletion contributors/devel/writing-good-e2e-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ designing, writing and debugging your end-to-end tests. In
particular, "flaky" tests, which pass most of the time but fail
intermittently for difficult-to-diagnose reasons are extremely costly
in terms of blurring our regression signals and slowing down our
automated merge queue. Up-front time and effort designing your test
automated merge velocity. Up-front time and effort designing your test
to be reliable is very well spent. Bear in mind that we have hundreds
of tests, each running in dozens of different environments, and if any
test in any test environment fails, we have to assume that we
Expand Down
4 changes: 2 additions & 2 deletions contributors/guide/contributor-cheatsheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ A list of common resources when contributing to Kubernetes.
## Workflow

- [Gubernator Dashboard - k8s.reviews](https://k8s-gubernator.appspot.com/pr)
- [Submit Queue](https://submit-queue.k8s.io)
- [Tide](https://prow.k8s.io/tide)
- [Bot commands](https://go.k8s.io/bot-commands)
- [GitHub labels](https://go.k8s.io/github-labels)
- [Release Buckets](https://gcsweb.k8s.io/gcs/kubernetes-release/)
- Developer Guide
- [Cherry Picking Guide](/contributors/devel/cherry-picks.md) - [Queue](https://cherrypick.k8s.io/#/queue)
- [Cherry Picking Guide](/contributors/devel/cherry-picks.md)
- [Kubernetes Code Search](https://cs.k8s.io/), maintained by [@dims](https://github.com/dims)


Expand Down
24 changes: 9 additions & 15 deletions contributors/guide/owners.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,21 +209,15 @@ is the state of today.

## Automation using OWNERS files

### ~[`mungegithub`](https://git.k8s.io/test-infra/mungegithub)~ is deprecated

Mungegithub's blunderbuss and submit-queue mungers are currently used for kubernetes/kubernetes. Their
equivalents are the prow blunderbuss plugin, and prow's tide cmd. These docs will be removed once
kubernetes/kubernetes has transitioned over to tide.

~Mungegithub polls GitHub, and "munges" things it finds, including issues and pull requests. It is
stateful, in that restarting it means it loses track of which things it has munged at what time.~

- ~[munger:
blunderbuss](https://git.k8s.io/test-infra/mungegithub/mungers/blunderbuss.go)~
- ~responsible for determining **reviewers** and assigning to them~
- [munger:
submit-queue](https://git.k8s.io/test-infra/mungegithub/mungers/submit-queue.go)
- responsible for merging PR's
Kubernetes uses the Prow Blunderbuss plugin and Tide.
Tide uses GitHub queries to select PRs into “tide pools”, runs as many in a
batch as it can (“tide comes in”), and merges them (“tide goes out”).

- [Blunderbuss plugin](https://git.k8s.io/test-infra/prow/plugins/blunderbuss):
- responsible for determining **reviewers**
- [Tide](https://git.k8s.io/test-infra/prow/cmd/tide):
- responsible for automatically running batch tests and merging multiple PRs together whenever possible.
- responsible for retriggering stale PR tests.
- responsible for updating a GitHub status check explaining why a PR can't be merged (eg: a
missing `lgtm` or `approved` label)

Expand Down
20 changes: 11 additions & 9 deletions contributors/guide/pull-requests.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Here's the process the pull request goes through on its way from submission to m

1. If you're **not** a member of the Kubernetes organization, a Reviewer/Kubernetes Member checks that the pull request is safe to test. If so, they comment `/ok-to-test`. Pull requests by Kubernetes organization [members](/community-membership.md) do not need this step. Now the pull request is considered to be trusted, and the pre-submit tests will run:

1. Automatic tests run. See the current list of tests on the [MERGE REQUIREMENTS tab, at this link](https://submit-queue.k8s.io/#/info)
1. Automatic tests run. See the current list of tests at this [link](https://prow.k8s.io/?repo=kubernetes%2Fkubernetes&type=presubmit)
1. If tests fail, resolve issues by pushing edits to your pull request branch
1. If the failure is a flake, anyone on trusted pull requests can comment `/retest` to rerun failed tests

Expand All @@ -82,24 +82,26 @@ Here's the process the pull request goes through on its way from submission to m
1. (Optional) Some reviewers prefer that you squash commits at this step
1. Follow the bot suggestions to assign an OWNER who will add the `/approve` label to the pull request

Once the tests pass, all failures are commented as flakes, or the reviewer adds the labels `/lgtm` and `/approved`, the pull request enters the final merge queue. The merge queue is needed to make sure no incompatible changes have been introduced by other pull requests since the tests were last run on your pull request.
Once the tests pass and the reviewer adds the `lgtm` and `approved` labels, the pull request enters the final merge pool. The merge pool is needed to make sure no incompatible changes have been introduced by other pull requests since the tests were last run on your pull request.
<!-- TODO: create parallel instructions for reviewers -->

The [GitHub "munger"](https://git.k8s.io/test-infra/mungegithub) submit-queue plugin will manage the merge queue automatically.
[Tide](https://git.k8s.io/test-infra/prow/cmd/tide) will manage the merge pool
automatically. It uses GitHub queries to select PRs into “tide pools”,
runs as many in a batch as it can (“tide comes in”), and merges them (“tide goes out”).

1. The pull request enters the merge queue ([http://submit-queue.k8s.io](http://submit-queue.k8s.io))
1. The merge queue triggers a test re-run with the comment `/test all [submit-queue is verifying that this pull request is safe to merge]`
1. Author has signed the CLA (`cncf-cla: yes` label added to pull request)
1. No changes made since last `lgtm` label applied
1. The pull request enters the [merge pool](https://prow.k8s.io/tide)
if the merge criteria are met. The [PR dashboard](https://prow.k8s.io/pr) shows
the difference between your PR's state and the merge criteria so that you can
easily see all criteria that are not being met and address them.
1. If tests fail, resolve issues by pushing edits to your pull request branch
1. If the failure is a flake, anyone can comment `/retest` if the pull request is trusted
1. If tests pass, the merge queue automatically merges the pull request
1. If tests pass, Tide automatically merges the pull request

That's the last step. Your pull request is now merged.

## Marking Unfinished Pull Requests

If you want to solicit reviews before the implementation of your pull request is complete, you should hold your pull request to ensure that the merge queue does not pick it up and attempt to merge it. There are two methods to achieve this:
If you want to solicit reviews before the implementation of your pull request is complete, you should hold your pull request to ensure that Tide does not pick it up and attempt to merge it. There are two methods to achieve this:

1. You may add the `/hold` or `/hold cancel` comment commands
2. You may add or remove a `WIP` or `[WIP]` prefix to your pull request title
Expand Down

0 comments on commit f1e2ffa

Please sign in to comment.