diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index d8985cfd78013..908a032bf833e 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -17,7 +17,7 @@ Resolves #[Issue number to be closed when this PR is merged] - [ ] All tests pass - [ ] New functionality has been documented. - [ ] New functionality has javadoc added -- [ ] Failing checks are inspected and point to the corresponding known issue(s) +- [ ] Failing checks are inspected and point to the corresponding known issue(s) (See: [Troubleshooting Failing Builds](../blob/main/CONTRIBUTING.md#troubleshooting-failing-builds)) - [ ] Commits are signed per the DCO using --signoff - [ ] Commit changes are listed out in CHANGELOG.md file (See: [Changelog](../blob/main/CONTRIBUTING.md#changelog)) - [ ] Public documentation issue/PR [created](https://github.com/opensearch-project/documentation-website/issues/new/choose) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cfc01c359de67..4a1162cf2558b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -166,7 +166,7 @@ If we reject the PR, we will close the pull request with a comment explaining wh ## Troubleshooting Failing Builds -The OpenSearch testing framework uses a lot of randomization internally to cover as many edge cases and variations as possible. Unfortunately, this posses a challenge since in most cases, every build is unique and may not trigger a particular flow. It leads straight to so called flaky tests - the tests which flip randomly from success to failure without any code changes. +The OpenSearch testing framework offers many capabilities but exhibits significant complexity (it does lot of randomization internally to cover as many edge cases and variations as possible). Unfortunately, this posses a challenge by making it harder to discover important issues/bugs in straightforward way and may lead to so called flaky tests - the tests which flip randomly from success to failure without any code changes. If your pull request reports a failing test(s) on one of the checks, please: - look if there is an existing [issue](https://github.com/opensearch-project/OpenSearch/issues) reported for the test in question