Skip to content

Commit

Permalink
Contribution guide to document actual guide for pull requests
Browse files Browse the repository at this point in the history
Currently contribution guide does not reflect actual flow to raise a new PR and hence it's not clear (for a new contributors) what exactly needs to be done to make a PR for Spark repository and test it as per expectation. This PR addresses that by following:

- It describes in the Pull request section of the Contributing page the actual procedure and takes a contributor through a step by step process.
- It removes optional "Running tests in your forked repository" section on Developer Tools page which is obsolete now and doesn't reflect reality anymore i.e. it says we can test by clicking “Run workflow” button which is not available anymore as workflow does not use "workflow_dispatch" event trigger anymore and was removed in  apache/spark#32092
- Instead it documents the new procedure that above PR introduced i.e. contributors needs to use their own GitHub free workflow credits to test new changes they are purposing and a Spark Actions workflow will expect that to be completed before marking PR to be ready for a review.
- Some general wording was copied from  "Running tests in your forked repository" section on Developer Tools page but main content was rewritten to meet objective
- Also fixed URL to developer-tools.html to be resolved by parser (that converted it into relative URI) instead of using hard coded absolute URL.

Tested imperically with `bundle exec jekyll serve` and static files were generated with `bundle exec jekyll build` commands

This closes https://issues.apache.org/jira/browse/SPARK-37996

Author: khalidmammadov <xmamedov@hotmail.com>

Closes #378 from khalidmammadov/fix_contribution_workflow_guide.
  • Loading branch information
khalidmammadov authored and srowen committed Feb 4, 2022
1 parent 87d73df commit 991df19
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 47 deletions.
21 changes: 15 additions & 6 deletions contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,9 +322,16 @@ Example: `Fix typos in Foo scaladoc`

<h3>Pull request</h3>

Before creating a pull request in Apache Spark, it is important to check if tests can pass on your branch because
our GitHub Actions workflows automatically run tests for your pull request/following commits
and every run burdens the limited resources of GitHub Actions in Apache Spark repository.
Below steps will take your through the process.


1. <a href="https://help.github.com/articles/fork-a-repo/">Fork</a> the GitHub repository at
<a href="https://github.com/apache/spark">https://github.com/apache/spark</a> if you haven't already
1. Clone your fork, create a new branch, push commits to the branch.
1. Go to "Actions" tab on your forked repository and enable "Build and test" and "Report test results" workflows
1. Clone your fork and create a new branch
1. Consider whether documentation or tests need to be added or updated as part of the change,
and add them as needed.
1. When you add tests, make sure the tests are self-descriptive.
Expand Down Expand Up @@ -355,14 +362,16 @@ and add them as needed.
...
```
1. Consider whether benchmark results should be added or updated as part of the change, and add them as needed by
<a href="https://spark.apache.org/developer-tools.html#github-workflow-benchmarks">Running benchmarks in your forked repository</a>
<a href="{{site.baseurl}}/developer-tools.html#github-workflow-benchmarks">Running benchmarks in your forked repository</a>
to generate benchmark results.
1. Run all tests with `./dev/run-tests` to verify that the code still compiles, passes tests, and
passes style checks. Alternatively you can run the tests via GitHub Actions workflow by
<a href="https://spark.apache.org/developer-tools.html#github-workflow-tests">Running tests in your forked repository</a>.
passes style checks.
If style checks fail, review the Code Style Guide below.
1. Push commits to your branch. This will trigger "Build and test" and "Report test results" workflows
on your forked repository and start testing and validating your changes.
1. <a href="https://help.github.com/articles/using-pull-requests/">Open a pull request</a> against
the `master` branch of `apache/spark`. (Only in special cases would the PR be opened against other branches.)
the `master` branch of `apache/spark`. (Only in special cases would the PR be opened against other branches). This
will trigger workflows "On pull request*" (on Spark repo) that will look/watch for successful workflow runs on "your" forked repository (it will wait if one is running).
1. The PR title should be of the form `[SPARK-xxxx][COMPONENT] Title`, where `SPARK-xxxx` is
the relevant JIRA number, `COMPONENT `is one of the PR categories shown at
<a href="https://spark-prs.appspot.com/">spark-prs.appspot.com</a> and
Expand Down Expand Up @@ -475,4 +484,4 @@ We expect this code of conduct to be honored by everyone who participates in the

This code <u>is not exhaustive or complete</u>. It serves to distill our common understanding of a collaborative, shared environment and goals. We expect it to be followed in spirit as much as in the letter, so that it can enrich all of us and the technical communities in which we participate.

For more information and specific guidelines, refer to the <a href="https://www.apache.org/foundation/policies/conduct.html">Apache Software Foundation Code of Conduct</a>.
For more information and specific guidelines, refer to the <a href="https://www.apache.org/foundation/policies/conduct.html">Apache Software Foundation Code of Conduct</a>.
17 changes: 0 additions & 17 deletions developer-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,23 +246,6 @@ Kubernetes, and more importantly, minikube have rapid release cycles, and point

Apache Spark leverages GitHub Actions that enables continuous integration and a wide range of automation. Apache Spark repository provides several GitHub Actions workflows for developers to run before creating a pull request.

<a name="github-workflow-tests"></a>
<h4>Running tests in your forked repository</h4>

Before creating a pull request in Apache Spark, it is important to check if tests can pass on your branch because our GitHub Acrions workflows automatically run tests for your pull request/following commits, and every run burdens the limited resources of GitHub Actions in Apache Spark repository.

Apache Spark repository has a workflow that enables you to run the same tests for a branch in your own forked repository that does not burden the resource from Apache Spark repository.

For example, suppose that you have a branch named "your_branch" for a pull request.
To run tests on "your_branch" and check test results:

- Click the "Actions" tab in your forked repository.
- Select the "Build and test" workflow in the "All workflows" list.
- Click the "Run workflow" button and enter "your_branch" in the "Target branch to run" field.
- Once the "Build and test" workflow is finished, click the "Report test results" workflow to check test results.

<img src="/images/running-tests-using-github-actions.png" style="width: 100%; max-width: 800px;" />

<a name="github-workflow-benchmarks"></a>
<h4>Running benchmarks in your forked repository</h4>

Expand Down
Binary file removed images/running-tests-using-github-actions.png
Binary file not shown.
18 changes: 13 additions & 5 deletions site/contributing.html
Original file line number Diff line number Diff line change
Expand Up @@ -503,10 +503,16 @@ <h3>JIRA</h3>

<h3>Pull request</h3>

<p>Before creating a pull request in Apache Spark, it is important to check if tests can pass on your branch because
our GitHub Actions workflows automatically run tests for your pull request/following commits
and every run burdens the limited resources of GitHub Actions in Apache Spark repository.
Below steps will take your through the process.</p>

<ol>
<li><a href="https://help.github.com/articles/fork-a-repo/">Fork</a> the GitHub repository at
<a href="https://github.com/apache/spark">https://github.com/apache/spark</a> if you haven&#8217;t already</li>
<li>Clone your fork, create a new branch, push commits to the branch.</li>
<li>Go to &#8220;Actions&#8221; tab on your forked repository and enable &#8220;Build and test&#8221; and &#8220;Report test results&#8221; workflows</li>
<li>Clone your fork and create a new branch</li>
<li>Consider whether documentation or tests need to be added or updated as part of the change,
and add them as needed.
<ol>
Expand Down Expand Up @@ -543,14 +549,16 @@ <h3>Pull request</h3>
</ol>
</li>
<li>Consider whether benchmark results should be added or updated as part of the change, and add them as needed by
<a href="https://spark.apache.org/developer-tools.html#github-workflow-benchmarks">Running benchmarks in your forked repository</a>
<a href="/developer-tools.html#github-workflow-benchmarks">Running benchmarks in your forked repository</a>
to generate benchmark results.</li>
<li>Run all tests with <code class="language-plaintext highlighter-rouge">./dev/run-tests</code> to verify that the code still compiles, passes tests, and
passes style checks. Alternatively you can run the tests via GitHub Actions workflow by
<a href="https://spark.apache.org/developer-tools.html#github-workflow-tests">Running tests in your forked repository</a>.
passes style checks.
If style checks fail, review the Code Style Guide below.</li>
<li>Push commits to your branch. This will trigger &#8220;Build and test&#8221; and &#8220;Report test results&#8221; workflows
on your forked repository and start testing and validating your changes.</li>
<li><a href="https://help.github.com/articles/using-pull-requests/">Open a pull request</a> against
the <code class="language-plaintext highlighter-rouge">master</code> branch of <code class="language-plaintext highlighter-rouge">apache/spark</code>. (Only in special cases would the PR be opened against other branches.)
the <code class="language-plaintext highlighter-rouge">master</code> branch of <code class="language-plaintext highlighter-rouge">apache/spark</code>. (Only in special cases would the PR be opened against other branches). This
will trigger workflows &#8220;On pull request*&#8221; (on Spark repo) that will look/watch for successful workflow runs on &#8220;your&#8221; forked repository (it will wait if one is running).
<ol>
<li>The PR title should be of the form <code class="language-plaintext highlighter-rouge">[SPARK-xxxx][COMPONENT] Title</code>, where <code class="language-plaintext highlighter-rouge">SPARK-xxxx</code> is
the relevant JIRA number, <code class="language-plaintext highlighter-rouge">COMPONENT </code>is one of the PR categories shown at
Expand Down
19 changes: 0 additions & 19 deletions site/developer-tools.html
Original file line number Diff line number Diff line change
Expand Up @@ -374,25 +374,6 @@ <h3>Testing with GitHub actions workflow</h3>

<p>Apache Spark leverages GitHub Actions that enables continuous integration and a wide range of automation. Apache Spark repository provides several GitHub Actions workflows for developers to run before creating a pull request.</p>

<p><a name="github-workflow-tests"></a></p>
<h4>Running tests in your forked repository</h4>

<p>Before creating a pull request in Apache Spark, it is important to check if tests can pass on your branch because our GitHub Acrions workflows automatically run tests for your pull request/following commits, and every run burdens the limited resources of GitHub Actions in Apache Spark repository.</p>

<p>Apache Spark repository has a workflow that enables you to run the same tests for a branch in your own forked repository that does not burden the resource from Apache Spark repository.</p>

<p>For example, suppose that you have a branch named &#8220;your_branch&#8221; for a pull request.
To run tests on &#8220;your_branch&#8221; and check test results:</p>

<ul>
<li>Click the &#8220;Actions&#8221; tab in your forked repository.</li>
<li>Select the &#8220;Build and test&#8221; workflow in the &#8220;All workflows&#8221; list.</li>
<li>Click the &#8220;Run workflow&#8221; button and enter &#8220;your_branch&#8221; in the &#8220;Target branch to run&#8221; field.</li>
<li>Once the &#8220;Build and test&#8221; workflow is finished, click the &#8220;Report test results&#8221; workflow to check test results.</li>
</ul>

<p><img src="/images/running-tests-using-github-actions.png" style="width: 100%; max-width: 800px;" /></p>

<p><a name="github-workflow-benchmarks"></a></p>
<h4>Running benchmarks in your forked repository</h4>

Expand Down
Binary file removed site/images/running-tests-using-github-actions.png
Binary file not shown.

0 comments on commit 991df19

Please sign in to comment.