Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new dist option 'loadgroup' #733

Merged
merged 4 commits into from
Nov 29, 2021

Conversation

baekdohyeop
Copy link
Contributor

@baekdohyeop baekdohyeop commented Nov 28, 2021

Checklist

  • Make sure to include reasonable tests for your change if necessary
  • Add change log

Intro

There are currently several options for distributing tests,
but there is still no suitable option for the following cases:

Case 1

In this case, it is efficient to divide all tests into different sessions.

@pytest.mark.parametrize('param', [A, B, C, D])
def test_something_heavy(param):
    do_something_heavy_test

Case 2

In this case, it is efficient to run all tests in the same session.

def test_something_light_1(heavy_fixture_cannot_filelock):
    do_something_light_test

def test_something_light_2(heavy_fixture_cannot_filelock):
    do_something_light_test

def test_something_light_3(heavy_fixture_cannot_filelock):
    do_something_light_test

Limit

If you use the loadscope option, all tests in Case 1 are performed in same session,
If the load option is used, all tests of Case 2 may be performed in different sessions.

Suggestion

Use the following xgroup mark and specify the name through the parameter.
Then, tests with the same name are executed in the same session.

@pytest.mark.xgroup(name="same_session")
def test_something_light_1(heavy_fixture_cannot_filelock):
    do_something_light_test

@pytest.mark.xgroup(name="same_session")
def test_something_light_2(heavy_fixture_cannot_filelock):
    do_something_light_test

@pytest.mark.xgroup(name="same_session")
def test_something_light_3(heavy_fixture_cannot_filelock):
    do_something_light_test

Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @baekdohyeop for following up with this!

Left a few comments, please take a look.

Besides, could you please add a description of the new option to the README? Probably showing some code would be helpful there too.

src/xdist/remote.py Outdated Show resolved Hide resolved
src/xdist/scheduler/loadgroup.py Outdated Show resolved Hide resolved
src/xdist/remote.py Outdated Show resolved Hide resolved
changelog/733.feature.rst Outdated Show resolved Hide resolved
src/xdist/plugin.py Outdated Show resolved Hide resolved
gname
"""
if nodeid.rfind("@") > nodeid.rfind("]"):
# check the index of ']' to avoid the case: parametrize mark value has '@'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

src/xdist/remote.py Outdated Show resolved Hide resolved
src/xdist/remote.py Outdated Show resolved Hide resolved
testing/acceptance_test.py Show resolved Hide resolved
nicoddemus and others added 4 commits November 29, 2021 02:59
This test started to fail in the 'py38-pytestmain' environment, the
cause being PytestRemovedIn7Warning being raised by the conftest file
of the test itself. Ignoring it is fine, the purpose of the test is to ensure
the hook is not called by pytest-xdist.
@baekdohyeop
Copy link
Contributor Author

@nicoddemus
I'm not good at English, so I'd appreciate it if you could improve it.
Thank you.

@nicoddemus
Copy link
Member

I'm not good at English, so I'd appreciate it if you could improve it.

Your English is very good, don't worry about it. 😁

I'm merging as is, and plan to give a quick overhaul of the README in a separate PR. Thanks!

@nicoddemus nicoddemus merged commit 630c1eb into pytest-dev:master Nov 29, 2021
@baekdohyeop
Copy link
Contributor Author

Thanks!

bors bot added a commit to microsoft/Qcodes that referenced this pull request Dec 11, 2021
3668: Bump hypothesis from 6.31.0 to 6.31.3 r=jenshnielsen a=dependabot[bot]

Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.31.0 to 6.31.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/HypothesisWorks/hypothesis/releases">hypothesis's releases</a>.</em></p>
<blockquote>
<h2>Hypothesis for Python - version 6.31.3</h2>
<p>This patch improves annotations on some of Hypothesis' internal
functions, in order to deobfuscate the signatures of some strategies.
In particular, strategies shared between hypothesis.extra.numpy and
the hypothesis.extra.array_api extra will benefit from this patch.</p>
<p><em><a href="https://hypothesis.readthedocs.io/en/latest/changes.html#v6-31-3">The canonical version of these notes (with links) is on readthedocs.</a></em></p>
<h2>Hypothesis for Python - version 6.31.2</h2>
<p>This patch fix invariants display in stateful falsifying examples
(issue <a href="https://github-redirect.dependabot.com/HypothesisWorks/hypothesis/issues/3185">#3185</a>).</p>
<p><em><a href="https://hypothesis.readthedocs.io/en/latest/changes.html#v6-31-2">The canonical version of these notes (with links) is on readthedocs.</a></em></p>
<h2>Hypothesis for Python - version 6.31.1</h2>
<p>This patch updates &quot;xps.indices()&quot; so no flat indices are generated,
i.e. generated indices will now always explicitly cover each axes of
an array if no ellipsis is present. This is to be consistent with a
specification change that dropped support for flat indexing (<a href="https://github-redirect.dependabot.com/HypothesisWorks/hypothesis/issues/272">#272</a>).</p>
<p><em><a href="https://hypothesis.readthedocs.io/en/latest/changes.html#v6-31-1">The canonical version of these notes (with links) is on readthedocs.</a></em></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/6173b2a8783fac5bab3fa478d3a5b0719f93b942"><code>6173b2a</code></a> Bump hypothesis-python version to 6.31.3 and update changelog</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/8bcb914b145eb074dc6f03c968f0c8fea9da03e7"><code>8bcb914</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/HypothesisWorks/hypothesis/issues/3179">#3179</a> from HypothesisWorks/proxies</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/0f5768c7dcd653fdc01d7466b94e8d1814f1a96c"><code>0f5768c</code></a> add annotations to <a href="https://github.com/proxies"><code>`@​proxies</code></a></li>`
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/d79427b2db3279a09451a07ecee4c6faad8dc494"><code>d79427b</code></a> Bump hypothesis-python version to 6.31.2 and update changelog</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/5d79bc0861a17de25c93d28c253a2ac627e3a569"><code>5d79bc0</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/HypothesisWorks/hypothesis/issues/3186">#3186</a> from touilleMan/issue-3185</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/ecced4c597ddfdf5a1781198d4e1384c69f55e05"><code>ecced4c</code></a> Don't print_blob here</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/e0d182c141797629831f316282184e7746baba21"><code>e0d182c</code></a> Inline invariant printing</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/af25acd13ae9fcfcf46a8f93f044cd4db1e1cefb"><code>af25acd</code></a> Bump hypothesis-python version to 6.31.1 and update changelog</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/9ec426731b3ee0940ee1db63052cac0ddcd6aa9c"><code>9ec4267</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/HypothesisWorks/hypothesis/issues/3184">#3184</a> from honno/xp-filter-flat-indices</li>
<li><a href="https://github.com/HypothesisWorks/hypothesis/commit/8e760caa7dda729b0ecf0f986d13d2803337d70f"><code>8e760ca</code></a> Skip coverage of flat indices logic in <code>BasicIndexStrategy</code></li>
<li>Additional commits viewable in <a href="https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.31.0...hypothesis-python-6.31.3">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=hypothesis&package-manager=pip&previous-version=6.31.0&new-version=6.31.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

3669: Bump pytest-xdist from 2.4.0 to 2.5.0 r=jenshnielsen a=dependabot[bot]

Bumps [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) from 2.4.0 to 2.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p>
<blockquote>
<h1>pytest-xdist 2.5.0 (2021-12-10)</h1>
<h2>Features</h2>
<ul>
<li>
<p><code>[#722](pytest-dev/pytest-xdist#722) &lt;https://github.com/pytest-dev/pytest-xdist/issues/722&gt;</code>_: Full compatibility with pytest 7 - no deprecation warnings or use of legacy features.</p>
</li>
<li>
<p><code>[#733](pytest-dev/pytest-xdist#733) &lt;https://github.com/pytest-dev/pytest-xdist/issues/733&gt;</code>_: New <code>--dist=loadgroup</code> option, which ensures all tests marked with <code>`@pytest.mark.xdist_group</code>` run in the same session/worker. Other tests run distributed as in <code>--dist=load</code>.</p>
</li>
</ul>
<h2>Trivial Changes</h2>
<ul>
<li>
<p><code>[#708](pytest-dev/pytest-xdist#708) &lt;https://github.com/pytest-dev/pytest-xdist/issues/708&gt;</code>_: Use <code>`@pytest.hookspec</code>` decorator to declare hook options in <code>newhooks.py</code> to avoid warnings in <code>pytest 7.0</code>.</p>
</li>
<li>
<p><code>[#719](pytest-dev/pytest-xdist#719) &lt;https://github.com/pytest-dev/pytest-xdist/issues/719&gt;</code>_: Use up-to-date <code>setup.cfg</code>/<code>pyproject.toml</code> packaging setup.</p>
</li>
<li>
<p><code>[#720](pytest-dev/pytest-xdist#720) &lt;https://github.com/pytest-dev/pytest-xdist/issues/720&gt;</code>_: Require pytest&gt;=6.2.0.</p>
</li>
<li>
<p><code>[#721](pytest-dev/pytest-xdist#721) &lt;https://github.com/pytest-dev/pytest-xdist/issues/721&gt;</code>_: Started using type annotations and mypy checking internally. The types are incomplete and not published.</p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/13f39349c6950a881c1fe4fcd5984af2e8b7c220"><code>13f3934</code></a> Remove unnecessary skip from test_logfinish_hook as we require pytest&gt;=6.2</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c76d5622f17135e892965d742377870eb9b07933"><code>c76d562</code></a> Skip test_warning_captured_deprecated_in_pytest_6 in pytest&gt;=7.1</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/5f78c7155e66ab73bdc7631c4ac6bfe684b82500"><code>5f78c71</code></a> Fix CHANGELOG header</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c8bbc03e49d5a53b5da808c7328e8f3ad6ed2d7e"><code>c8bbc03</code></a> Release 2.5.0</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/8dbf3677dc7cc26ada33cf8a27d7ac51a9be467b"><code>8dbf367</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/738">#738</a> from pytest-dev/pre-commit-ci-update-config</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/a25c14bef59ad728e39cabc64f71190aaad73b0a"><code>a25c14b</code></a> [pre-commit.ci] pre-commit autoupdate</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/110c114025202d11570737be823de158d1bb8d99"><code>110c114</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/734">#734</a> from nicoddemus/revamp-readme</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/83bdbf4b95c914a889d1faa8fba8d506bcc2f8c7"><code>83bdbf4</code></a> Revamp README</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/630c1eb6f2c31dcb4c38c75bb62f868237cdde94"><code>630c1eb</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/733">#733</a> from baekdohyeop/feature-loadgroup</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/62e50d00977b41e175b5f119381f9db760459ddc"><code>62e50d0</code></a> Address review</li>
<li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v2.4.0...v2.5.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-xdist&package-manager=pip&previous-version=2.4.0&new-version=2.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants