Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unpin pytest #1409

Merged
merged 2 commits into from
Sep 17, 2024
Merged

unpin pytest #1409

merged 2 commits into from
Sep 17, 2024

Conversation

braingram
Copy link
Collaborator

@braingram braingram commented Sep 11, 2024

This PR removes the upper pin for pytest and updates the lower pin.

It's unclear why #1085 added the upper pin.

This caused issues during testing with stasis as some packages use pytest > 8.0.

Regression tests all passed: https://github.com/spacetelescope/RegressionTests/actions/runs/10814310251

Checklist

  • for a public change, added a towncrier news fragment in changes/

    echo "changed something" > changes/<PR#>.<changetype>.rst

    • changes/<PR#>.general.rst: infrastructure or miscellaneous change
    • changes/<PR#>.docs.rst
    • changes/<PR#>.stpipe.rst
    • changes/<PR#>.associations.rst
    • changes/<PR#>.scripts.rst
    • changes/<PR#>.mosaic_pipeline.rst
    • changes/<PR#>.patch_match.rst

    steps

    • changes/<PR#>.dq_init.rst
    • changes/<PR#>.saturation.rst
    • changes/<PR#>.refpix.rst
    • changes/<PR#>.linearity.rst
    • changes/<PR#>.dark_current.rst
    • changes/<PR#>.jump_detection.rst
    • changes/<PR#>.ramp_fitting.rst
    • changes/<PR#>.assign_wcs.rst
    • changes/<PR#>.flatfield.rst
    • changes/<PR#>.photom.rst
    • changes/<PR#>.flux.rst
    • changes/<PR#>.source_detection.rst
    • changes/<PR#>.tweakreg.rst
    • changes/<PR#>.skymatch.rst
    • changes/<PR#>.outlier_detection.rst
    • changes/<PR#>.resample.rst
    • changes/<PR#>.source_catalog.rst
  • updated relevant tests

  • updated relevant documentation

  • updated relevant milestone(s)

  • added relevant label(s)

  • ran regression tests, post a link to the Jenkins job below. How to run regression tests on a PR

@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Sep 11, 2024
Copy link

codecov bot commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.52%. Comparing base (9fa24f2) to head (1040e46).
Report is 278 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1409   +/-   ##
=======================================
  Coverage   78.52%   78.52%           
=======================================
  Files         117      117           
  Lines        7833     7833           
=======================================
  Hits         6151     6151           
  Misses       1682     1682           
Flag Coverage Δ *Carryforward flag
nightly 62.24% <ø> (ø) Carriedforward from 9fa24f2

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@braingram braingram marked this pull request as ready for review September 11, 2024 15:00
@braingram braingram requested a review from a team as a code owner September 11, 2024 15:00
@ddavis-stsci
Copy link
Collaborator

This fails out of the box so I think we need more work

pip list | grep "pytest "
177:pytest                        8.3.3
(rcal_dev) bash>pytest romancal/
/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pytest_doctestplus/plugin.py:559: PytestRemovedIn9Warning: The (path: py.path.local) argument is deprecated, please use (file_path: pathlib.Path)
see https://docs.pytest.org/en/latest/deprecations.html#py-path-local-arguments-for-hooks-replaced-with-pathlib-path
  def pytest_collect_file(self, path, parent):
/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pytest_doctestplus/plugin.py:505: PytestRemovedIn9Warning: The (path: py.path.local) argument is deprecated, please use (collection_path: pathlib.Path)
see https://docs.pytest.org/en/latest/deprecations.html#py-path-local-arguments-for-hooks-replaced-with-pathlib-path
  def pytest_ignore_collect(self, path, config):
=============================================================================== test session starts ================================================================================
platform darwin -- Python 3.11.0, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/ddavis/src/Roman/romancal
configfile: pyproject.toml
plugins: remotedata-0.4.0, asdf-3.3.0, hypothesis-6.82.0, romancal-0.16.4.dev4+g908f0f9.d20240910, doctestplus-0.13.0, cov-4.1.0, filter-subpackage-0.1.2, mock-3.11.1, astropy-header-0.2.2, anyio-3.7.1, astropy-0.10.0, ci-watson-0.6.1, openfiles-0.5.0, env-0.8.2, arraydiff-0.5.0
collected 0 items / 1 error

====================================================================================== ERRORS ======================================================================================
________________________________________________________________________________ ERROR collecting . ________________________________________________________________________________
../../../miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pluggy/_hooks.py:513: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
../../../miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
../../../miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pytest_doctestplus/plugin.py:512: in pytest_ignore_collect
    collect_ignore = config._getconftest_pathlist("collect_ignore",
E   TypeError: Config._getconftest_pathlist() got an unexpected keyword argument 'rootpath'
============================================================================= short test summary info ==============================================================================
ERROR . - TypeError: Config._getconftest_pathlist() got an unexpected keyword argument 'rootpath'
================================================================================= 1 error in 0.09s

@braingram
Copy link
Collaborator Author

braingram commented Sep 11, 2024

It looks like your doctestplus is out of date. The CI passes (and my tests do locally) since they're using a newer doctestplus.
https://github.com/spacetelescope/romancal/actions/runs/10813753786/job/29998700580?pr=1409#step:10:374

The test deps for romancal don't all have lower pins and doctestplus is not a direct dependency (it's a dependency of pytest-astropy). I added a lower pin for pytest-astropy. Would you try it again?

@ddavis-stsci
Copy link
Collaborator

That fails even more spectacularly,

pip install -e ".[dev]"
Obtaining file:///Users/ddavis/src/Roman/romancal
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
    req = self._make_requirement_from_install_req(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 185, in _make_candidate_from_link
    self._editable_candidate_cache[link] = EditableCandidate(
                                           ^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 318, in __init__
    super().__init__(
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 328, in _prepare_distribution
    return self._factory.preparer.prepare_editable_requirement(self._ireq)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 687, in prepare_editable_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 31, in prepare_distribution_metadata
    self.req.load_pyproject_toml()
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 484, in load_pyproject_toml
    pyproject_toml_data = load_pyproject_toml(
                          ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_internal/pyproject.py", line 64, in load_pyproject_toml
    pp_toml = tomli.loads(f.read())
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_vendor/tomli/_parser.py", line 102, in loads
    pos = key_value_rule(src, pos, out, header, parse_float)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_vendor/tomli/_parser.py", line 326, in key_value_rule
    pos, key, value = parse_key_value_pair(src, pos, parse_float)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_vendor/tomli/_parser.py", line 369, in parse_key_value_pair
    pos, value = parse_value(src, pos, parse_float)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_vendor/tomli/_parser.py", line 616, in parse_value
    return parse_array(src, pos, parse_float)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ddavis/miniconda3/envs/rcal_dev/lib/python3.11/site-packages/pip/_vendor/tomli/_parser.py", line 428, in parse_array
    raise suffixed_err(src, pos, "Unclosed array")
pip._vendor.tomli.TOMLDecodeError: Unclosed array (at line 63, column 5)

@braingram
Copy link
Collaborator Author

I have no idea what is wrong with your environment but that's not what I see locally or in the CI. Since it's passing for me and for the CI would you try with a fresh environment as yours appears to be broken?

Copy link
Collaborator

@mairanteodoro mairanteodoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to apply the changes in this PR and then install romancal successfully on my machine. I did not run into the issues reported by @ddavis-stsci.

Looks good to me!
Thanks, @braingram!

@braingram
Copy link
Collaborator Author

Thanks! @ddavis-stsci let me know if you continue to run into issues and I'd be happy to help sort them out.

@braingram braingram merged commit cef755e into spacetelescope:main Sep 17, 2024
30 checks passed
@braingram braingram deleted the unpin_pytest branch September 17, 2024 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file no-changelog-entry-needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants