Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python 3.13.0-rc.1 to CI in prep for real 3.13 release #3600

Closed
wants to merge 8 commits into from

Conversation

benclifford
Copy link
Collaborator

@benclifford benclifford commented Aug 26, 2024

This usually flushes out various incompatibilities in both parsl and dependencies - which I'll hack out of the tests in order to find the next set of breakage.

Type of change

  • New feature

@benclifford
Copy link
Collaborator Author

This fails on greenlet. There is some discussion about getting greenlet working on 3.12 here: python-greenlet/greenlet#396

greenlet is needed for sqlalchemy which is needed for parsl[monitoring], so this dependency isn't blocking no-monitoring parsl.

@benclifford benclifford marked this pull request as draft August 26, 2024 13:18
@benclifford
Copy link
Collaborator Author

@AymenFJA and @andre-merzky - radical pilot is failing import with Python 3.13 release candidate 1, in our CI. I think this is a general radical pilot problem, not specific to the parsl/radical integration.

parsl/tests/conftest.py:184: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
parsl/tests/configs/local_radical.py:4: in <module>
    from parsl.executors.radical import RadicalPilotExecutor, ResourceConfig
parsl/executors/radical/__init__.py:1: in <module>
    from parsl.executors.radical.executor import RadicalPilotExecutor
parsl/executors/radical/executor.py:27: in <module>
    from .rpex_resources import CLIENT, MPI, ResourceConfig
parsl/executors/radical/rpex_resources.py:7: in <module>
    import radical.pilot as rp
.venv/lib/python3.13/site-packages/radical/pilot/__init__.py:12: in <module>
    import radical.utils as _ru
.venv/lib/python3.13/site-packages/radical/utils/__init__.py:19: in <module>
    from .atfork         import *
.venv/lib/python3.13/site-packages/radical/utils/atfork/__init__.py:24: in <module>
    fix_logging_module()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def fix_logging_module():
    
        # monkeypatching can be disabled by setting RADICAL_UTILS_NO_ATFORK
        if 'RADICAL_UTILS_NO_ATFORK' in os.environ:
            return
    
        logging = sys.modules.get('logging')
        # Prevent fixing multiple times as that would cause a deadlock.
        if logging and getattr(logging, 'fixed_for_atfork', None):
            return
    
        if logging:
          # if os.environ.get('RADICAL_DEBUG'):
          #     warnings.warn('logging module already imported before fixup.')
            pass
    
        import logging
        if logging.getLogger().handlers:
            # We could register each lock with atfork for these handlers but if
            # these exist, other loggers or not yet added handlers could as well.
            # Its safer to insist that this fix is applied before logging has been
            # configured.
            _warn('Import `radical` modules before `logging` to avoid the '
                  'application to deadlock on `fork()`!')
    
>       logging._acquireLock()
E       AttributeError: module 'logging' has no attribute '_acquireLock'

This method disappeared in python/cpython@74723e1

cc parsl issue #2343 which is a general tracker for this kind of fork related issue.

@andre-merzky
Copy link

Thanks @benclifford , I'll have a look at this ASAP.

@andre-merzky
Copy link

Yes, some monkeypatch we do on logging is not needed anymore, thanks for the ping! This will be fixed in the next release. How urgent is that for you @benclifford?

@benclifford
Copy link
Collaborator Author

How urgent is that for you @benclifford?

Ideally, Parsl would work with Python 3.13 on release day, 2024-10-01, https://peps.python.org/pep-0719/ but that is not a hard requirement, and right now greenlet (see earlier in this PR) is a more serious problem.

@benclifford
Copy link
Collaborator Author

re greenlet, python-greenlet/greenlet#396 was merged a few hours ago, so next step there would be watching it propagate into releases

@benclifford
Copy link
Collaborator Author

superceded by #3646

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants