docker/env/hydra.sh crashes on Python 3.10

[nodep]

Issue description

• This issue is a regression.
• It is unknown if this issue is a regression.

Steps to Reproduce

Running ./docker/env/hydra.sh list-images -c aws on a system with python 3.10 installed crashes with:

Traceback (most recent call last):
  File "/home/ferenc/src/scylla-cluster-tests/sdcm/utils/get_username.py", line 15, in <module>
    import getpass
  File "/usr/lib/python3.10/getpass.py", line 17, in <module>
    import contextlib
  File "/usr/lib/python3.10/contextlib.py", line 5, in <module>
    from collections import deque
  File "/usr/lib/python3.10/collections/__init__.py", line 36, in <module>
    from operator import eq as _eq
ImportError: cannot import name 'eq' from 'operator' (/home/ferenc/src/scylla-cluster-tests/sdcm/utils/operator/__init__.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 62, in apport_excepthook
    if not enabled():
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 24, in enabled
    import re
  File "/usr/lib/python3.10/re.py", line 127, in <module>
    import functools
  File "/usr/lib/python3.10/functools.py", line 18, in <module>
    from collections import namedtuple
  File "/usr/lib/python3.10/collections/__init__.py", line 36, in <module>
    from operator import eq as _eq
ImportError: cannot import name 'eq' from 'operator' (/home/ferenc/src/scylla-cluster-tests/sdcm/utils/operator/__init__.py)

Original exception was:
Traceback (most recent call last):
  File "/home/ferenc/src/scylla-cluster-tests/sdcm/utils/get_username.py", line 15, in <module>
    import getpass
  File "/usr/lib/python3.10/getpass.py", line 17, in <module>
    import contextlib
  File "/usr/lib/python3.10/contextlib.py", line 5, in <module>
    from collections import deque
  File "/usr/lib/python3.10/collections/__init__.py", line 36, in <module>
    from operator import eq as _eq
ImportError: cannot import name 'eq' from 'operator' (/home/ferenc/src/scylla-cluster-tests/sdcm/utils/operator/__init__.py)

How frequently does it reproduce?

This can be reproduced on every run when python 3.10 is installed

Installation details

$ python3 --version
Python 3.10.12

[vponomaryov]

I cannot reproduce it:

$ ./docker/env/hydra.sh list-images -c aws
There is scylladb/hydra:v1.72-switch-to-uv-3 in local cache, using it.
Going to run './sct.py  list-images -c aws'...
%path%/.local/lib/python3.10/site-packages/paramiko/pkey.py:82: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "cipher": algorithms.TripleDES,
%path%/.local/lib/python3.10/site-packages/paramiko/transport.py:256: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "class": algorithms.TripleDES,
logged in as arn:aws:iam::797456418907:user/qa
New directory created: %path%/sct-results/20240729-141516-222656-list-images
Looking for AMIs match [master:latest]
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                         AMI Machine Images for master:latest in region eu-west-1                                                                         |
+---------+---------------------------------------------+-----------------------+--------------------------+---------------------------------------------+---------+--------+---------------+--------------+
| Backend | Name                                        | ImageId               | CreationDate             | NameTag                                     | BuildId | Arch   | ScyllaVersion | OwnerId      |
+---------+---------------------------------------------+-----------------------+--------------------------+---------------------------------------------+---------+--------+---------------+--------------+
| AWS     | scylla-6.2.0-dev-x86_64-2024-07-29T01-58-56 | ami-02e3517b5fa9f4ebf | 2024-07-28T23:09:03.000Z | scylla-6.2.0-dev-x86_64-2024-07-29T01-58-56 | N\A     | x86_64 | None          | 158855661827 |
+---------+---------------------------------------------+-----------------------+--------------------------+---------------------------------------------+---------+--------+---------------+--------------+

$ python3 --version
Python 3.10.12

[nodep]

This seems to be one of those dreaded "It happens on my machine".
Do you have any ideas what I could check? For me, it fails every time, unless I comment out the line in docker/env/hydra.sh :

RUN_BY_USER=$(python3 "${SCT_DIR}/sdcm/utils/get_username.py")

Maybe it is not the python version (clearly it is not just that, since it works on your system).

[vponomaryov]

@nodep

The error is really strange:

ImportError: cannot import name 'eq' from 'operator' (/home/ferenc/src/scylla-cluster-tests/sdcm/utils/operator/__init__.py)

The sdcm/utils/operator/__init__.py module is totally unrelated to the code that must be called.

So, check your paths and such stuff.
I think that the python modules lookup order is wrong.

The /usr/lib/python3.10/operator.py module should have been picked up for the operator name.

[fruch]

@nodep

The error is really strange:

ImportError: cannot import name 'eq' from 'operator' (/home/ferenc/src/scylla-cluster-tests/sdcm/utils/operator/__init__.py)

The sdcm/utils/operator/__init__.py module is totally unrelated to the code that must be called.

So, check your paths and such stuff. I think that the python modules lookup order is wrong.

The /usr/lib/python3.10/operator.py module should have been picked up for the operator name.

Python put the current directory of the executing script first on the path, later versions had some was to prevent that

Anyhow, we need to remove that call it's pointless

[vponomaryov]

@nodep

have you solved the problem you had?
Is there anything else that can be done in scope of this issue?

[fruch]

@nodep

have you solved the problem you had? Is there anything else that can be done in scope of this issue?

commenting that part solved the issue, and we merge the PR that removed that offending code.

@nodep would be nice if you can confirm that SCT master doesn't generente you this error anymore.

[fruch]

@nodep I'm guessing it's working for now

closing this one

[nodep]

@nodep
have you solved the problem you had? Is there anything else that can be done in scope of this issue?

commenting that part solved the issue, and we merge the PR that removed that offending code.

@nodep would be nice if you can confirm that SCT master doesn't generente you this error anymore.

Sorry about the late reply, but yes, the latest version on master does not generate the error any more.
Thank you!