Packaging for openSUSE: checks failing due to "invalid syntax", "ImportError"

[kastl-ars]

Hi,

as mashumaro is a requirement for flux-local, I started packaging it for openSUSE.

The package (in version 3.13.1) seems to work, but the checks are failing. The package is built for python 3.10, 3.11 and 3.12, the checks seems to fail on all three versions.

This is the full output for python3.10:

[    5s] + pytest-3.10 --ignore=_build.python310 --ignore=_build.python311 -v
[    5s] ============================= test session starts ==============================
[    5s] platform linux -- Python 3.10.15, pytest-8.3.2, pluggy-1.5.0 -- /usr/bin/python3.10
[    5s] cachedir: .pytest_cache
[    5s] rootdir: /home/abuild/rpmbuild/BUILD/mashumaro-3.13.1
[    5s] configfile: pyproject.toml
[    6s] collecting ... collected 174 items / 13 errors
[    6s] 
[    6s] ==================================== ERRORS ====================================
[    6s] ____________________ ERROR collecting tests/test_config.py _____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_config.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_config.py:12: in <module>
[    6s]     from .entities import (
[    6s] E   ImportError: attempted relative import with no known parent package
[    6s] __________________ ERROR collecting tests/test_data_types.py ___________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_data_types.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_data_types.py:63: in <module>
[    6s]     from tests.entities import MyUntypedNamedTupleWithDefaults, TDefaultInt
[    6s] E   ModuleNotFoundError: No module named 'tests'
[    6s] ____________________ ERROR collecting tests/test_dialect.py ____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_dialect.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_dialect.py:29: in <module>
[    6s]     from .conftest import add_unpack_method
[    6s] E   ImportError: attempted relative import with no known parent package
[    6s] ___________________ ERROR collecting tests/test_generics.py ____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_generics.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_generics.py:7: in <module>
[    6s]     from tests.entities import MyGenericDataClass, SerializableTypeGenericList
[    6s] E   ModuleNotFoundError: No module named 'tests'
[    6s] _______________ ERROR collecting tests/test_generics_pep_695.py ________________
[    6s] /usr/lib/python3.10/site-packages/_pytest/python.py:493: in importtestmodule
[    6s]     mod = import_path(
[    6s] /usr/lib/python3.10/site-packages/_pytest/pathlib.py:582: in import_path
[    6s]     importlib.import_module(module_name)
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] <frozen importlib._bootstrap>:1050: in _gcd_import
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:1027: in _find_and_load
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:688: in _load_unlocked
[    6s]     ???
[    6s] /usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:165: in exec_module
[    6s]     source_stat, co = _rewrite_test(fn, self.config)
[    6s] /usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:345: in _rewrite_test
[    6s]     tree = ast.parse(source, filename=strfn)
[    6s] /usr/lib64/python3.10/ast.py:50: in parse
[    6s]     return compile(source, filename, mode, flags,
[    6s] E     File "/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_generics_pep_695.py", line 11
[    6s] E       class Foo[T](DataClassJSONMixin):
[    6s] E                ^
[    6s] E   SyntaxError: invalid syntax
[    6s] _____________________ ERROR collecting tests/test_json.py ______________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_json.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_json.py:11: in <module>
[    6s]     from .entities import MyEnum
[    6s] E   ImportError: attempted relative import with no known parent package
[    6s] ____________________ ERROR collecting tests/test_literal.py ____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_literal.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_literal.py:11: in <module>
[    6s]     from tests.entities import MyEnum
[    6s] E   ModuleNotFoundError: No module named 'tests'
[    6s] _____________________ ERROR collecting tests/test_meta.py ______________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_meta.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_meta.py:65: in <module>
[    6s]     from .entities import (
[    6s] E   ImportError: attempted relative import with no known parent package
[    6s] _______________ ERROR collecting tests/test_metadata_options.py ________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_metadata_options.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_metadata_options.py:6: in <module>
[    6s]     import ciso8601
[    6s] E   ModuleNotFoundError: No module named 'ciso8601'
[    6s] ____________________ ERROR collecting tests/test_pep_563.py ____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_pep_563.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_pep_563.py:17: in <module>
[    6s]     from .conftest import add_unpack_method
[    6s] E   ImportError: attempted relative import with no known parent package
[    6s] ____________________ ERROR collecting tests/test_pep_695.py ____________________
[    6s] /usr/lib/python3.10/site-packages/_pytest/python.py:493: in importtestmodule
[    6s]     mod = import_path(
[    6s] /usr/lib/python3.10/site-packages/_pytest/pathlib.py:582: in import_path
[    6s]     importlib.import_module(module_name)
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] <frozen importlib._bootstrap>:1050: in _gcd_import
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:1027: in _find_and_load
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
[    6s]     ???
[    6s] <frozen importlib._bootstrap>:688: in _load_unlocked
[    6s]     ???
[    6s] /usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:165: in exec_module
[    6s]     source_stat, co = _rewrite_test(fn, self.config)
[    6s] /usr/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:345: in _rewrite_test
[    6s]     tree = ast.parse(source, filename=strfn)
[    6s] /usr/lib64/python3.10/ast.py:50: in parse
[    6s]     return compile(source, filename, mode, flags,
[    6s] E     File "/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_pep_695.py", line 9
[    6s] E       type MyDate = date
[    6s] E            ^^^^^^
[    6s] E   SyntaxError: invalid syntax
[    6s] _____________________ ERROR collecting tests/test_types.py _____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_types.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_types.py:12: in <module>
[    6s]     from tests.entities import GenericSerializableList, GenericSerializableWrapper
[    6s] E   ModuleNotFoundError: No module named 'tests'
[    6s] _____________________ ERROR collecting tests/test_union.py _____________________
[    6s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_union.py'.
[    6s] Hint: make sure your test modules/packages have valid Python names.
[    6s] Traceback:
[    6s] /usr/lib64/python3.10/importlib/__init__.py:126: in import_module
[    6s]     return _bootstrap._gcd_import(name[level:], package, level)
[    6s] tests/test_union.py:9: in <module>
[    6s]     from tests.utils import same_types
[    6s] E   ModuleNotFoundError: No module named 'tests'
[    6s] =========================== short test summary info ============================
[    6s] ERROR tests/test_config.py
[    6s] ERROR tests/test_data_types.py
[    6s] ERROR tests/test_dialect.py
[    6s] ERROR tests/test_generics.py
[    6s] ERROR tests/test_generics_pep_695.py
[    6s] ERROR tests/test_json.py
[    6s] ERROR tests/test_literal.py
[    6s] ERROR tests/test_meta.py
[    6s] ERROR tests/test_metadata_options.py
[    6s] ERROR tests/test_pep_563.py
[    6s] ERROR tests/test_pep_695.py
[    6s] ERROR tests/test_types.py
[    6s] ERROR tests/test_union.py
[    6s] !!!!!!!!!!!!!!!!!!! Interrupted: 13 errors during collection !!!!!!!!!!!!!!!!!!!
[    6s] ============================== 13 errors in 0.66s ==============================

One of the errors seems to be from a missing dependency (ciso8601), but the others are looking bad.

Any idea what could cause them? Am I missing a dependency?

Kind Regards,
Johannes

[Fatal1ty]

Hi @kastl-ars

I'm not familiar with packaging pipeline for openSUSE but it seems like the problem with discovering tests. Normally, you need to prepare a virtual environment in order to run the tests. Let me explain why you shouldn't have these exceptions.

[    6s] E       class Foo[T](DataClassJSONMixin):
[    6s] E                ^
[    6s] E   SyntaxError: invalid syntax

Import from test_generics_pep_695.py shouldn't have happened on Python 3.10 because this module is added to collect_ignore:

• mashumaro/tests/conftest.py

  Lines 5 to 9 in cdee557

  	if not PY_312_MIN:
  	collect_ignore = [
  	"test_generics_pep_695.py",
  	"test_pep_695.py",
  	]

[    6s]     import ciso8601
[    6s] E   ModuleNotFoundError: No module named 'ciso8601'

This ModuleNotFoundError shouldn't have happened if you had prepared the virtual environment with all the dependencies.

[    6s]     from tests.entities import MyEnum
[    6s] E   ModuleNotFoundError: No module named 'tests'

This shouldn't have happened because tests package is in a local directory and Python should be able to import it.

I don't know what could be causing the problem you are experiencing, but to diagnose it I would look into:

• pytest version
• missing environment variables or arguments to pytest

[kastl-ars]

Thanks for your reply. The error regarding ciso8601 was a missing dependency.

As for the other errors, I'll look into the exact pytest calls and see if I can find out, what goes wrong.

[kastl-ars]

I think it boils down to this error, that I could not solve this far. I'll ask the openSUSE Python packaging experts for advice.

[   20s] ____________________ ERROR collecting tests/test_config.py _____________________
[   20s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/mashumaro-3.13.1/tests/test_config.py'.
[   20s] Hint: make sure your test modules/packages have valid Python names.
[   20s] Traceback:
[   20s] /usr/lib64/python3.11/importlib/__init__.py:126: in import_module
[   20s]     return _bootstrap._gcd_import(name[level:], package, level)
[   20s] tests/test_config.py:12: in <module>
[   20s]     from .entities import (
[   20s] E   ImportError: attempted relative import with no known parent package

All of the No module named 'tests' errors seems to be a fallout of this error.

[kastl-ars]

After asking for help on the openSUSE python mailinglist, @danigm found the root cause. The PyPI tarball is apparently missing the tests/utils.py file.

Using the tarball from the GitHub release page works, with Python 3.10/3.11/3.12 on openSUSE Tumbleweed.

Thanks a lot for your help! Have a nice day.

Kind Regards,
Johannes

[Fatal1ty]

@kastl-ars Hi!

Good news, I fixed the broken "tests" folder inclusion in the tarball by adding MANIFEST.in file here:

• Include complete tests in sdist #262

It turned out that the folder with the tests was added by default using the following pattern from the Creating a Source Distribution section of setuptools documentation:

Files that match the following glob patterns: tests/test*.py, test/test*.py;

[johanneskastl]

Thanks for fixing this!