Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tox-docker configuration parsing broken since tox 4.0.13 #2821

Closed
pdecat opened this issue Jan 4, 2023 · 2 comments · Fixed by #2822
Closed

tox-docker configuration parsing broken since tox 4.0.13 #2821

pdecat opened this issue Jan 4, 2023 · 2 comments · Fixed by #2822
Labels
bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.

Comments

@pdecat
Copy link
Contributor

pdecat commented Jan 4, 2023

Issue

tox-docker plugin configuration parsing of ports is broken since tox 4.0.13.

ports configuration option is documented as:

A multi-line list of port mapping specifications, as HOST_PORT:CONTAINER_PORT/PROTO, ...

With 4.0.12, the host port is properly set on the started container and forwarded to the container port.

But with tox 4.0.13, host port of the started container is random.
Given the short changelog for that release, I suspect it to be caused by #2744
Same issue occurs with 4.0.15, 4.0.19, and 4.2.2.

Environment

Provide at least:

  • OS: Ubuntu 22.10
  • pip list of the host Python where tox is installed:
# poetry run pip list
Package            Version     Editable project location
------------------ ----------- -------------------------------------------------------------------------------------
attrs              21.4.0
black              22.3.0
boto3              1.23.6
botocore           1.26.6
cachetools         5.2.0
certifi            2022.5.18.1
cfgv               3.3.1
chardet            5.1.0
charset-normalizer 2.0.12
click              8.1.3
colorama           0.4.6
distlib            0.3.6
docker             5.0.3
filelock           3.9.0
flake8             4.0.1
freezegun          1.2.1
identify           2.5.1
idna               3.3
iniconfig          1.1.1
jmespath           1.0.0
mccabe             0.6.1
mock               4.0.3
mypy-extensions    0.4.3
nodeenv            1.6.0
packaging          22.0
pathspec           0.9.0
pip                22.3.1
platformdirs       2.6.2
pluggy             1.0.0
pre-commit         2.19.0
py                 1.11.0
pycodestyle        2.8.0
pyflakes           2.4.0
pyparsing          3.0.9
pyproject_api      1.4.0
pytest             6.2.5
pytest-mock        3.7.0
python-dateutil    2.8.2
PyYAML             6.0
requests           2.27.1
s3transfer         0.5.2
setuptools         65.6.3
six                1.16.0
toml               0.10.2
tomli              2.0.1
tox                4.2.2
tox-docker         4.0.0a2
urllib3            1.26.9
virtualenv         20.17.1
websocket-client   1.3.2
wheel              0.38.4

Output of running tox

Provide the output of tox -rvv:

tox -rvv
/home/patrick/workspaces/myproject/.venv/lib/python3.10/site-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (5.1.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
py39: 298 W remove tox env folder /home/patrick/workspaces/myproject/.tox/py39 [tox/tox_env/api.py:321]
py39: 308 I find interpreter for spec PythonSpec(major=3, minor=9) [virtualenv/discovery/builtin.py:56]
py39: 308 D discover exe for PythonInfo(spec=CPython3.10.6.final.0-64, exe=/home/patrick/workspaces/myproject/.venv/bin/python, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:437]
py39: 308 D filesystem is case-sensitive [virtualenv/info.py:24]
py39: 310 D got python info of /usr/bin/python3.10 from /home/patrick/.local/share/virtualenv/py_info/1/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 310 I proposed PythonInfo(spec=CPython3.10.6.final.0-64, system=/usr/bin/python3.10, exe=/home/patrick/workspaces/myproject/.venv/bin/python, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
py39: 310 D discover PATH[0]=/home/patrick/workspaces/myproject/.venv/bin [virtualenv/discovery/builtin.py:108]
py39: 311 D got python info of /home/patrick/workspaces/myproject/.venv/bin/python3 from /home/patrick/.local/share/virtualenv/py_info/1/f40e8b45bd8922e09aa5a9356903620c1975d00691ae8ec9747c29426f7f443d.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 311 D discover exe from cache /usr - exact False: PythonInfo({'architecture': 64, 'base_exec_prefix': '/usr', 'base_prefix': '/usr', 'distutils_install': {}, 'exec_prefix': '/usr', 'executable': '/home/patrick/workspaces/myproject/.venv/bin/python', 'file_system_encoding': 'utf-8', 'has_venv': True, 'implementation': 'CPython', 'max_size': 9223372036854775807, 'original_executable': '/usr/bin/python3.10', 'os': 'posix', 'path': ['/home/patrick/.local/pipx/venvs/tox/lib/python3.10/site-packages/virtualenv/discovery', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/patrick/.local/lib/python3.10/site-packages', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages'], 'platform': 'linux', 'prefix': '/usr', 'real_prefix': None, 'stdout_encoding': 'utf-8', 'sysconfig': {'makefile_filename': '/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile'}, 'sysconfig_paths': {'data': '{base}', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages', 'platstdlib': '{platbase}/{platlibdir}/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'scripts': '{base}/bin', 'stdlib': '{installed_base}/{platlibdir}/python{py_version_short}'}, 'sysconfig_scheme': 'posix_prefix', 'sysconfig_vars': {'PYTHONFRAMEWORK': '', 'abiflags': '', 'base': '/usr', 'installed_base': '/usr', 'platbase': '/usr', 'platlibdir': 'lib', 'py_version_short': '3.10'}, 'system_executable': '/usr/bin/python3.10', 'system_stdlib': '/usr/lib/python3.10', 'system_stdlib_platform': '/usr/lib/python3.10', 'version': '3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', 'version_info': VersionInfo(major=3, minor=10, micro=6, releaselevel='final', serial=0), 'version_nodot': '310'}) [virtualenv/discovery/py_info.py:435]
py39: 311 D discover exe from cache /usr - exact False: PythonInfo({'architecture': 64, 'base_exec_prefix': '/usr', 'base_prefix': '/usr', 'distutils_install': {}, 'exec_prefix': '/usr', 'executable': '/home/patrick/workspaces/myproject/.venv/bin/python3', 'file_system_encoding': 'utf-8', 'has_venv': True, 'implementation': 'CPython', 'max_size': 9223372036854775807, 'original_executable': '/usr/bin/python3.10', 'os': 'posix', 'path': ['/home/patrick/.local/pipx/venvs/tox/lib/python3.10/site-packages/virtualenv/discovery', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/patrick/.local/lib/python3.10/site-packages', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages'], 'platform': 'linux', 'prefix': '/usr', 'real_prefix': None, 'stdout_encoding': 'utf-8', 'sysconfig': {'makefile_filename': '/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile'}, 'sysconfig_paths': {'data': '{base}', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages', 'platstdlib': '{platbase}/{platlibdir}/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'scripts': '{base}/bin', 'stdlib': '{installed_base}/{platlibdir}/python{py_version_short}'}, 'sysconfig_scheme': 'posix_prefix', 'sysconfig_vars': {'PYTHONFRAMEWORK': '', 'abiflags': '', 'base': '/usr', 'installed_base': '/usr', 'platbase': '/usr', 'platlibdir': 'lib', 'py_version_short': '3.10'}, 'system_executable': '/usr/bin/python3.10', 'system_stdlib': '/usr/lib/python3.10', 'system_stdlib_platform': '/usr/lib/python3.10', 'version': '3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', 'version_info': VersionInfo(major=3, minor=10, micro=6, releaselevel='final', serial=0), 'version_nodot': '310'}) [virtualenv/discovery/py_info.py:435]

Way too verbose to post completely.

Minimal example

If possible, provide a minimal reproducer for the issue:

With this tox.ini:

[docker:dynamoDB]
image = amazon/dynamodb-local:latest
ports = 8000:8000/tcp

[testenv]
docker =
    dynamoDB
commands =
    pytest tests/ {posargs}

With tox 4.0.12:

# docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED         STATUS        PORTS                                       NAMES
d3c1a6403b59   amazon/dynamodb-local:latest   "java -jar DynamoDBL…"   2 seconds ago   Up 1 second   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp   dynamoDB-tox-364494

With tox 4.0.13 and higher:

# docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED         STATUS         PORTS                                         NAMES
4de548f795c3   amazon/dynamodb-local:latest   "java -jar DynamoDBL…"   2 seconds ago   Up 2 seconds   0.0.0.0:32790->8000/tcp, :::32790->8000/tcp   dynamoDB-tox-363635
@gaborbernat gaborbernat added bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted. labels Jan 4, 2023
@gaborbernat
Copy link
Member

This is because ports = 8000:8000/tcp translates as a factor filter for an environment named 8000, so the argument for port loaded is 8000/tcp that's not sufficient. I'd consider it a bug that tox-docker does not fail with a hard error. PR to fix this is welcome, likely we should require : not just : to trigger a factor filter for a given line.

@pdecat
Copy link
Contributor Author

pdecat commented Jan 4, 2023

Submitted #2822

gaborbernat pushed a commit that referenced this issue Jan 5, 2023
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Resolves #2821
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants