Cannot use ecs-deploy without installing setuptools

[jnyrup]

I'm using ecs-deploy in Azure DevOps and this morning our deploy pipeline crashed with an error No module named 'pkg_resources'.

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.0/x64/bin/ecs", line 5, in <module>
    from ecs_deploy.cli import ecs
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/cli.py", line 12, in <module>
    from ecs_deploy.ecs import DeployAction, ScaleAction, RunAction, EcsClient, DiffAction, \
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/ecs.py", line 12, in <module>
    from dictdiffer import diff
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/dictdiffer/__init__.py", line 15, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

As a workaround we manually install setuptools, but I guess that shouldn't be required to use ecs-deploy?

I'm not well-versed in the python ecosystem, but as far as I can tell, the dependency on pkg_resources comes from Dictdiffer.
Its dependency on pkg_resources was removed in inveniosoftware/dictdiffer#139 and released in v0.9.0

I see that ecs-deploy explicitly depends on v0.8.0 which was the newest available at the time of its inclusion in ecs-deploy.

ecs-deploy/setup.py

Line 21 in 79812c7

'dictdiffer==0.8.0',

Fast-forwarding to today, would it be possible (perhaps even desirable) to bump ecs-deploy's dependency on dictdiffer to 0.9.0?

In case you're interested in what changed with our dependencies, here's the diff in the resolved modules between the last successful run and the first failing one.

- boto3-1.28.59
+ boto3-1.28.62
- botocore-1.31.59
+ botocore-1.31.62
- urllib3-1.26.17
+ urllib3-2.0.6

more log

Collecting ecs-deploy
  Downloading ecs-deploy-1.14.0.tar.gz (33 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting click<9,>=7.1.2 (from ecs-deploy)
  Obtaining dependency information for click<9,>=7.1.2 from https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting click-log==0.3.2 (from ecs-deploy)
  Downloading click_log-0.3.2-py2.py3-none-any.whl (4.6 kB)
Collecting botocore>=1.17.47 (from ecs-deploy)
  Obtaining dependency information for botocore>=1.17.47 from https://files.pythonhosted.org/packages/a8/3f/74138007b045447eac6141c8144efe8e1c9f377cf56c85edfe1111a22f97/botocore-1.31.62-py3-none-any.whl.metadata
  Downloading botocore-1.31.62-py3-none-any.whl.metadata (6.1 kB)
Collecting boto3>=1.14.47 (from ecs-deploy)
  Obtaining dependency information for boto3>=1.14.47 from https://files.pythonhosted.org/packages/63/e5/8fc4a69186cb15b0dba9c428da73233c89eb18ee03ce56f6bde205ea2006/boto3-1.28.62-py3-none-any.whl.metadata
  Downloading boto3-1.28.62-py3-none-any.whl.metadata (6.7 kB)
Collecting future (from ecs-deploy)
  Downloading future-0.18.3.tar.gz (840 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 840.9/840.9 kB 27.0 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting requests (from ecs-deploy)
  Obtaining dependency information for requests from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting dictdiffer==0.8.0 (from ecs-deploy)
  Downloading dictdiffer-0.8.0-py2.py3-none-any.whl (16 kB)
Collecting jmespath<2.0.0,>=0.7.1 (from boto3>=1.14.47->ecs-deploy)
  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.8.0,>=0.7.0 (from boto3>=1.14.47->ecs-deploy)
  Obtaining dependency information for s3transfer<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/5a/4b/fec9ce18f8874a96c5061422625ba86c3ee1e6587ccd92ff9f5bf7bd91b2/s3transfer-0.7.0-py3-none-any.whl.metadata
  Downloading s3transfer-0.7.0-py3-none-any.whl.metadata (1.8 kB)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore>=1.17.47->ecs-deploy)
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 62.7 MB/s eta 0:00:00
Collecting urllib3<2.1,>=1.25.4 (from botocore>=1.17.47->ecs-deploy)
  Obtaining dependency information for urllib3<2.1,>=1.25.4 from https://files.pythonhosted.org/packages/26/40/9957270221b6d3e9a3b92fdfba80dd5c9661ff45a664b47edd5d00f707f5/urllib3-2.0.6-py3-none-any.whl.metadata
  Downloading urllib3-2.0.6-py3-none-any.whl.metadata (6.6 kB)
Collecting charset-normalizer<4,>=2 (from requests->ecs-deploy)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/e1/ee/04bbbe050ac4d4ac92ca3fa4f9ada00ee7d2095a91ccadb9a2065943a1bf/charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (32 kB)
Collecting idna<4,>=2.5 (from requests->ecs-deploy)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 21.6 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests->ecs-deploy)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore>=1.17.47->ecs-deploy)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading boto3-1.28.62-py3-none-any.whl (135 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.8/135.8 kB 40.0 MB/s eta 0:00:00
Downloading botocore-1.31.62-py3-none-any.whl (11.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.2/11.2 MB 117.1 MB/s eta 0:00:00
Downloading click-8.1.7-py3-none-any.whl (97 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 34.1 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 25.8 MB/s eta 0:00:00
Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 40.9 MB/s eta 0:00:00
Downloading charset_normalizer-3.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.7/138.7 kB 48.9 MB/s eta 0:00:00
Downloading s3transfer-0.7.0-py3-none-any.whl (79 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 29.9 MB/s eta 0:00:00
Downloading urllib3-2.0.6-py3-none-any.whl (123 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.8/123.8 kB 46.4 MB/s eta 0:00:00
Building wheels for collected packages: ecs-deploy, future
  Building wheel for ecs-deploy (pyproject.toml): started
  Building wheel for ecs-deploy (pyproject.toml): finished with status 'done'
  Created wheel for ecs-deploy: filename=ecs_deploy-1.14.0-py2.py3-none-any.whl size=27650 sha256=82362d4fd07a7b4b548504aed2c2de9792896343e3e6011c80fb94b243815586
  Stored in directory: /home/vsts/.cache/pip/wheels/52/4b/ff/b92eb003aa3c22df5a2b66a73f10baef4dfe4d8f2048f2e11a
  Building wheel for future (pyproject.toml): started
  Building wheel for future (pyproject.toml): finished with status 'done'
  Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492024 sha256=a005d59d745a91709c82d3dd43511e2d551a6da5eb2af0d0444afddf8ae7ab75
  Stored in directory: /home/vsts/.cache/pip/wheels/f1/54/27/4e5cb4bd05116c097877e85a206a2e08d301aaa9b7af719909
Successfully built ecs-deploy future
Installing collected packages: dictdiffer, urllib3, six, jmespath, idna, future, click, charset-normalizer, certifi, requests, python-dateutil, click-log, botocore, s3transfer, boto3, ecs-deploy
Successfully installed boto3-1.28.62 botocore-1.31.62 certifi-2023.7.22 charset-normalizer-3.3.0 click-8.1.7 click-log-0.3.2 dictdiffer-0.8.0 ecs-deploy-1.14.0 future-0.18.3 idna-3.4 jmespath-1.0.1 python-dateutil-2.8.2 requests-2.31.0 s3transfer-0.7.0 six-1.16.0 urllib3-2.0.6
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.0/x64/bin/ecs", line 5, in <module>
    from ecs_deploy.cli import ecs
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/cli.py", line 12, in <module>
    from ecs_deploy.ecs import DeployAction, ScaleAction, RunAction, EcsClient, DiffAction, \
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/ecs_deploy/ecs.py", line 12, in <module>
    from dictdiffer import diff
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/dictdiffer/__init__.py", line 15, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

[fabfuel]

Hi Jonas,

thanks for reporting this. It seems to be related to Python 3.12, I will look into it!

Best
Fabian