Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError: cannot import name 'packaging' from 'pkg_resources' #3176

Open
Sieltek opened this issue Jun 5, 2024 · 4 comments
Open

ImportError: cannot import name 'packaging' from 'pkg_resources' #3176

Sieltek opened this issue Jun 5, 2024 · 4 comments

Comments

@Sieltek
Copy link

Sieltek commented Jun 5, 2024 •
edited
Loading

🐛 Describe the bug

Problem seems to arise when loading a model using install_py_dep_per_model=true which will upgrade the setuptools version to latest 70.0 and thus breaking the handler loading.

Solution might be to replace the deprecated import in base handler from pkg_resources import packaging by import packaging

Error logs

2024-06-05T10:24:37,046 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Backend worker process died.
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 108, in load
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module, function_name = self._load_handler_file(handler)
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 153, in _load_handler_file
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name)
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/usr/lib64/python3.11/importlib/init.py", line 126, in import_module
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1149, in _find_and_load_unlocked
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 690, in _load_unlocked
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 940, in exec_module
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/handler.py", line 1, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from handler_yolo import YoloHandler
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/handler_yolo.py", line 6, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from ts.torch_handler.object_detector import ObjectDetector
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/object_detector.py", line 8, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from .vision_handler import VisionHandler
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/vision_handler.py", line 16, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from .base_handler import BaseHandler
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/base_handler.py", line 13, in
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from pkg_resources import packaging
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ImportError: cannot import name 'packaging' from 'pkg_resources' (/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/pkg_resources/init.py)
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG -
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - During handling of the above exception, another exception occurred:
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG -
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 263, in
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - worker.run_server()
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 231, in run_server
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 194, in handle_connection
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 131, in load_model
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - service = model_loader.load(
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 110, in load
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = self._load_default_handler(handler)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 159, in _load_default_handler
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name, "ts.torch_handler")
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/usr/lib64/python3.11/importlib/init.py", line 126, in import_module
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1128, in _find_and_load_unlocked
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1142, in _find_and_load_unlocked
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'ts.torch_handler.handler'
2024-06-05T10:24:37,061 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2024-06-05T10:24:37,062 [DEBUG] W-9000-crochet_yolov8s_512_240207_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2024-06-05T10:24:37,062 [DEBUG] W-9000-crochet_yolov8s_512_240207_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died., responseTimeout:120sec

Installation instructions

python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org pip==24.0 setuptools==69.5.1

python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir numpy==1.24.3 cython==0.29.34 wheel==0.40.0 pillow==9.3.0 psutil==5.9.5 requests==2.31.0 captum==0.6.0 packaging==23.1 pynvml==11.4.1 pyyaml==6.0

python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir torch==2.3.0+cpu torchvision==0.18.0+cpu torchtext==0.18.0 torchaudio==2.3.0+cpu --extra-index-url https://download.pytorch.org/whl/cpu

python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir torchserve==0.11.0 torch-model-archiver==0.11.0 torch-workflow-archiver==0.2.13

mkdir /model-server
mkdir /model-server/model-store
mkdir /model-server/wf-store

Model Packaging

torch-model-archiver --model-name my_model_name --handler handler.py --extra-files index_to_name.json --serialized-file best.pt -r requirements.txt -v 1.0 --force

crash:

requirements.txt:
numpy

workaround:

requirements.txt:
numpy
setuptools==69.5.1

loader failing at

serve/ts/torch_handler/base_handler.py
l.13 from pkg_resources import packaging

config.properties

echo "
inference_address=http://0.0.0.0:8080
management_address=http://0.0.0.0:8081
metrics_address=http://0.0.0.0:8082
grpc_inference_port=7070
grpc_management_port=7071
cors_allowed_origin=*
cors_allowed_methods=GET, POST, PUT, DELETE, OPTIONS
default_workers_per_model=2
job_queue_size=1000
model_store=/model-server/model-store
workflow_store=/model-server/wf-store
load_models=all
prefer_direct_buffer=true
install_py_dep_per_model=true
enable_metrics_api=true
prefer_direct_buffer=true
disable_system_metrics=false
metrics_mode=prometheus
cpu_launcher_enable=true
cpu_launcher_args=--use_logical_core
" > /model-server/config.properties

Versions

Environment headers

Torchserve branch:

torchserve==0.11.0
torch-model-archiver==0.11.0

Python version: 3.11 (64-bit runtime)
Python executable: /usr/bin/python3

Versions of relevant python libraries:
captum==0.6.0
numpy==1.24.3
psutil==5.9.5
requests==2.31.0
torch==2.3.0+cpu
torch-model-archiver==0.11.0
torch-workflow-archiver==0.2.13
torchaudio==2.3.0+cpu
torchserve==0.11.0
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
wheel==0.40.0
torch==2.3.0+cpu
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
torchaudio==2.3.0+cpu

Java Version:

OS: N/A
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18)
Clang version: N/A
CMake version: N/A

Environment:
library_path (LD_/DYLD_):

Repro instructions

torchserve --start --ts-config /model-server/config.properties

Possible Solution

Specify setuptool version in model.mar requirement.txt

workaround:

requirements.txt:
numpy
setuptools==69.5.1
@namannandan namannandan mentioned this issue Jun 5, 2024
Merged
2 tasks
@Lix1993
Copy link

Lix1993 commented Jun 12, 2024

python -m pip install setuptools==69.5.1

@romilbhardwaj romilbhardwaj mentioned this issue Jun 12, 2024
Merged
6 tasks
@aittalam aittalam mentioned this issue Jun 13, 2024
Closed
@RainJayTsai
Copy link

Exporting export PIP_NO_DEPS=true in the environment is also a workaround

If running in a Docker environment, you can use the following command:

docker run -e PIP_NO_CACHE_DIR=true -e PIP_NO_DEPS=true ...

This sets PIP_NO_CACHE_DIR=true and PIP_NO_DEPS=true as environment variables within the Docker container.

BUT If you choose to run with PIP_NO_CACHE_DIR=true and PIP_NO_DEPS=true in environment, you need to manage the dependencies manually.

@WardLT WardLT mentioned this issue Jun 21, 2024
Merged
@ebsmothers ebsmothers mentioned this issue Jun 24, 2024
Closed
@jtharrell2023
Copy link

Also, if you get this or similar when trying to host Whisper, notice the comments in the overview regarding setuptools-rust.

@udaij12
Copy link
Collaborator

udaij12 commented Jun 24, 2024 •
edited
Loading

#3187 Solves this by changing the import to import packaging.version which eliminates the need to pin setuptools

@MyNiuuu MyNiuuu mentioned this issue Jun 26, 2024
Closed
@balakumar-s balakumar-s mentioned this issue Jun 27, 2024
Closed
@Anibaaal Anibaaal mentioned this issue Aug 12, 2024
Closed
@limeberri limeberri mentioned this issue Aug 16, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@RainJayTsai @Lix1993 @udaij12 @Sieltek @jtharrell2023