Issue in deepspeed installation with python 3.8

[amalislam675]

I have tried to installed the deepspeed using pip install deepspeed. But, it is not installed correctly within my environment. However, in my environment Python 3.8.13, Pytorch 1.12.0 with cuda_tool_kit 11.3, transformers=4.9.2 are installed.
Some lines of error are mentioned below:
/lib/python3.8/site-packages/pip/_internal/network/utils.py", line 63, in response_chunks
/lib/python3.8/site-packages/pip/_vendor/urllib3/response.py", line 573, in stream
data = self.read(amt=amt, decode_content=decode_content)
.....................
is there anyway to install?

[mrwyattii]

@amalislam675 could you please share the full output of the error?

[amalislam675]

@mrwyattii this is the full output of the error which I have got.
Traceback (most recent call last):
File "", line 1, in
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/deepspeed/init.py", line 15, in
from .runtime.engine import DeepSpeedEngine, DeepSpeedOptimizerCallable, DeepSpeedSchedulerCallable
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 20, in
from tensorboardX import SummaryWriter
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/init.py", line 5, in
from .torchvis import TorchVis
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/torchvis.py", line 11, in
from .writer import SummaryWriter
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/writer.py", line 15, in
from .event_file_writer import EventFileWriter
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/event_file_writer.py", line 28, in
from .proto import event_pb2
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/proto/event_pb2.py", line 15, in
from tensorboardX.proto import summary_pb2 as tensorboardX_dot_proto_dot_summary__pb2
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/proto/summary_pb2.py", line 15, in
from tensorboardX.proto import tensor_pb2 as tensorboardX_dot_proto_dot_tensor__pb2
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/tensorboardX/proto/tensor_pb2.py", line 15, in
from tensorboardX.proto import resource_handle_pb2 as tensorboardX_dot_proto_dot_resource__handle__pb2
File "//mnt/user1/envs/pretraining/lib/python3.8/site-packages/tensorboardX/proto/resource_handle_pb2.py", line 35, in
_descriptor.FieldDescriptor(
File "//mnt/user1/.conda/envs/pretraining/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

[mrwyattii]

This error looks very similar to something we encountered recently and fixed in #1988. I don't believe this is a problem with DeepSpeed, but with TensorBoard/Protobuf.

Could you try downgrading your protobuf install (to anything below version 4.21.0) and see if that fixes things: pip install "protobuf<4.21.0"

[amalislam675]

Thankyou, I will try.

[mrwyattii]

@amalislam675 did this work for you?

[amalislam675]

@mrwyattii , It works fine. Thankyou.