import error with protobuf==4.21.0 #13159

peterdudfield · 2022-05-26T10:35:09Z

🐛 Bug

import error when using new protobuf version

To Reproduce

pip install pytorch_lightning protobuf

python

import pytorch_lightning as pl

Results

import pytorch_lightning as pl
/usr/local/lib/python3.9/site-packages/pytorch_lightning/__init__.py:30: in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
/usr/local/lib/python3.9/site-packages/pytorch_lightning/callbacks/__init__.py:26: in <module>
    from pytorch_lightning.callbacks.pruning import ModelPruning
/usr/local/lib/python3.9/site-packages/pytorch_lightning/callbacks/pruning.py:[31](https://github.com/openclimatefix/nowcasting_forecast/runs/6607770425?check_suite_focus=true#step:4:32): in <module>
    from pytorch_lightning.core.lightning import LightningModule
/usr/local/lib/python3.9/site-packages/pytorch_lightning/core/__init__.py:16: in <module>
    from pytorch_lightning.core.lightning import LightningModule
/usr/local/lib/python3.9/site-packages/pytorch_lightning/core/lightning.py:40: in <module>
    from pytorch_lightning.loggers import LightningLoggerBase, LoggerCollection
/usr/local/lib/python3.9/site-packages/pytorch_lightning/loggers/__init__.py:18: in <module>
    from pytorch_lightning.loggers.tensorboard import TensorBoardLogger
/usr/local/lib/python3.9/site-packages/pytorch_lightning/loggers/tensorboard.py:26: in <module>
    from torch.utils.tensorboard import SummaryWriter
/usr/local/lib/python3.9/site-packages/torch/utils/tensorboard/__init__.py:10: in <module>
    from .writer import FileWriter, SummaryWriter  # noqa: F401
/usr/local/lib/python3.9/site-packages/torch/utils/tensorboard/writer.py:9: in <module>
    from tensorboard.compat.proto.event_pb2 import SessionLog
/usr/local/lib/python3.9/site-packages/tensorboard/compat/proto/event_pb2.py:17: in <module>
    from tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2
/usr/local/lib/python3.9/site-packages/tensorboard/compat/proto/summary_pb2.py:17: in <module>
    from tensorboard.compat.proto import tensor_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__pb2
/usr/local/lib/python3.9/site-packages/tensorboard/compat/proto/tensor_pb2.py:16: in <module>
    from tensorboard.compat.proto import resource_handle_pb2 as tensorboard_dot_compat_dot_proto_dot_resource__handle__pb2
/usr/local/lib/python3.9/site-packages/tensorboard/compat/proto/resource_handle_pb2.py:16: in <module>
    from tensorboard.compat.proto import tensor_shape_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__shape__pb2
/usr/local/lib/python3.9/site-packages/tensorboard/compat/proto/tensor_shape_pb2.py:[36](https://github.com/openclimatefix/nowcasting_forecast/runs/6607770425?check_suite_focus=true#step:4:37): in <module>
    _descriptor.FieldDescriptor(
/usr/local/lib/python3.9/site-packages/google/protobuf/descriptor.py:[56](https://github.com/openclimatefix/nowcasting_forecast/runs/6607770425?check_suite_focus=true#step:4:57)0: in __new__
    _message.Message._CheckCalledFromGeneratedFile()
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E   
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Expected behavior

No import error

Environment

PyTorch Lightning Version (e.g., 1.5.0): pytorch-lightning-1.6.3
PyTorch Version (e.g., 1.10): torch-1.11.0
Python version (e.g., 3.9): 3.9
OS (e.g., Linux): from docker image python:3.9-slim
CUDA/cuDNN version:
GPU models and configuration: CPU
How you installed PyTorch (conda, pip, source): pip
If compiling from source, the output of torch.__config__.show():
Any other relevant information: I also get this error on my macbook

Additional context

Filed on this github action

cc @tchaton @rohitgr7

The text was updated successfully, but these errors were encountered:

peterdudfield · 2022-05-26T10:38:32Z

Could be related to this - protocolbuffers/protobuf#10051

akihironitta · 2022-05-26T12:40:54Z

It seems so. For anyone seeing this issue, a quick workaround is to downgrade protobuf version as shown in the error:

pip install "protobuf<4.21.0"

EDIT on June 9, 2022:
Upgrading tensorboard to 2.9.1 or later should also work:

pip install tensorboard -U

adamjstewart · 2022-05-26T18:37:16Z

This is an issue with tensorboard, not with pytorch-lightning, correct? Can someone open an issue with tensorboard?

ashnair1 · 2022-05-27T06:10:17Z

Tensorboard devs have pinned protobuf - tensorflow/tensorboard@fd4f5ff

akihironitta · 2022-05-27T13:43:50Z

(To protect our CI from errors like this for stability, ) This issue is not something we could've avoided with our current dependency freezing logic using dependabot because Lightning doesn't have a direct dependency to protobuf and tensorboard releases didn't set the upper version of protobuf.

akihironitta · 2022-05-27T14:35:17Z

Quickly checked our CI jobs and saw a few failures due to this issue, but they all seem to work fine now possibly because tensorboard pinned protobuf version already.

adamjstewart · 2022-05-27T14:53:17Z

The reason CI works now is that protobuf yanked the latest release from PyPI: https://pypi.org/project/protobuf/#history. Once another release comes out things will break again unless tensorboard has a new release that contains the pin.

akihironitta · 2022-05-27T15:47:26Z

@adamjstewart Thank you very much for the information! I'll keep this issue open so that we do something before it breaks our CI again.

https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078 ``` E TypeError: Descriptors cannot not be created directly. E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. E If you cannot immediately regenerate your protos, some other possible workarounds are: E 1. Downgrade the protobuf package to 3.20.x or lower. E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). E E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates ``` protocolbuffers/protobuf#10051 Lightning-AI/pytorch-lightning#13159

Summary: All the unittests jobs are failing due to import error due to protobuf and scipy. This commit pins the versions of them to an older version. ## protobuf https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078 ``` E TypeError: Descriptors cannot not be created directly. E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. E If you cannot immediately regenerate your protos, some other possible workarounds are: E 1. Downgrade the protobuf package to 3.20.x or lower. E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). E E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates ``` protocolbuffers/protobuf#10051 Lightning-AI/pytorch-lightning#13159 ## scipy (pypocketfft) 1.8.1 is causing issue. https://app.circleci.com/pipelines/github/pytorch/audio/10980/workflows/470a9361-4cc5-4d7c-9264-28fc8b86f1cb/jobs/692267 ``` ../env/lib/python3.9/site-packages/librosa/core/audio.py:11: in <module> import scipy.signal ../env/lib/python3.9/site-packages/scipy/signal/__init__.py:309: in <module> from . import _sigtools, windows ../env/lib/python3.9/site-packages/scipy/signal/windows/__init__.py:41: in <module> from ._windows import * ../env/lib/python3.9/site-packages/scipy/signal/windows/_windows.py:7: in <module> from scipy import linalg, special, fft as sp_fft ../env/lib/python3.9/site-packages/scipy/fft/__init__.py:91: in <module> from ._helper import next_fast_len ../env/lib/python3.9/site-packages/scipy/fft/_helper.py:3: in <module> from ._pocketfft import helper as _helper ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py:3: in <module> from .basic import * ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py:6: in <module> from . import pypocketfft as pfft E ImportError: /home/circleci/project/env/lib/python3.9/site-packages/torch/lib/../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/circleci/project/env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so) Pull Request resolved: #2422 Reviewed By: atalman Differential Revision: D36764198 Pulled By: mthrok fbshipit-source-id: 897a79fe9c3165206c2e747147fd0f257fc4f683

Summary: It fails at importing pytorch_lightning. This issue is caused by protobuf and fixed in lightning by pinning "protobuf<=3.20.1": Lightning-AI/pytorch-lightning#13159 Reproduce: (recipes) ➜ python dlrm_main.py Traceback (most recent call last): File "dlrm_main.py", line 14, in <module> import pytorch_lightning as pl File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/__init__.py", line 30, in <module> from pytorch_lightning.callbacks import Callback # noqa: E402 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/callbacks/__init__.py", line 26, in <module> from pytorch_lightning.callbacks.pruning import ModelPruning File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/callbacks/pruning.py", line 31, in <module> from pytorch_lightning.core.lightning import LightningModule File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/core/__init__.py", line 16, in <module> from pytorch_lightning.core.lightning import LightningModule File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/core/lightning.py", line 40, in <module> from pytorch_lightning.loggers import LightningLoggerBase File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/loggers/__init__.py", line 18, in <module> from pytorch_lightning.loggers.tensorboard import TensorBoardLogger File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/pytorch_lightning/loggers/tensorboard.py", line 26, in <module> from torch.utils.tensorboard import SummaryWriter File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/torch/utils/tensorboard/__init__.py", line 10, in <module> from .writer import FileWriter, SummaryWriter # noqa: F401 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py", line 9, in <module> from tensorboard.compat.proto.event_pb2 import SessionLog File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/tensorboard/compat/proto/event_pb2.py", line 17, in <module> from tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/tensorboard/compat/proto/summary_pb2.py", line 17, in <module> from tensorboard.compat.proto import tensor_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__pb2 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/tensorboard/compat/proto/tensor_pb2.py", line 16, in <module> from tensorboard.compat.proto import resource_handle_pb2 as tensorboard_dot_compat_dot_proto_dot_resource__handle__pb2 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/tensorboard/compat/proto/resource_handle_pb2.py", line 16, in <module> from tensorboard.compat.proto import tensor_shape_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__shape__pb2 File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/tensorboard/compat/proto/tensor_shape_pb2.py", line 42, in <module> serialized_options=None, file=DESCRIPTOR), File "/Users/stevenliu/opt/anaconda3/envs/recipes/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 560, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates Reviewed By: colin2328 Differential Revision: D36875536 fbshipit-source-id: 0233696282b1f49dd0e061024e6ddb965df1d8df

dubeykanishka · 2022-06-07T17:46:03Z

It seems so. For anyone seeing this issue, a quick workaround is to downgrade protobuf version as shown in the error:
pip install "protobuf<4.21.0"

Thank You so much.

peterdudfield added the needs triage Waiting to be triaged by maintainers label May 26, 2022

ashnair1 mentioned this issue May 26, 2022

Update kornia version in environment.yml microsoft/torchgeo#542

Merged

akihironitta added bug Something isn't working 3rd party Related to a 3rd-party and removed needs triage Waiting to be triaged by maintainers labels May 26, 2022

adamjstewart mentioned this issue May 26, 2022

Pin protobuf<4 microsoft/torchgeo#543

Merged

akihironitta added the priority: 0 High priority task label May 27, 2022

akihironitta self-assigned this May 27, 2022

akihironitta removed the priority: 0 High priority task label May 27, 2022

akihironitta closed this as completed May 27, 2022

akihironitta reopened this May 27, 2022

akihironitta added this to the 1.6.x milestone May 27, 2022

hrzn mentioned this issue May 28, 2022

fixed change log unit8co/darts#981

Merged

akihironitta mentioned this issue May 28, 2022

Pin protobuf<=3.20.1 #13177

Merged

8 tasks

mthrok mentioned this issue May 30, 2022

Pin test tool versions in CI pytorch/audio#2422

Closed

This was referenced May 31, 2022

new: add quaterion progress bar, add trainer defaults, move callbacks… qdrant/quaterion#114

Merged

[docs] recent protobuf update is incompatible with lightning and tensorboard qdrant/quaterion#115

Closed

carmocca closed this as completed in #13177 May 31, 2022

mrwyattii mentioned this issue May 31, 2022

Fix for lightning tests microsoft/DeepSpeed#1988

Merged

akihironitta mentioned this issue Jun 2, 2022

Tracking - pytorch lightning fails to import with protobuf-4.21.1 #13212

Closed

justjhong mentioned this issue Jun 3, 2022

Installing scvi-tools with conda and with pip scverse/scvi-tools#1551

Closed

akihironitta mentioned this issue Jun 9, 2022

Unpin protobuf version and update tensorboard version #13259

Merged

8 tasks

This was referenced Sep 11, 2022

Resuming should allow to differentiate what to resume (steps/opti/weights) #5339

Open

Update tests for latest PL release Lightning-Universe/lightning-transformers#284

Merged

ab93 mentioned this issue Jan 6, 2023

Pin protobuf to 3.20 numaproj/numalogic#123

Closed

shchur mentioned this issue Oct 20, 2023

Fix version range for lightning awslabs/gluonts#3023

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import error with protobuf==4.21.0 #13159

import error with protobuf==4.21.0 #13159

peterdudfield commented May 26, 2022 •

edited by github-actions bot

Loading

peterdudfield commented May 26, 2022 •

edited

Loading

akihironitta commented May 26, 2022 •

edited

Loading

adamjstewart commented May 26, 2022

ashnair1 commented May 27, 2022

akihironitta commented May 27, 2022

akihironitta commented May 27, 2022

adamjstewart commented May 27, 2022

akihironitta commented May 27, 2022

dubeykanishka commented Jun 7, 2022

import error with protobuf==4.21.0 #13159

import error with protobuf==4.21.0 #13159

Comments

peterdudfield commented May 26, 2022 • edited by github-actions bot Loading

🐛 Bug

To Reproduce

Expected behavior

Environment

Additional context

peterdudfield commented May 26, 2022 • edited Loading

akihironitta commented May 26, 2022 • edited Loading

adamjstewart commented May 26, 2022

ashnair1 commented May 27, 2022

akihironitta commented May 27, 2022

akihironitta commented May 27, 2022

adamjstewart commented May 27, 2022

akihironitta commented May 27, 2022

dubeykanishka commented Jun 7, 2022

peterdudfield commented May 26, 2022 •

edited by github-actions bot

Loading

peterdudfield commented May 26, 2022 •

edited

Loading

akihironitta commented May 26, 2022 •

edited

Loading