AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer' on latest version #16935

smithj51 · 2024-05-23T16:42:13Z

What version of protobuf and what language are you using?
Version: v27.0
Language: Python

What operating system (Linux, Windows, ...) and version?
Windows

What runtime / compiler are you using (e.g., python version or gcc version)
python in gitlab ci/cd

What did you do?
Steps to reproduce the behavior:
python pip install following packages:
MarkupSafe, werkzeug, click, pathspec, logbook, protobuf, pytz, Babel, text-unidecode, python-slugify, leather, six, isodate, pytimeparse, future, parsedatetime, agate, pkgutil-resolve-name, attrs, rpds-py, referencing, zipp, importlib-resources, jsonschema-specifications, jsonschema, python-dateutil, hologram, pycparser, cffi, charset-normalizer, certifi, urllib3, idna, requests, packaging, Jinja2, typing-extensions, msgpack, mashumaro, minimal-snowplow-tracker, colorama, networkx, dbt-extractor, pyyaml, sqlparse, dbt-core, googleapis-common-protos, proto-plus, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, google-api-core, google-crc32c, google-cloud-core, google-resumable-media, google-cloud-storage, grpcio, grpc-google-iam-v1, google-cloud-dataproc, google-cloud-bigquery, dbt-bigquery, portalocker, platformdirs, tblib, importlib-metadata, distlib, filelock, virtualenv, isolate, isolate-proto, structlog, types-python-dateutil, grpc-interceptor, frozenlist, async-timeout, multidict, yarl, aiosignal, aiohttp, pyjwt, cryptography, auth0-python, opentelemetry-semantic-conventions, wrapt, deprecated, opentelemetry-api, opentelemetry-sdk, pydantic, datadog-api-client, dill, mdurl, markdown-it-py, pygments, rich, sniffio, h11, exceptiongroup, anyio, httpcore, httpx, starlette, fastapi, fal, backports.functools-lru-cache, backoff, monotonic, posthog, greenlet, sqlalchemy, deprecation, astor, agate-sql, numpy, pandas, dbt-fal, slack-sdk, tabulate, pyarrow, db-dtypes

What did you expect to see
compatibility with google APIs

What did you see instead?

ERROR: googleapis-common-protos 1.63.0 has requirement protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0.dev0,>=3.19.5, but you'll have protobuf 5.27.0 which is incompatible.
ERROR: proto-plus 1.23.0 has requirement protobuf<5.0.0dev,>=3.19.0, but you'll have protobuf 5.27.0 which is incompatible.
ERROR: google-api-core 2.19.0 has requirement protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0.dev0,>=3.19.5, but you'll have protobuf 5.27.0 which is incompatible.
ERROR: grpc-google-iam-v1 0.13.0 has requirement protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you'll have protobuf 5.27.0 which is incompatible.
ERROR: google-cloud-dataproc 5.9.3 has requirement protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you'll have protobuf 5.27.0 which is incompatible.

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
Full error log from gitlab ci/cd job log:

Traceback (most recent call last):
  File "/usr/local/bin/dbt", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1077, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 943, in make_context
    self.parse_args(ctx, args)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1644, in parse_args
    rest = super().parse_args(ctx, args)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1408, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 2400, in handle_parse_result
    value = self.process_value(ctx, value)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 2362, in process_value
    value = self.callback(ctx, self, value)
  File "/usr/local/lib/python3.8/site-packages/dbt/cli/params.py", line 500, in _version_callback
    click.echo(get_version_information())
  File "/usr/local/lib/python3.8/site-packages/dbt/version.py", line 24, in get_version_information
    plugin_version_msg = _get_plugins_msg(installed)
  File "/usr/local/lib/python3.8/site-packages/dbt/version.py", line 104, in _get_plugins_msg
    for name, version_s in _get_dbt_plugins_info():
  File "/usr/local/lib/python3.8/site-packages/dbt/version.py", line 211, in _get_dbt_plugins_info
    mod = importlib.import_module(f"dbt.adapters.{plugin_name}.__version__")
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line [1014](https://gitlab.com/spigotinc/mobile/dbt/ios-eet/-/jobs/6924644813#L1014), in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/dbt/adapters/bigquery/__init__.py", line 5, in <module>
    from dbt.adapters.bigquery.impl import BigQueryAdapter, GrantTarget  # noqa
  File "/usr/local/lib/python3.8/site-packages/dbt/adapters/bigquery/impl.py", line 30, in <module>
    from dbt.adapters.bigquery.python_submissions import (
  File "/usr/local/lib/python3.8/site-packages/dbt/adapters/bigquery/python_submissions.py", line 8, in <module>
    from google.cloud import storage, dataproc_v1  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/__init__.py", line 21, in <module>
    from .services.autoscaling_policy_service import (
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/services/autoscaling_policy_service/__init__.py", line 16, in <module>
    from .async_client import AutoscalingPolicyServiceAsyncClient
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/services/autoscaling_policy_service/async_client.py", line 49, in <module>
    from google.cloud.dataproc_v1.services.autoscaling_policy_service import pagers
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/services/autoscaling_policy_service/pagers.py", line 27, in <module>
    from google.cloud.dataproc_v1.types import autoscaling_policies
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/types/__init__.py", line 16, in <module>
    from .autoscaling_policies import (
  File "/usr/local/lib/python3.8/site-packages/google/cloud/dataproc_v1/types/autoscaling_policies.py", line 21, in <module>
    import proto  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/proto/__init__.py", line 15, in <module>
    from .enums import Enum
  File "/usr/local/lib/python3.8/site-packages/proto/enums.py", line 19, in <module>
    from proto import _file_info
  File "/usr/local/lib/python3.8/site-packages/proto/_file_info.py", line 24, in <module>
    from proto.marshal.rules.message import MessageRule
  File "/usr/local/lib/python3.8/site-packages/proto/marshal/__init__.py", line 15, in <module>
    from .marshal import Marshal
  File "/usr/local/lib/python3.8/site-packages/proto/marshal/marshal.py", line 25, in <module>
    from proto.marshal import compat
  File "/usr/local/lib/python3.8/site-packages/proto/marshal/compat.py", line 43, in <module>
    map_composite_types += (_message.MessageMapContainer,)
AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer'

Anything else we should know about your project / environment
using docker image python:3.8.1-slim-buster

The text was updated successfully, but these errors were encountered:

yhs0602 · 2024-05-24T03:48:49Z

Happens in python 3.9

google-api-core==2.12.0
google-api-python-client==2.100.0
google-auth==2.23.3
google-auth-httplib2==0.1.1
google-cloud-appengine-logging==1.1.2
google-cloud-audit-log==0.2.2
google-cloud-core==2.3.1
google-cloud-firestore==2.12.0
google-cloud-logging==3.10.0
...
proto-plus==1.22.3
protobuf==3.20.3

also.

Edit
Seems like datadog-init issues?

/dd_tracer/python/bin/ddtrace-run gunicorn ...

works fine, but

/app/datadog-init /dd_tracer/python/bin/ddtrace-run gunicorn ...

Doesn't

Edit 2:
datadog-init internally use protobuf-5.27.0, which was causing the issue.

ianmclaughlin-po · 2024-05-25T02:11:08Z

@yhs0602 How did you fix that issue with datadog-init? I am pinning protobuf==3.20.3 and still getting issues

doudoujay · 2024-05-27T06:17:42Z

@yhs0602 how did you solve this?

ianmclaughlin-po · 2024-05-28T15:58:03Z

@doudoujay This is my solution although this seems risky to be deployed to a production environment.

I have my application pip dependencies installed in a base image
I use a builder image to force protobuf==5.26.1 install and remove protobuf==5.27.0
Pull in base image and run application as normal

So I found the issue by comparing an old version of my working docker image and new one. Inside my broken container:

(base) ➜  ~ broken
root@288a9552d8c1:/api# pip show protobuf
Name: protobuf
Version: 4.25.3
Summary:
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: 3-Clause BSD License
Location: /usr/local/lib/python3.10/site-packages
Requires:
Required-by: ddtrace, google-api-core, google-cloud-appengine-logging, google-cloud-audit-log, google-cloud-logging, googleapis-common-protos, grpc-google-iam-v1, grpcio-status, proto-plus
root@288a9552d8c1:/api# /app/datadog-init pip show protobuf
Name: protobuf
Version: 5.27.0
Summary:
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: 3-Clause BSD License
Location: /dd_tracer/python
Requires:
Required-by: ddtrace, google-api-core, google-cloud-appengine-logging, google-cloud-audit-log, google-cloud-logging, googleapis-common-protos, grpc-google-iam-v1, grpcio-status, proto-plus

Inside my old working container:

(base) ➜  Downloads working
root@b5987abfa0e3:/api# pip show protobuf
Name: protobuf
Version: 4.25.3
Summary:
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: 3-Clause BSD License
Location: /usr/local/lib/python3.10/site-packages
Requires:
Required-by: ddtrace, google-api-core, google-cloud-appengine-logging, google-cloud-audit-log, google-cloud-logging, googleapis-common-protos, grpc-google-iam-v1, grpcio-status, proto-plus
root@b5987abfa0e3:/api# /app/datadog-init pip show protobuf
Name: protobuf
Version: 5.26.1
Summary:
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: 3-Clause BSD License
Location: /dd_tracer/python
Requires:
Required-by: ddtrace, google-api-core, google-cloud-appengine-logging, google-cloud-audit-log, google-cloud-logging, googleapis-common-protos, grpc-google-iam-v1, grpcio-status, proto-plus

Basically datadog-init is bundling it's own python dependencies without pinning them strongly so when the protobuf==5.27.0 was released this causing a conflict. In reality the lower versions of protobuf are not supposed to work with the high versions at all they only happened to be compatible it seems to me.

So my original docker image looked like this:

FROM python:3.10.8
COPY . . 
RUN pip install -r requirements.txt
<other random dependencies>

# Prep for datadog
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init

# set up a directory to run out of
WORKDIR /api

# Expose the running port
EXPOSE 5150

# set the app we want to run
ENV FLASK_APP=app.py
ENV LOGGING='formatted'

# Move app over
COPY . .

RUN chmod +x /api/entrypoint.sh

ENTRYPOINT ["/app/datadog-init", "/api/entrypoint.sh"]

My solution docker image looks like this:

# Use python:3.10.8 as the builder
FROM python:3.10.8 as builder

# Prep for datadog
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ ddtrace
RUN /app/datadog-init pip uninstall -y protobuf==5.27.0
RUN /app/datadog-init python -m pip install --force-reinstall --ignore-installed protobuf==5.26.1

# Use the base image
FROM <base image where I install other pip dependencies>

# Copy datadog-init and resolved dependencies from the builder
COPY --from=builder /app/datadog-init /app/datadog-init
COPY --from=builder /dd_tracer /dd_tracer

# set up a directory to run out of
WORKDIR /api

# Expose the running port
EXPOSE 5150

# set the app we want to run
ENV FLASK_APP=app.py
ENV LOGGING='formatted'

WORKDIR /api/dockerbuild
RUN python CustomCerts.py

# Move back to the api directory
WORKDIR /api

# Move app over
COPY . .

RUN chmod +x /api/entrypoint.sh


ENTRYPOINT ["/app/datadog-init"]
CMD ["/dd_tracer/python/bin/ddtrace-run", "/api/entrypoint.sh"]

doudoujay · 2024-05-28T17:55:04Z

my simple solution --

proto-plus==1.24.0.dev1

:)

calin-cocan · 2024-05-31T07:48:46Z

ddtrace relies on any protobuf >=3
The solution that we adopted was to specify in protobuf version before installing ddtrace.

COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ protobuf==5.26.1 ddtrace

A similar approach we took for the case when we had another dependencies that relies even on older protobuf version (mlflow)

COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ protobuf==4.25.3 ddtrace

deannagarcia · 2024-06-04T21:02:35Z

It looks like python proto plus just submitted a change to fix compatibility. Can y'all check to see if that fixes your issue?

ianmclaughlin-po · 2024-06-05T00:45:46Z

It looks like python proto plus just submitted a change to fix compatibility. Can y'all check to see if that fixes your issue?

Yes that fixed it for me using the 1.24.0.dev1 release

thlurte · 2024-06-18T16:00:01Z

It looks like python proto plus just submitted a change to fix compatibility. Can y'all check to see if that fixes your issue?

thank you, i was able to resolve the issue

parthea · 2024-07-04T20:17:53Z

@deannagarcia, This issue can be closed now as a formal release of proto-plus was published 2 weeks ago: https://pypi.org/project/proto-plus/1.24.0/

anilkumarmyla · 2024-07-10T15:03:01Z

Still have the issue with upgraded libraries
(no error)
google-api-core-2.19.1 google-api-python-client-2.136.0 google-auth-2.31.0 google-auth-httplib2-0.2.0 google-cloud-resource-manager-1.12.3 googleapis-common-protos-1.63.2 grpc-google-iam-v1-0.13.1 grpcio-1.64.1 grpcio-status-1.62.2 httplib2-0.22.0 idna-3.7 proto-plus-1.24.0 protobuf-4.25.3
(error)
google-api-core-2.19.1 google-api-python-client-2.137.0 google-auth-2.32.0 google-auth-httplib2-0.2.0 google-cloud-resource-manager-1.12.4 googleapis-common-protos-1.63.2 grpc-google-iam-v1-0.13.1 grpcio-1.64.1 grpcio-status-1.64.1 httplib2-0.22.0 idna-3.7 proto-plus-1.24.0 protobuf-5.27.2

parthea · 2024-07-10T15:08:46Z

@anilkumarmyla, Please could you share the full stack trace?

Please also share the output of

>>> import proto
>>> proto.__version__

anilkumarmyla · 2024-07-10T20:11:02Z

@parthea

bash-3.2$ python3 --version
Python 3.12.4
bash-3.2$ python3 -m venv .venv
bash-3.2$ source .venv/bin/activate

(.venv) bash-3.2$ gcloud --verbosity=debug --version
DEBUG: Running [gcloud.version] with arguments: []
DEBUG: Chosen display Format:flattened[no-pad,separator=" "]
INFO: Display format: "flattened[no-pad,separator=" "]"
Google Cloud SDK 483.0.0
alpha 2024.06.28
bq 2.1.6
core 2024.06.28
gcloud-crc32c 1.0.0
gsutil 5.30

(.venv) bash-3.2$ pip3 install proto-plus
Collecting proto-plus
  Using cached proto_plus-1.24.0-py3-none-any.whl.metadata (2.2 kB)
Collecting protobuf<6.0.0dev,>=3.19.0 (from proto-plus)
  Using cached protobuf-5.27.2-cp38-abi3-macosx_10_9_universal2.whl.metadata (592 bytes)
Using cached proto_plus-1.24.0-py3-none-any.whl (50 kB)
Using cached protobuf-5.27.2-cp38-abi3-macosx_10_9_universal2.whl (412 kB)
Installing collected packages: protobuf, proto-plus
Successfully installed proto-plus-1.24.0 protobuf-5.27.2

[notice] A new release of pip is available: 24.0 -> 24.1.2
[notice] To update, run: pip install --upgrade pip

(.venv) bash-3.2$ gcloud --verbosity=debug --version
DEBUG: Running [gcloud.version] with arguments: []
DEBUG: Chosen display Format:flattened[no-pad,separator=" "]
INFO: Display format: "flattened[no-pad,separator=" "]"
DEBUG: module 'google._upb._message' has no attribute 'MessageMapContainer'
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 998, in Execute
    resources = calliope_command.Run(cli=self, args=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 817, in Run
    display_info=self.ai.display_info).Display()
                                       ^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/calliope/display.py", line 487, in Display
    self._printer.Print(self._resources)
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 299, in Print
    self.AddRecord(resources, delimit=intermediate)
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 239, in AddRecord
    self._AddRecord(self._process_record(record), delimit)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py", line 517, in Evaluate
    return self._Project(obj, self._projection.Tree(), flag)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py", line 433, in _Project
    import proto  # pylint: disable=g-import-not-at-top
    ^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/__init__.py", line 15, in <module>
    from .enums import Enum
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/enums.py", line 19, in <module>
    from proto import _file_info
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/_file_info.py", line 24, in <module>
    from proto.marshal.rules.message import MessageRule
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/marshal/__init__.py", line 15, in <module>
    from .marshal import Marshal
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/marshal/marshal.py", line 25, in <module>
    from proto.marshal import compat
  File "/opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto/marshal/compat.py", line 43, in <module>
    map_composite_types += (_message.MessageMapContainer,)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer'
ERROR: gcloud crashed (AttributeError): module 'google._upb._message' has no attribute 'MessageMapContainer'

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics

(.venv) bash-3.2$ pip3 install protobuf==5.26.1
Collecting protobuf==5.26.1
  Downloading protobuf-5.26.1-cp37-abi3-macosx_10_9_universal2.whl.metadata (592 bytes)
Downloading protobuf-5.26.1-cp37-abi3-macosx_10_9_universal2.whl (404 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 404.0/404.0 kB 2.7 MB/s eta 0:00:00
Installing collected packages: protobuf
  Attempting uninstall: protobuf
    Found existing installation: protobuf 5.27.2
    Uninstalling protobuf-5.27.2:
      Successfully uninstalled protobuf-5.27.2
Successfully installed protobuf-5.26.1

[notice] A new release of pip is available: 24.0 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
(.venv) bash-3.2$ gcloud --verbosity=debug --version
DEBUG: Running [gcloud.version] with arguments: []
DEBUG: Chosen display Format:flattened[no-pad,separator=" "]
INFO: Display format: "flattened[no-pad,separator=" "]"
Google Cloud SDK 483.0.0
alpha 2024.06.28
bq 2.1.6
core 2024.06.28
gcloud-crc32c 1.0.0
gsutil 5.30

(.venv) bash-3.2$ python3
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import proto
>>> proto.__version__
'1.24.0'
>>>

parthea · 2024-07-10T20:40:14Z

Thanks for sharing the stack trace @anilkumarmyla! The stack trace is slightly different from the one in #16935 (comment) in that the error originates from proto-plus via the gcloud library in /opt/homebrew/Caskroom/google-cloud-sdk/481.0.0/google-cloud-sdk/lib/third_party/proto, rather than the installed version. This issue is being investigated. (Googlers see internal issue b/352040706)

anilkumarmyla · 2024-07-10T22:20:36Z

@parthea i'm able to reproduce the same using a docker container that has google-cloud-cli rpm package installed. In this setup I'm unable to reproduce if i'm using a venv, but can reproduce on the base system as below

[root@da99b97f53ef /]# python3 --version
Python 3.11.3
[root@da99b97f53ef /]# gcloud --verbosity=debug --version
DEBUG: Running [gcloud.version] with arguments: []
DEBUG: Chosen display Format:flattened[no-pad,separator=" "]
INFO: Display format: "flattened[no-pad,separator=" "]"
Google Cloud SDK 481.0.0
alpha 2024.06.14
beta 2024.06.14
bq 2.1.6
bundled-python3-unix 3.11.8
core 2024.06.14
gcloud-crc32c 1.0.0
gsutil 5.30
DEBUG: SDK update checks are disabled.
[root@da99b97f53ef /]# pip3 install --user proto-plus
Installing collected packages: protobuf, proto-plus
Successfully installed proto-plus-1.24.0 protobuf-5.27.2
[root@da99b97f53ef /]# gcloud --verbosity=debug --version
DEBUG: Running [gcloud.version] with arguments: []
DEBUG: Chosen display Format:flattened[no-pad,separator=" "]
INFO: Display format: "flattened[no-pad,separator=" "]"
DEBUG: module 'google._upb._message' has no attribute 'MessageMapContainer'
Traceback (most recent call last):
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 998, in Execute
    resources = calliope_command.Run(cli=self, args=args)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 817, in Run
    display_info=self.ai.display_info).Display()
                                       ^^^^^^^^^
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/calliope/display.py", line 487, in Display
    self._printer.Print(self._resources)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 299, in Print
    self.AddRecord(resources, delimit=intermediate)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 239, in AddRecord
    self._AddRecord(self._process_record(record), delimit)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py", line 517, in Evaluate
    return self._Project(obj, self._projection.Tree(), flag)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py", line 433, in _Project
    import proto  # pylint: disable=g-import-not-at-top
    ^^^^^^^^^^^^
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/__init__.py", line 15, in <module>
    from .enums import Enum
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/enums.py", line 19, in <module>
    from proto import _file_info
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/_file_info.py", line 24, in <module>
    from proto.marshal.rules.message import MessageRule
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/marshal/__init__.py", line 15, in <module>
    from .marshal import Marshal
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/marshal/marshal.py", line 25, in <module>
    from proto.marshal import compat
  File "/usr/lib64/google-cloud-sdk/lib/third_party/proto/marshal/compat.py", line 43, in <module>
    map_composite_types += (_message.MessageMapContainer,)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer'
ERROR: gcloud crashed (AttributeError): module 'google._upb._message' has no attribute 'MessageMapContainer'

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics

parthea · 2024-07-12T15:44:08Z

A fix was merged in proto-plus which handles the AttributeError. I expect this fix will make it into the next gcloud CLI release.

anilkumarmyla · 2024-07-12T20:14:10Z

@parthea thanks for the fix, do let us know when it makes it into the gcloud release

BenoitNivelles · 2024-09-23T12:29:06Z

I have a python script based on Gemini API SDK working ok till yesterday 2024-09-22
Today:
The "import google.generativeai as genai" results in the "Attribute error...."
My environment.

Windows
Python 3.12.3 in Spyder 5
google.generativeai 0.8.1 (conda-forge)
proto-plus 1.24.0
protobuf 5.28.1 (pypi)

@parthea I did your suggestion:

import proto
This command ends up like this:
....
from proto.marshal import compat

File ~\AppData\Local\anaconda3\envs\llm-poc\Lib\site-packages\proto\marshal\compat.py:60
map_composite_types += (_message.MessageMapContainer,)

AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer'

Any short term fix to allow the project development to resume.

Thank you.

Benoit

smithj51 added the untriaged auto added to all issues by default when created. label May 23, 2024

ianmclaughlin-po mentioned this issue May 28, 2024

Google cloud logging no longer compatible with protobuf 5.x googleapis/python-logging#899

Closed

honglooker removed the untriaged auto added to all issues by default when created. label Jun 12, 2024

diefans mentioned this issue Oct 24, 2024

Inconsistent protobuf versions kclvm->python client kcl-lang/lib#158

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer' on latest version #16935

AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer' on latest version #16935

smithj51 commented May 23, 2024

yhs0602 commented May 24, 2024 •

edited

Loading

ianmclaughlin-po commented May 25, 2024

doudoujay commented May 27, 2024

ianmclaughlin-po commented May 28, 2024

doudoujay commented May 28, 2024

calin-cocan commented May 31, 2024 •

edited

Loading

deannagarcia commented Jun 4, 2024

ianmclaughlin-po commented Jun 5, 2024

thlurte commented Jun 18, 2024

parthea commented Jul 4, 2024

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 10, 2024 •

edited

Loading

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 10, 2024

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 12, 2024

anilkumarmyla commented Jul 12, 2024

BenoitNivelles commented Sep 23, 2024

AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer' on latest version #16935

AttributeError: module 'google._upb._message' has no attribute 'MessageMapContainer' on latest version #16935

Comments

smithj51 commented May 23, 2024

yhs0602 commented May 24, 2024 • edited Loading

ianmclaughlin-po commented May 25, 2024

doudoujay commented May 27, 2024

ianmclaughlin-po commented May 28, 2024

doudoujay commented May 28, 2024

calin-cocan commented May 31, 2024 • edited Loading

deannagarcia commented Jun 4, 2024

ianmclaughlin-po commented Jun 5, 2024

thlurte commented Jun 18, 2024

parthea commented Jul 4, 2024

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 10, 2024 • edited Loading

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 10, 2024

anilkumarmyla commented Jul 10, 2024

parthea commented Jul 12, 2024

anilkumarmyla commented Jul 12, 2024

BenoitNivelles commented Sep 23, 2024

yhs0602 commented May 24, 2024 •

edited

Loading

calin-cocan commented May 31, 2024 •

edited

Loading

parthea commented Jul 10, 2024 •

edited

Loading