From eaa8f24aa626bf0432f8d137eed8e070850a6d98 Mon Sep 17 00:00:00 2001 From: Joshua Haberman Date: Fri, 3 Mar 2023 08:57:13 -0800 Subject: [PATCH] Updated obsolete Python README.md PiperOrigin-RevId: 513840426 --- python/README.md | 235 +++++++++++++++++++++-------------------------- 1 file changed, 105 insertions(+), 130 deletions(-) diff --git a/python/README.md b/python/README.md index 6549fb67000b..dc4b1d07ffb3 100644 --- a/python/README.md +++ b/python/README.md @@ -1,130 +1,105 @@ -Protocol Buffers - Google's data interchange format -=================================================== - -Copyright 2008 Google Inc. - -This directory contains the Python Protocol Buffers runtime library. - -Normally, this directory comes as part of the protobuf package, available -from: - - https://developers.google.com/protocol-buffers/ - -The complete package includes the C++ source code, which includes the -Protocol Compiler (protoc). If you downloaded this package from PyPI -or some other Python-specific source, you may have received only the -Python part of the code. In this case, you will need to obtain the -Protocol Compiler from some other source before you can use this -package. - -Development Warning -=================== - -The pure python performance is slow. For better performance please -use python c++ implementation. - -Installation -============ - -1) Make sure you have Python 3.7 or newer. If in doubt, run: - - $ python -V - -2) Make sure you have Bazel 0.5.4 or later (or CMake 3.5 or later). - -3) If you do not have setuptools installed, note that it will be - downloaded and installed automatically as soon as you run `setup.py`. - If you would rather install it manually, you may do so by following - the instructions on [this page](https://packaging.python.org/en/latest/tutorials/installing-packages/). - -4) Build the C++ code, or install a binary distribution of `protoc`. If - you install a binary distribution, make sure that it is the same - version as this package. If in doubt, run: - - $ protoc --version - -5) Build and run the tests: - - $ python setup.py build - $ python setup.py test - - To build, test, and use the C++ implementation, you must first compile - `libprotobuf.so` using either [Bazel](../README.md) or [CMake](../src/README.md): - - On OS X: - - If you are running a Homebrew-provided Python, you must make sure another - version of protobuf is not already installed, as Homebrew's Python will - search `/usr/local/lib` for `libprotobuf.so` before it searches the compiled - binaries. - - You can either unlink Homebrew's protobuf or install the `libprotobuf` you - built earlier: - - $ brew unlink protobuf - - or - - $ (cd .. && cmake . && make install) - - On other *nix: - - You must make `libprotobuf.so` dynamically available. You can either - install libprotobuf you built earlier, or set `LD_LIBRARY_PATH`: - - $ (cd .. && cmake . && make -j20 install) - - or - - $ export LD_LIBRARY_PATH=../bazel-bin - - To build the C++ implementation run: - - $ python setup.py build --cpp_implementation - - Then run the tests like so: - - $ python setup.py test --cpp_implementation - - If some tests fail, this library may not work correctly on your - system. Continue at your own risk. - - Please note that there is a known problem with some versions of - Python on Cygwin which causes the tests to fail after printing the - error: `sem_init: Resource temporarily unavailable`. This appears - to be a [bug either in Cygwin or in - Python](http://www.cygwin.com/ml/cygwin/2005-07/msg01378.html). - - We do not know if or when it might be fixed. We also do not know - how likely it is that this bug will affect users in practice. - -6) Install: - - $ python setup.py install - - or: - - $ (cd .. && make install) - $ python setup.py install --cpp_implementation - - This step may require superuser privileges. - NOTE: To use C++ implementation, you need to export an environment - variable before running your program. See the "C++ Implementation" - section below for more details. - -Usage -===== - -The complete documentation for Protocol Buffers is available via the -web at: - - https://developers.google.com/protocol-buffers/ - -C++ Implementation -================== - -The C++ implementation for Python messages is built as a Python extension to -improve the overall protobuf Python performance. - -To use the C++ implementation, you need to install the C++ protobuf runtime -library, please see instructions in the parent directory. +# Protocol Buffers Python + +This directory contains the Protobuf library for Python. + +In most cases you should install the library using `pip` or another package +manager: + +``` +$ pip install protobuf +``` + +The packages released on https://pypi.org/project/protobuf/#files include both a +source distribution and binary wheels. + +For user documentation about how to use Protobuf Python, see +https://protobuf.dev/getting-started/pythontutorial/ + +# Building packages from this repo + +If for some reason you wish to build the packages directly from this repo, you +can use the following Bazel commands: + +``` +$ bazel build @upb//python/dist:source_wheel +$ bazel build @upb//python/dist:binary_wheel +``` + +The binary wheel will build against whatever version of Python is installed on +your system. The source package is always the same and does not depend on a +local version of Python. + +# Implementation backends + +There are three separate implementations of Python Protobuf. All of them offer +the same API and are thus functionally the same, though they have very different +performance characteristics. + +The runtime library contains a switching layer that can choose between these +backends at runtime. Normally it will choose between them automatically, using +priority-ordered list, and skipping any backends that are not available. However +you can request a specific backend by setting the +`PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION` environment variable to one of the +following values: + +1. **upb**: Built on the + [upb C library](https://github.com/protocolbuffers/upb), this is a new + extension module + [released in 4.21.0](https://protobuf.dev/news/2022-05-06/). It offers + better performance than any of the previous backends, and it is now the + default. It is distributed in our PyPI packages, and requires no special + installation. The code for this module lives in + [@upb/python](https://github.com/protocolbuffers/upb/tree/main/python). +1. **cpp**: This extension module wraps the C++ protobuf library. It is + deprecated and is no longer released in our PyPI packages, however it is + still used in some legacy cases where apps want to perform zero-copy message + sharing between Python and C++. It must be installed separately before it + can be used. The code for this module lives in + [google/protobuf/pyext](https://github.com/protocolbuffers/protobuf/tree/main/python/google/protobuf/pyext). +1. **python**: The pure-Python backend, this does not require any extension + module to be present on the system. The code for the pure-Python backend + lives in [google/protobuf/internal](google/protobuf/internal) + +The order given above is the general priority order, with `upb` being preferred +the most and the `python` backend preferred the least. However this ordering can +be overridden by the presence of a +`google.protobuf.internal._api_implementation` module. See the logic in +[api_implementation.py](https://github.com/protocolbuffers/protobuf/blob/main/python/google/protobuf/internal/api_implementation.py) +for details. + +You can check which backend you are using with the following snippet: + +``` +$ python +Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> from google.protobuf.internal import api_implementation +>>> print(api_implementation.Type()) +upb +``` + +This is not an officially-supported or stable API, but it is useful for ad hoc +diagnostics. + +More information about sharing messages between Python and C++ is available +here: https://protobuf.dev/reference/python/python-generated/#sharing-messages + +# Code generator + +The code for the Protobuf Python code generator lives in +[//src/google/protobuf/compiler/python](https://github.com/protocolbuffers/protobuf/tree/main/src/google/protobuf/compiler/python). +The code generator can output two different files for each proto `foo.proto`: + +* **foo_pb2.py**: The module you import to actually use the protos. +* **foo_pb2.pyi**: A stub file that describes the interface of the protos. + +The `foo_pb2.pyi` file is useful for IDEs or for users who want to read the +output file. The `foo_pb2.py` file is optimized for fast loading and is not +readable at all. + +Note that the pyi file is only generated if you pass the `pyi_out` option to +`protoc`: + +``` +$ protoc --python_out=pyi_out:output_dir +```