Enable alternative build system for Python in source builds

[bkmgit]

What language does this apply to?
Python bindings

Describe the problem you are trying to solve.
Bazel is not packaged in some Linux distributions because it bundles a number of jar files and binaries, as such cannot be used to build the Python bindings from source release.

Describe the solution you'd like
Enable CMake as an additional build system for the Python bindings.

Describe alternatives you've considered
Meson or Muon are other options, though this would add to the number of build systems already in use.

Additional context
Can make a pull request if this would be considered.

[zhangskz]

We generally lean towards supporting build-from-source through Bazel rather than per-language build systems to keep our support surfaces maintainable. For example, we recently dropped support for building Java from source via Maven accordingly.

Its unlikely we would be able to support this without substantial demand, esp. given CMake doesn't seem to be a commonly used build system for Python. We would likely only be able to accept this as a community-maintained contribution (if at all).

Could you instead simply install Bazel directly via Bazelisk, or request support for Bazel in these Linux distributions as-needed? I would like to understand if we could solve the usability of Bazel for Python Protobuf in these cases instead.

[bkmgit]

The points below apply to Fedora Linux

• It is difficult to build Bazel from source that does not contain any pre-compiled code. Early versions of Bazel seemed to allow this, so one could try and bootstrap Bazel starting from early releases to build successive releases, see https://docs.fedoraproject.org/en-US/packaging-guidelines/what-can-be-packaged/#prebuilt-binaries-or-libraries
• All jar files should be compiled from source, see https://docs.fedoraproject.org/en-US/packaging-guidelines/Java/#_pre_built_dependencies thus some cleanup is required of Git sources

[zhangskz]

I'm not familiar with the specifics of Fedora, but it looks like the recommended solution from Bazel is to use the unofficial vbatts/bazel package per https://docs.bazel.build/versions/5.4.1/install-redhat.html and https://docs.bazel.build/versions/5.4.1/install.html#community-supported-packages

Looks like there is community support that should allow Bazel to work with Fedora Linux, but Bazel folks may be able to better point you in the right direction: https://github.com/bazelbuild/bazel/issues

[bkmgit]

Copr is not reviewed so cannot be used to add official packages in the main distribution. As indicated in bazelbuild/bazel#18550 new releases on Copr are not forthcoming.

As many things depend on protobuf, this is holding up many updates. Fortunately there are other implementations, but patching to use ohter implementations would take some time.

Have not checked other build systems such as Pants, Buck2 or Please, but if one of these might be easier to support than CMake, that would be useful to know.

There was a previous thread on supporting Bazel in Fedora bazelbuild/bazel#18550

[zhangskz]

Which protobuf python implementation (e.g. PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION) is your request to support building from CMake for? Is this specific to Protobuf-C++, or other implemetnations? Note that the default implementation is upb per https://protobuf.dev/news/2022-05-06/#python-updates

Realistically, I believe looking into improving Bazel support in Fedora is probably still the right way to proceed here feasibility may vary depending on which python implementation(s) this is for.

[bkmgit]

Adding Bazel support would take some time. There are many jar files bundled with Bazel that would need to be rebuilt. Bootstrapping Bazel would also take time.