-
Notifications
You must be signed in to change notification settings - Fork 15.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python: support for optional python_package (or similar) to make import explicit #7061
Comments
I this this is a duplicate of #2283 I also really need it. For now i think we have todo rewriting of the imports after the code has been generated. |
I face the same issue where the compiled proto files generated have the wrong import statements. |
Very old problem and no fix expected |
I'm also interested in this. |
Hi, Any progress on this issue? |
same problem here. 🤦♂️ |
same problem |
We also have the same problem... |
Is there some automated viable workaround? It seems like a necessity to conform to language guidelines and restrictions. Instead of renaming hyphen to underscore automagically, I would vote it be explicit. I'm seeing it break down with other tools, i.e. bazel, and am having to fork and rename packages. |
@daicoden at the moment the only "automated" workaround I'm using is some bash vodoo to edit what I need after compiling the protobufs. Something along this: sed -i 's/^import \(.\+\) as/from . import \1 as/' some/pkg/protobufs/*.py |
This code works for me for relative imports in the generated package
|
Another solution could be to have an option to generate all Python code in a single module, as this would solve the "absolute imports between modules" issue, that I personally solve using the sed trick mentioned above. |
Likewise, this is a challenge. Without firm convention and support here, interoperability within our greatly multi-lingual system is strained by competing conventions and workarounds. Editing the file that boldly says "Do Not Edit" asks for trouble. |
* We're experiencing issues related to Python import paths in the generated protobuf and grpc code. See protocolbuffers/protobuf#7061 Co-authored-by: edouard-sn <58398928+edouard-sn@users.noreply.github.com> Signed-off-by: Diego ROJAS <rojasdiegopro@gmail.com>
I'm trying to work with https://github.com/cloudevents/spec/blob/3da5643ebceb39637406a7e30903dbac81cf92d2/cloudevents/formats/cloudevents.proto package io.cloudevents.v1;
// ...
message CloudEvent {
// ...
}
// ... When using protoc this puts python code in This then conflicts with the python builtin io module AFAICT: https://docs.python.org/3/library/io.html It would be great to have some solution to this. |
I setup an example illustrating this problem https://github.com/aucampia/issue-20221116-python_protobuf_cloudevents/tree/da10645893a3085c5bd76642f12aef67a222266a |
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment. This issue is labeled |
This issue is still relevant and needs a solution. |
Another way to approach this is to allow customizing the module prefix. The module name is parsed by this helper: protobuf/src/google/protobuf/compiler/python/helpers.cc Lines 31 to 35 in 643b0a6
I suggested a similar fix on grpclib, see vmagamedov/grpclib#196 . |
Update: I made a PR to support a custom module prefix in the Python imports for protobuf but I'm not familiar with the edge cases of such a feature. |
What language does this apply to?
Python.
Describe the problem you are trying to solve.
The Python code generator uses the directory structure of the
.proto
files to generate the generated Python code directory layout and imports (e.g. when Protobuf file A imports Protobuf file B). It doesn't sanitize Python reserved keywords.This can be problematic in two cases:
The directory structure contains a reserved keyword. For example when it contains
as
. In pythonimport as
fails but alsoimport as.foo.bar
is invalid.When in case of A depending on B, A will contain an absolute import that might not be on the Python path (e.g. when the generated code is packaged so instead of
import b
it should doimport mypackage.b
). This was mentioned as a fix in the discussions: python: use relative imports in generated modules #1491, but I would consider this a workaround.Describe the solution you'd like
For several languages, there are ways to make the package / module name explicit by defining an
option
, e.g.option go_package
,option java_package
, ... (https://developers.google.com/protocol-buffers/docs/proto3#packages). (I believe there is also aruby_package
#4627).I believe an optional
option python_package
could solve the above issues as aas/api.proto
file then could contain anoption python_package = "as_pb.api"
(or"my_package.as_pb.api"
).The Protobuf code generator will then take the
python_package
into account when:import as_pb.api
(as_pb/api.py
).python_package
.Describe alternatives you've considered
My initial approach was to re-define the structure of the original Protobuf files. However, when the Protobuf files are used to generate other languages bindings, this has side-effects (e.g. the Go code now ends up in
as_pb/api
, it should be inas/api
.I believe the supported languages should not be leading in how the Protobuf structure looks like.
Additional context
(https://www.python.org/dev/peps/pep-0020/)
😉
See also brocaar/chirpstack-api#5 .
The text was updated successfully, but these errors were encountered: