Skip to content
This repository has been archived by the owner on Feb 17, 2023. It is now read-only.

Commit

Permalink
Allow relative imports when entire prefix is stripped.
Browse files Browse the repository at this point in the history
When running under Python 3, PEP 0328 formalized `import package` as an
absolute import. This PEP introduced a new syntax, `import .package` as
a relative import (of a "sub" package). Finally, the syntax
`from . import bar` was made to import a "peer" package.

Python gRPC code generated by the gRPC compiler uses the gRPC package name
as the absolute import, which works great if you own that particular namespace,
and can afford to allocate that namespace to that protobuf globally, or are running
under the (now EOL) Python 2.

There are some existing bugs (e.g. protocolbuffers/protobuf#881)
on GitHub and elsewhere on the internet that outline this issue, and tend to be
closed WONTFIX or people work around it by sed'ing the output protobuf files.

This change allows the use of the prefixes (and a previously broken codepath)
to work around this bug by doing something that semantically feels right-ish.

The gRPC Python plugin accepts arguments to strip prefixes via the
`--grpc_python_out` flag. For example:

    --grpc_python_out=grpc_2_0,mycorp.:.

will cause the generated imports to become:

    `from myservice.v1 import baz`

rather than

    `from mycorp.myservice.v1 import baz`

Now, when when this is taken to the extreme, and the *entire* prefix except
for the final dot is stripped -- for instance:

    --grpc_python_out=grpc_2_0,mycorp.myservice.v1:.

THe existing gRPC plugin will generate the (buggy) output of:

    `from  import baz`

This patch changes the behavior of that (buggy) output to become:

    `from . import baz`

Which will allow the import of a peer package under Python 3 without having
to exist in a global namespace that matches the protobuf package tree.
  • Loading branch information
paultag committed Feb 13, 2020
1 parent a28d67f commit abb0b21
Show file tree
Hide file tree
Showing 14 changed files with 21 additions and 15 deletions.
10 changes: 8 additions & 2 deletions src/compiler/python_generator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -625,8 +625,14 @@ bool PrivateGenerator::PrintPreamble(grpc_generator::Printer* out) {
if (last_dot_pos == grpc::string::npos) {
var["ImportStatement"] = "import " + module_name;
} else {
var["ImportStatement"] = "from " + module_name.substr(0, last_dot_pos) +
" import " +
auto import_module = module_name.substr(0, last_dot_pos);
if (import_module.size() == 0) {
// If `strip_prefixes` is set, and the entire prefix is stripped
// up to (but not including!) the dot, we'll leave the dot as
// written, which will turn this import into a relaitive import.
import_module = ".";
}
var["ImportStatement"] = "from " + import_module + " import " +
module_name.substr(last_dot_pos + 1);
}
out->Print(var, "$ImportStatement$ as $ModuleAlias$\n");
Expand Down
2 changes: 1 addition & 1 deletion third_party/abseil-cpp
Submodule abseil-cpp updated 48 files
+6 −3 CONTRIBUTING.md
+144 −0 FAQ.md
+1 −0 absl/CMakeLists.txt
+0 −18 absl/abseil.podspec.gen.py
+5 −1 absl/base/BUILD.bazel
+2 −0 absl/base/CMakeLists.txt
+10 −4 absl/base/config.h
+30 −9 absl/base/internal/atomic_hook.h
+6 −3 absl/base/internal/atomic_hook_test.cc
+2 −1 absl/base/internal/atomic_hook_test_helper.cc
+7 −5 absl/base/internal/raw_logging.cc
+2 −1 absl/base/internal/raw_logging.h
+2 −2 absl/base/internal/spinlock.cc
+2 −2 absl/base/internal/sysinfo_test.cc
+5 −0 absl/base/log_severity_test.cc
+13 −19 absl/container/btree_test.cc
+215 −214 absl/container/internal/btree.h
+3 −3 absl/container/internal/btree_container.h
+2 −2 absl/container/node_hash_set.h
+11 −2 absl/debugging/failure_signal_handler.cc
+1 −1 absl/debugging/symbolize.h
+3 −1 absl/flags/BUILD.bazel
+1 −0 absl/flags/CMakeLists.txt
+8 −0 absl/flags/config.h
+0 −7 absl/flags/flag.cc
+6 −42 absl/flags/flag.h
+10 −9 absl/flags/flag_test.cc
+18 −4 absl/flags/internal/commandlineflag.h
+47 −87 absl/flags/internal/flag.cc
+250 −190 absl/flags/internal/flag.h
+9 −12 absl/flags/internal/registry.cc
+72 −0 absl/functional/CMakeLists.txt
+4 −4 absl/random/distributions.h
+0 −1 absl/strings/charconv.cc
+14 −0 absl/strings/internal/resize_uninitialized_test.cc
+1 −2 absl/strings/internal/str_format/bind.cc
+12 −12 absl/strings/internal/str_format/bind_test.cc
+0 −24 absl/strings/internal/str_format/extension.cc
+0 −51 absl/strings/internal/str_format/extension.h
+32 −5 absl/strings/internal/str_format/parser.cc
+10 −3 absl/strings/internal/str_format/parser.h
+26 −6 absl/strings/internal/str_format/parser_test.cc
+19 −11 absl/strings/string_view.h
+34 −0 absl/strings/string_view_benchmark.cc
+23 −0 absl/strings/string_view_test.cc
+10 −7 absl/synchronization/mutex.cc
+2 −2 absl/time/clock.cc
+1 −0 ci/macos_xcode_cmake.sh
2 changes: 1 addition & 1 deletion third_party/bloaty
Submodule bloaty updated 145 files
2 changes: 1 addition & 1 deletion third_party/boringssl-with-bazel
2 changes: 1 addition & 1 deletion third_party/cares/cares
Submodule cares updated 58 files
+19 −3 CMakeLists.txt
+11 −0 Makefile.inc
+1 −0 README.md
+1 −1 acountry.c
+2 −2 adig.c
+1 −1 ahost.c
+59 −0 ares.h
+2 −2 ares__close_sockets.c
+266 −0 ares__parse_into_addrinfo.c
+261 −0 ares__readaddrinfo.c
+494 −0 ares__sortaddrinfo.c
+0 −2 ares_android.c
+6 −0 ares_config.h.cmake
+4 −4 ares_create_query.c
+37 −0 ares_freeaddrinfo.3
+57 −0 ares_freeaddrinfo.c
+195 −0 ares_getaddrinfo.3
+765 −0 ares_getaddrinfo.c
+0 −2 ares_getenv.c
+5 −4 ares_gethostbyname.c
+7 −2 ares_init.c
+4 −4 ares_init_options.3
+7 −0 ares_ipv6.h
+35 −48 ares_library_init_android.3
+2 −0 ares_parse_a_reply.3
+126 −179 ares_parse_a_reply.c
+2 −0 ares_parse_aaaa_reply.3
+126 −176 ares_parse_aaaa_reply.c
+41 −1 ares_private.h
+40 −38 ares_process.c
+5 −7 ares_search.c
+3 −3 ares_version.h
+1 −0 c-ares-config.cmake.in
+140 −0 m4/cares-functions.m4
+1 −1 m4/xc-cc-check.m4
+4 −0 test/CMakeLists.txt
+3 −1 test/Makefile.inc
+6 −14 test/README.md
+57 −0 test/ares-test-ai.h
+18 −0 test/ares-test-init.cc
+105 −4 test/ares-test-internal.cc
+67 −2 test/ares-test-live.cc
+9 −0 test/ares-test-misc.cc
+723 −0 test/ares-test-mock-ai.cc
+1 −1 test/ares-test-mock.cc
+74 −0 test/ares-test.cc
+60 −3 test/ares-test.h
+1 −0 test/fuzzcheck.sh
+ test/fuzzinput/clusterfuzz-5650695891451904
+ test/fuzzinput/clusterfuzz-5651369832218624
+ test/fuzzinput/clusterfuzz-5674462260756480
+ test/fuzzinput/clusterfuzz-5680630672654336
+ test/fuzzinput/clusterfuzz-5683497160671232
+ test/fuzzinput/clusterfuzz-5687310655422464
+ test/fuzzinput/clusterfuzz-5695341573177344
+ test/fuzzinput/clusterfuzz-5697835103682560
+ test/fuzzinput/clusterfuzz-5728518081609728
+ test/fuzzinput/clusterfuzz-5732960017317888
2 changes: 1 addition & 1 deletion third_party/envoy-api
Submodule envoy-api updated 588 files
2 changes: 1 addition & 1 deletion third_party/gflags
2 changes: 1 addition & 1 deletion third_party/googleapis
Submodule googleapis updated 2628 files
2 changes: 1 addition & 1 deletion third_party/googletest
Submodule googletest updated 121 files
2 changes: 1 addition & 1 deletion third_party/libuv
Submodule libuv updated 91 files
+9 −2 .github/stale.yml
+1 −0 AUTHORS
+243 −182 CMakeLists.txt
+91 −1 ChangeLog
+0 −1 Makefile.am
+0 −3 README.md
+3 −2 common.gypi
+1 −4 configure.ac
+1 −1 docs/src/design.rst
+11 −11 docs/src/fs.rst
+1 −1 docs/src/fs_event.rst
+3 −3 docs/src/guide/about.rst
+3 −3 docs/src/guide/introduction.rst
+7 −7 docs/src/guide/networking.rst
+27 −20 docs/src/guide/processes.rst
+6 −8 docs/src/guide/threads.rst
+7 −7 docs/src/guide/utilities.rst
+4 −4 docs/src/index.rst
+16 −7 docs/src/misc.rst
+8 −3 docs/src/pipe.rst
+1 −1 docs/src/sphinx-plugins/manpage.py
+ docs/src/static/diagrams.key/Data/st0-311.jpg
+ docs/src/static/diagrams.key/Data/st1-475.jpg
+5 −0 docs/src/tcp.rst
+1 −1 docs/src/version.rst
+3 −3 include/uv/version.h
+0 −22 samples/.gitignore
+0 −21 samples/socks5-proxy/.gitignore
+0 −53 samples/socks5-proxy/LICENSE
+0 −46 samples/socks5-proxy/Makefile
+0 −46 samples/socks5-proxy/build.gyp
+0 −736 samples/socks5-proxy/client.c
+0 −139 samples/socks5-proxy/defs.h
+0 −131 samples/socks5-proxy/getopt.c
+0 −99 samples/socks5-proxy/main.c
+0 −271 samples/socks5-proxy/s5.c
+0 −94 samples/socks5-proxy/s5.h
+0 −241 samples/socks5-proxy/server.c
+0 −72 samples/socks5-proxy/util.c
+0 −180 src/unix/aix-common.c
+180 −0 src/unix/aix.c
+13 −71 src/unix/async.c
+36 −88 src/unix/core.c
+73 −76 src/unix/darwin-proctitle.c
+3 −3 src/unix/darwin.c
+4 −4 src/unix/freebsd.c
+54 −21 src/unix/fs.c
+4 −1 src/unix/fsevents.c
+243 −16 src/unix/ibmi.c
+3 −4 src/unix/internal.h
+14 −4 src/unix/linux-core.c
+20 −47 src/unix/linux-inotify.c
+0 −190 src/unix/linux-syscalls.c
+0 −65 src/unix/linux-syscalls.h
+4 −4 src/unix/netbsd.c
+15 −15 src/unix/openbsd.c
+5 −1 src/unix/pipe.c
+35 −43 src/unix/process.c
+42 −27 src/unix/proctitle.c
+2 −2 src/unix/random-devurandom.c
+1 −0 src/unix/signal.c
+6 −5 src/unix/stream.c
+10 −2 src/unix/tcp.c
+8 −1 src/unix/tty.c
+2 −2 src/unix/udp.c
+7 −2 src/win/core.c
+4 −0 src/win/pipe.c
+12 −58 src/win/tty.c
+14 −33 src/win/winsock.c
+6 −2 test/benchmark-multi-accept.c
+14 −0 test/run-tests.c
+4 −3 test/runner-unix.c
+4 −13 test/runner-win.c
+8 −3 test/runner.c
+1 −1 test/task.h
+4 −0 test/test-dlerror.c
+6 −0 test/test-env-vars.c
+3 −0 test/test-fs-copyfile.c
+9 −1 test/test-fs-event.c
+36 −15 test/test-fs.c
+5 −0 test/test-get-memory.c
+2 −0 test/test-list.h
+4 −0 test/test-platform-output.c
+2 −4 test/test-poll-close-doesnt-corrupt-stack.c
+2 −3 test/test-poll-closesocket.c
+6 −2 test/test-process-title-threadsafe.c
+61 −1 test/test-process-title.c
+1 −0 test/test-thread.c
+55 −4 test/test-udp-ipv6.c
+2 −0 test/test-udp-multicast-join.c
+1 −2 uv.gyp
2 changes: 1 addition & 1 deletion third_party/protobuf
2 changes: 1 addition & 1 deletion third_party/protoc-gen-validate
2 changes: 1 addition & 1 deletion third_party/udpa
Submodule udpa updated 39 files
+1 −0 .bazelrc
+1 −0 .bazelversion
+4 −0 .gitattributes
+1 −0 .gitignore
+10 −0 DEVELOPER.md
+75 −78 bazel/api_build_system.bzl
+2 −2 bazel/dependency_imports.bzl
+5 −0 bazel/repositories.bzl
+6 −0 bazel/repository_locations.bzl
+24 −0 ci/azure-pipelines.yml
+15 −0 ci/check.sh
+9 −0 go/go.mod
+231 −0 go/udpa/annotations/migrate.pb.go
+246 −0 go/udpa/annotations/migrate.pb.validate.go
+50 −0 go/udpa/annotations/sensitive.pb.go
+37 −0 go/udpa/annotations/sensitive.pb.validate.go
+93 −0 go/udpa/annotations/status.pb.go
+104 −0 go/udpa/annotations/status.pb.validate.go
+94 −0 go/udpa/annotations/versioning.pb.go
+106 −0 go/udpa/annotations/versioning.pb.validate.go
+129 −0 go/udpa/data/orca/v1/orca_load_report.pb.go
+134 −0 go/udpa/data/orca/v1/orca_load_report.pb.validate.go
+201 −0 go/udpa/service/orca/v1/orca.pb.go
+114 −0 go/udpa/service/orca/v1/orca.pb.validate.go
+92 −0 go/udpa/type/v1/typed_struct.pb.go
+114 −0 go/udpa/type/v1/typed_struct.pb.validate.go
+1 −0 repokitteh.star
+5 −6 test/build/BUILD
+4 −3 test/build/go_build_test.go
+36 −0 tools/generate_go_protobuf.py
+5 −0 udpa/annotations/BUILD
+49 −0 udpa/annotations/migrate.proto
+14 −0 udpa/annotations/sensitive.proto
+16 −0 udpa/annotations/status.proto
+17 −0 udpa/annotations/versioning.proto
+1 −7 udpa/data/orca/v1/BUILD
+1 −10 udpa/service/orca/v1/BUILD
+5 −0 udpa/type/v1/BUILD
+42 −0 udpa/type/v1/typed_struct.proto

0 comments on commit abb0b21

Please sign in to comment.