This repository has been archived by the owner on Apr 19, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 99
Refresh Python module #176
Closed
Closed
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
602cf28
Refresh Python module
Dreamsorcerer 0b3ec5f
Update test_client.py
Dreamsorcerer 4d0249c
Update test_client.py
Dreamsorcerer 0803966
Update min versions
Dreamsorcerer 51597aa
Update script instead of README
Dreamsorcerer 325c8a2
Update tests
Dreamsorcerer 43f2556
Merge master
Dreamsorcerer fa89859
Bump dependency and improve compatibility of script
Dreamsorcerer ab2d154
Merge master
Dreamsorcerer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
pb2 files can be regenerated by running: | ||
|
||
``` | ||
python3 -m grpc_tools.protoc -I../proto/ --python_out=gubernator/ --pyi_out=gubernator/ --grpc_python_out=gubernator/ --mypy_grpc_out=gubernator/ ../proto/*.proto | ||
sed -i '' 's/import gubernator_pb2/from gubernator import gubernator_pb2/' gubernator/gubernator_pb2_grpc.py* | ||
sed -i '' 's/import gubernator_pb2/from gubernator import gubernator_pb2/' gubernator/peers_pb2.py* | ||
sed -i '' 's/import peers_pb2/from gubernator import peers_pb2/' gubernator/peers_pb2_grpc.py* | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,8 @@ | ||
# This code is py3.7 and py2.7 compatible | ||
|
||
import gubernator.ratelimit_pb2_grpc as pb_grpc | ||
from datetime import datetime | ||
|
||
import time | ||
import grpc | ||
|
||
MILLISECOND = 1 | ||
SECOND = MILLISECOND * 1000 | ||
MINUTE = SECOND * 60 | ||
|
||
|
||
def sleep_until_reset(reset_time): | ||
now = datetime.now() | ||
time.sleep((reset_time - now).seconds) | ||
|
||
|
||
def V1Client(endpoint='127.0.0.1:9090'): | ||
channel = grpc.insecure_channel(endpoint) | ||
return pb_grpc.RateLimitServiceV1Stub(channel) | ||
Dreamsorcerer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
from gubernator.gubernator_pb2 import ( | ||
Algorithm, Behavior, GetRateLimitsReq, RateLimitReq, Status | ||
) | ||
from gubernator.gubernator_pb2_grpc import V1Stub | ||
from gubernator.peers_pb2_grpc import PeersV1Stub | ||
|
||
__all__ = ("Algorithm", "Behavior", "GetRateLimitsReq", "RateLimitReq", "Status", | ||
"PeersV1Stub", "V1Stub") |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you move this stuff to a bash script in
proto
directory? And include apip install
for the tools.And preferably we shouldn't need post-processing with
sed
commands. Looks like it's a matter of getting CWD or input paths right to fix that.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Literally spent a couple of hours looking at this. In summary, it appears that Google have no interest in supporting modules which would require relative imports. It might be possible to get something to work by changing the proto directory, but I figured that might break things elsewhere. Feel free to have a go yourself though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
protocolbuffers/protobuf#1491
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you can probably tell, we don't use this Python functionality anymore. At least, not for Python 3. I'm thinking the entire Python client needs to be reimplemented from protos and disregard any consideration for legacy requirements.
I committed some changes to my branch for a
genproto.sh
script that already exists but needed some cleaning up. It does generate the code for Go and Python ok. There's some old unit tests inpython/tests
, but no idea how compatible that is to now current code. It fails, as you might expect.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
sed
calls are not related to legacy (in fact, it apparently worked on Python 2 without rewriting the imports). It is because Google apparently just dumps all (thousands) of their files in a top-level directory that is on their PYTHONPATH.For an installable package, we can't expect users to change their PYTHONPATH to point to the package's source code (I know that sound ridiculous, but I literally don't understand why the protobuf maintainers in that thread don't appear to understand that). So,
sed
rewrites the imports to import from the package (i.e. it could be done either asfrom gubernator
orfrom .
).To test, run
pip install .
in your python directory, then (ensuring you're not in the source directory) open python andimport gubernator
. Without thesed
changes you'll get an import error.I'll go through the other changes next week.