Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v0.8.0 #139

Merged
merged 115 commits into from
Feb 16, 2019
Merged
Changes from 1 commit
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
fe76445
Merge pull request #128 from prkumar/stable
prkumar Dec 7, 2018
aca2522
Bump minor version
prkumar Dec 7, 2018
c489896
Deploy development versions to PyPI
prkumar Dec 7, 2018
f4d5f6b
DEPLOY: first development version to PyPI
prkumar Dec 7, 2018
b840174
DEPLOY: first development version to PyPI
prkumar Dec 7, 2018
fb6ba55
DEPLOY: try first development version to PyPI
prkumar Dec 7, 2018
d6e56f1
DEPLOY: try first development version to PyPI
prkumar Dec 7, 2018
85971f5
Update deployment criteria
prkumar Dec 7, 2018
2fb81c1
Compute deployment status after successful build
prkumar Dec 7, 2018
c79176a
Fix deployment script
prkumar Dec 7, 2018
2e49876
Fix deployment script
prkumar Dec 8, 2018
6cb37fa
Fix deployment logic
prkumar Dec 8, 2018
230ddb1
Verify tagged commits before they are deployed
prkumar Dec 8, 2018
818026e
Fix reference to script in .travis.yml
prkumar Dec 8, 2018
1da55d2
Deploy only on tagged commits
prkumar Dec 8, 2018
764449c
Add initial backend implementation
prkumar Dec 17, 2018
33315ab
Change package name from backend to io
prkumar Dec 17, 2018
4412381
Update documentation
prkumar Dec 19, 2018
067a17f
Add default and composite templates
prkumar Dec 19, 2018
4f76a8c
Update documentation
prkumar Dec 19, 2018
a7c8005
Fix B008 violation
prkumar Dec 19, 2018
266fbad
Fix broken logic with transitions
prkumar Dec 20, 2018
239fd59
Implement default, blocking strategy
prkumar Dec 20, 2018
775ba2b
Rewrite PreparedRequest to use the io backend
prkumar Dec 21, 2018
f481730
Add initial retry decorator
prkumar Dec 21, 2018
aef68de
Remove reference to math.inf
prkumar Dec 21, 2018
4fe3a75
Update requests to 2.20.0
prkumar Dec 21, 2018
97dde7d
Implement next to satisfy Iterator interface
prkumar Dec 21, 2018
2eba20f
Revert "Update requests to 2.20.0"
prkumar Dec 21, 2018
a129f64
Clean up retry implementation
prkumar Dec 22, 2018
56718a1
Implement ratelimit decorator
prkumar Dec 22, 2018
6119250
Merge branch 'master' into io
prkumar Dec 23, 2018
626b631
Add test for within rate limit
prkumar Dec 23, 2018
36730f1
Do not include io interfaces in coverage
prkumar Dec 26, 2018
f9cb87b
Rename classes to more suitable names
prkumar Dec 26, 2018
49f8216
Rename ExecutionStrategy to IOStrategy
prkumar Dec 26, 2018
55342cb
remove_ref_to_end_session
SakornW Dec 26, 2018
914797c
Add unit tests
prkumar Dec 26, 2018
ef71adf
Fix unit tests
prkumar Dec 26, 2018
3c6d075
Add docstring to retry decorator
prkumar Dec 27, 2018
c2b7297
Fix unit test for retry decorator
prkumar Dec 27, 2018
7da0326
Update docs for retry decorator
prkumar Dec 28, 2018
386776c
Update docs for ratelimit decorator
prkumar Dec 28, 2018
f093efa
Add expected exception to retry decorator
prkumar Dec 28, 2018
fa20b0f
Update unit tests for retry decorator
prkumar Dec 28, 2018
5696ca4
Update docs for retry decorator
prkumar Dec 28, 2018
b861e0f
Fix broken python 2.7 build
prkumar Dec 28, 2018
30cda09
Fix test coverage for aiohttp
prkumar Dec 28, 2018
a85074c
Fix test coverage for aiohttp
prkumar Dec 28, 2018
cef9de3
Skip line in retry.jitter w/ weird coverage
prkumar Dec 28, 2018
5e28d37
Merge pull request #132 from prkumar/io
prkumar Dec 28, 2018
9ba2558
Bump alpha version
prkumar Dec 28, 2018
6249749
Add requirements file for building docs
prkumar Dec 28, 2018
2b818f4
Update requirements file for building docs
prkumar Dec 29, 2018
c7bfdca
Update requirements file for building docs
prkumar Dec 29, 2018
18ecc06
Update requirements file for building docs
prkumar Dec 29, 2018
ca6dcf9
Update requirements file for building docs
prkumar Dec 29, 2018
d88c36f
Implement timeout argument
daa Dec 29, 2018
f37bdb0
add identity converter
daa Dec 29, 2018
0d2028a
rework identity converter key
daa Dec 29, 2018
08647ce
add Timeout to __all__
daa Dec 29, 2018
955ea7c
add Timeout link to dev docs
daa Dec 29, 2018
f03cbb6
fix case
daa Dec 29, 2018
8f74cfa
Rename when_raises argument to on_exception
prkumar Dec 29, 2018
956a7ce
Rearrange the retry impl
prkumar Dec 30, 2018
dbc61fc
Update API docs for retry decorator
prkumar Dec 30, 2018
00a63ac
Add retry decorator to quickstart
prkumar Dec 30, 2018
be6cbe4
Add raise_on_limit argument to ratelimit
prkumar Dec 30, 2018
e3cb876
Add ratelimit decorator to quickstart
prkumar Dec 30, 2018
f4d9648
Bump alpha version
prkumar Dec 30, 2018
782f07d
Fix mistake in quickstart
prkumar Dec 30, 2018
e7626ed
move session.close from atexit to __del__()
Jan 2, 2019
e016e37
fix flake8 validation
Jan 2, 2019
01bfd38
Remove catching AttributeError during __del__
Jan 3, 2019
c222663
Merge pull request #133 from daa/timeout-argument
prkumar Jan 4, 2019
1c3067a
close session only if it's auto created
Jan 4, 2019
7bd0978
Merge pull request #134 from SakornW/bugfix/session_mem_leak
prkumar Jan 4, 2019
f4d1ac7
Enhance retry decorator
prkumar Jan 8, 2019
64952e3
Pin pytest below 4.1 to protect python_twisted
prkumar Jan 9, 2019
f032281
Merge branch 'master' into feature/v0.8.0/retry-enhance
prkumar Jan 9, 2019
7a578be
Merge pull request #135 from prkumar/feature/v0.8.0/retry-enhance
prkumar Jan 9, 2019
e81003f
Add @daa to AUTHORS.rst
prkumar Jan 9, 2019
08b284b
Replace retry.when.bad_request with status_5xx
prkumar Jan 9, 2019
697ecb9
Clarify retry and ratelimit documentation
prkumar Jan 9, 2019
e1693d8
Reintroduce docs for annotating __init__ and _inject
prkumar Jan 10, 2019
66f4a0a
Group rate limit by host and port
prkumar Jan 11, 2019
851d82d
Refine explaination of group by
prkumar Jan 11, 2019
fe80721
Fix ratelimit docstring
prkumar Jan 11, 2019
8d1b091
Fix failing tests
prkumar Jan 11, 2019
7d1eb11
Add @SakornW to AUTHORS.rst
prkumar Jan 11, 2019
3fbba60
Expose base_url through request builder
prkumar Jan 11, 2019
354ecfc
Merge pull request #136 from prkumar/feature/v0.8.0/ratelimit-group-by
prkumar Jan 11, 2019
319cb9c
Fallback to default behavior if not limited
prkumar Jan 12, 2019
2550265
fix test_call_with_child_class to actually check what is supposed
daa Jan 22, 2019
6bc457f
apply class-level decorators to all consumer methods including inheri…
daa Jan 22, 2019
e5dfce5
add test to check if method annotation is not added when called on no…
daa Jan 22, 2019
9f74e92
Merge pull request #138 from daa/apply-class-decorators-to-all-consum…
prkumar Jan 22, 2019
50acf8f
Fix typos in serialization docs
prkumar Jan 22, 2019
0b525e8
Improve naming
prkumar Jan 23, 2019
2b68535
Add support for callbacks on after finish/fail
prkumar Jan 23, 2019
a870e8a
Fix tests
prkumar Jan 23, 2019
3015e7e
Rename RequestExecution.execute to start
prkumar Jan 23, 2019
e3e9dbf
Remove need for the Request interface
prkumar Jan 23, 2019
289dda0
Update .gitignore
prkumar Jan 24, 2019
92fe3c3
Fix integration tests
prkumar Jan 24, 2019
05ee2a8
Remove Request interface for HttpClientAdapter
prkumar Jan 25, 2019
7038e04
Fix unit tests
prkumar Jan 26, 2019
be5995c
Fix unit test coverage
prkumar Jan 28, 2019
315ec56
Attempt fix for broken Travis builds
prkumar Jan 28, 2019
4f11f6c
Attempt fix for broken Travis builds
prkumar Jan 28, 2019
5e19235
Update documentation
prkumar Feb 5, 2019
f09d983
Update documentation
prkumar Feb 5, 2019
12efc9a
Update CHANGELOG for v0.8.0
prkumar Feb 16, 2019
7979d2c
Add post release segment
prkumar Feb 16, 2019
9f909bc
Update CHANGELOG
prkumar Feb 16, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Expose base_url through request builder
  • Loading branch information
prkumar committed Jan 11, 2019
commit 3fbba604adbc300c7e629b603b778b226a0fd5ce
39 changes: 38 additions & 1 deletion tests/integration/test_ratelimit.py
Original file line number Diff line number Diff line change
@@ -7,6 +7,7 @@

# Constants
BASE_URL = "https://api.github.com/"
DIFFERENT_BASE_URL = "https://hostedgit.luolix.top/"


class CustomRateLimitException(RuntimeError):
@@ -15,7 +16,7 @@ class CustomRateLimitException(RuntimeError):

class GitHub(uplink.Consumer):
@uplink.ratelimit(
calls=1, period=10, raise_on_limit=True
calls=1, period=10, raise_on_limit=True, group_by=None
) # 1 call every 10 seconds
@uplink.get("users/{user}")
def get_user(self, user):
@@ -33,6 +34,16 @@ def get_repo(self, user, repo):
def get_comment(self, user, repo, comment):
pass

@uplink.ratelimit(calls=1, period=10, raise_on_limit=True, group_by=None)
@uplink.get("repos/{user}/{repo}/issues")
def get_issues(self, user, repo):
pass

@uplink.ratelimit(calls=1, period=10, raise_on_limit=True)
@uplink.get("repos/{user}/{repo}/issues/{issue}")
def get_issue(self, user, repo, issue):
pass


# Tests

@@ -70,3 +81,29 @@ def test_exceeded_limit_wait(mock_client):
elapsed = now() - start

assert elapsed >= 1


def test_limit_with_group_by_None(mock_client):
# Setup: consumers pointing to separate hosts
github1 = GitHub(base_url=BASE_URL, client=mock_client)
github2 = GitHub(base_url=DIFFERENT_BASE_URL, client=mock_client)

# Run
github1.get_issues("prkumar", "uplink")

# Verify: the rate limit should be applied globally for all instances
with pytest.raises(RateLimitExceeded):
github2.get_issues("prkumar", "uplink")


def test_limit_with_group_by_host_and_port(mock_client):
# Setup: consumers pointing to separate hosts
github1 = GitHub(base_url=BASE_URL, client=mock_client)
github2 = GitHub(base_url=DIFFERENT_BASE_URL, client=mock_client)

# Run
github1.get_issue("prkumar", "uplink", "issue1")

# Verify: the rate limit should be applied separately by host-port,
# so this request should be fine.
github2.get_issue("prkumar", "uplink", "issue2")
4 changes: 2 additions & 2 deletions tests/unit/test_helpers.py
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@ class Child(Parent):
class TestRequestBuilder(object):
def test_return_type(self):
# Setup
builder = helpers.RequestBuilder(None, {})
builder = helpers.RequestBuilder(None, {}, "base_url")

# Run
builder.return_type = str
@@ -35,7 +35,7 @@ def test_return_type(self):

def test_add_transaction_hook(self, transaction_hook_mock):
# Setup
builder = helpers.RequestBuilder(None, {})
builder = helpers.RequestBuilder(None, {}, "base_url")

# Run
builder.add_transaction_hook(transaction_hook_mock)
2 changes: 1 addition & 1 deletion uplink/builder.py
Original file line number Diff line number Diff line change
@@ -63,7 +63,7 @@ def prepare_request(self, request_builder):

def create_request_builder(self, definition):
registry = definition.make_converter_registry(self._converters)
return helpers.RequestBuilder(self._client, registry)
return helpers.RequestBuilder(self._client, registry, self._base_url)


class CallFactory(object):
7 changes: 6 additions & 1 deletion uplink/helpers.py
Original file line number Diff line number Diff line change
@@ -39,11 +39,12 @@ def set_api_definition(service, name, definition):


class RequestBuilder(object):
def __init__(self, client, converter_registry):
def __init__(self, client, converter_registry, base_url):
self._method = None
self._url = None
self._return_type = None
self._client = client
self._base_url = base_url

# TODO: Pass this in as constructor parameter
# TODO: Delegate instantiations to uplink.HTTPClientAdapter
@@ -65,6 +66,10 @@ def method(self):
def method(self, method):
self._method = method

@property
def base_url(self):
return self._base_url

@property
def url(self):
return self._url
14 changes: 7 additions & 7 deletions uplink/ratelimit.py
Original file line number Diff line number Diff line change
@@ -15,8 +15,8 @@
now = time.monotonic if hasattr(time, "monotonic") else time.time


def _get_host_and_port(url):
parsed_url = utils.urlparse.urlparse(url)
def _get_host_and_port(base_url):
parsed_url = utils.urlparse.urlparse(base_url)
return parsed_url.hostname, parsed_url.port


@@ -80,10 +80,10 @@ class ratelimit(decorators.MethodAnnotation):
defined time period (e.g., 15 calls every 15 minutes).

Note:
The limit is enforced separately for each host-port
combination. In other words, requests are grouped by host and
port, and the number of calls within a time period are counted
and capped separately for each group of requests.
The rate limit is enforced separately for each host-port
combination. Effectively, requests are grouped by host and
port, and the number of requests within a time period are
counted and capped separately for each group.

By default, when the limit is reached, the client will wait until
the current period is over before executing any subsequent
@@ -128,7 +128,7 @@ def __init__(
self._create_limit_reached_exception = None

def _get_limiter_for_request(self, request_builder):
key = self._group_by(request_builder.url.build())
key = self._group_by(request_builder.base_url)
try:
return self._limiter_cache[key]
except KeyError: