Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support of rate limit and timeout #1347

Merged
merged 54 commits into from
Aug 11, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
6ec7eee
wip timeout
bnjjj Jun 30, 2022
d95f006
wip timeout
bnjjj Jun 30, 2022
a6a39bb
add global rate limit
bnjjj Jul 1, 2022
69052fb
better error handling
bnjjj Jul 4, 2022
8933c76
remove some 'static trait bounds
Jul 5, 2022
d82ad8a
create the execution service on new requests
Jul 5, 2022
ad740bb
Merge branch 'api-1.0' into geal/remove-buffer-from-subgraphs
Jul 5, 2022
c86e57c
add an ExecutionServiceFactory
Jul 5, 2022
900da83
add a subgraph service factory
Jul 6, 2022
8d55f0a
try to clean up the API
Jul 6, 2022
75a0bcf
make a working version
Jul 7, 2022
4a78126
Merge branch 'api-1.0' of github.com:apollographql/router into bnjjj/…
bnjjj Jul 7, 2022
5244277
Merge branch 'geal/remove-buffer-from-subgraphs' of github.com:apollo…
bnjjj Jul 7, 2022
f89a9c1
add the MakeSubgraphService trait
Jul 7, 2022
b2626fe
remove unused code
Jul 7, 2022
8f3b807
wip
bnjjj Jul 7, 2022
be791f6
Merge branch 'geal/remove-buffer-from-subgraphs' of github.com:apollo…
bnjjj Jul 7, 2022
c7d11fe
fix tests
Jul 7, 2022
1c51f9a
useless import
bnjjj Jul 7, 2022
709aa7e
Merge branch 'geal/remove-buffer-from-subgraphs' of github.com:apollo…
bnjjj Jul 7, 2022
fed5dac
add tests
bnjjj Jul 7, 2022
4e91563
add tests
bnjjj Jul 8, 2022
80bb2a5
Merge branch 'api-1.0' of github.com:apollographql/router into bnjjj/…
bnjjj Jul 8, 2022
6de0678
Merge branch 'api-1.0' of github.com:apollographql/router into bnjjj/…
bnjjj Jul 8, 2022
dba0522
fix snapshot
bnjjj Jul 8, 2022
4557796
add documentation
bnjjj Jul 8, 2022
33c06ed
delete useless comments
bnjjj Jul 8, 2022
c72a79d
changelog
bnjjj Jul 8, 2022
aed3e49
Apply suggestions from code review
bnjjj Jul 11, 2022
3a419bc
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Jul 25, 2022
71bc88e
clean cargo.toml
bnjjj Jul 25, 2022
0013c3b
improve rate limit algorithm by using sliding window and directly rej…
bnjjj Jul 26, 2022
1ec4978
Merge branch 'bnjjj/feat_rate_limit_timeout' of github.com:apollograp…
bnjjj Jul 26, 2022
254b190
lint
bnjjj Jul 26, 2022
9e0c9de
update documentation and apply review comments
bnjjj Jul 27, 2022
f5f741a
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Jul 27, 2022
7ad5af2
fix test
bnjjj Jul 27, 2022
deeae78
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Jul 27, 2022
76bb88e
address review's comments
bnjjj Jul 27, 2022
8746a93
address review's comments
bnjjj Aug 1, 2022
27c5093
fix
bnjjj Aug 1, 2022
11912b2
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 1, 2022
8b52c6c
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 2, 2022
01f5090
fix config for traffic_shaping
bnjjj Aug 2, 2022
e68d320
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 2, 2022
33f6eb1
change rate_limit to global_rate_limit
bnjjj Aug 2, 2022
b1abba4
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 8, 2022
39d1406
address review comments
bnjjj Aug 8, 2022
3af8c76
Update apollo-router/src/plugins/traffic_shaping/rate/rate.rs
bnjjj Aug 8, 2022
8717d13
lower the latency for tests
bnjjj Aug 8, 2022
553388e
Merge branch 'bnjjj/feat_rate_limit_timeout' of github.com:apollograp…
bnjjj Aug 8, 2022
7ab5fc6
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 8, 2022
6fc0356
refactor + add documentation
bnjjj Aug 9, 2022
7a81ffd
Merge branch 'main' of github.com:apollographql/router into bnjjj/fea…
bnjjj Aug 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,21 @@ traffic_shaping:

By [@bnjjj](https://github.com/bnjjj) in https://github.com/apollographql/router/pull/1347

### Update span attributes to be compliant with the opentelemetry for GraphQL specs ([PR #1449](https://github.com/apollographql/router/pull/1449))

Change attribute name `query` to `graphql.document` and `operation_name` to `graphql.operation.name` in spans.

By [@bnjjj](https://github.com/bnjjj) in https://github.com/apollographql/router/pull/1449

### Configuration handling enhancements ([PR #1454](https://github.com/apollographql/router/pull/1454))

Router config handling now:
* Allows completely empty configuration without error.
* Prevents unknown tags at the root of the configuration from being silently ignored.

By [@bryncooke](https://github.com/bryncooke) in https://github.com/apollographql/router/pull/1454


## 🛠 Maintenance

## 📚 Documentation
Expand Down
8 changes: 4 additions & 4 deletions apollo-router/src/plugins/traffic_shaping/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -466,7 +466,7 @@ mod test {
test:
global_rate_limit:
capacity: 1
interval: 1sec
interval: 300ms
timeout: 500ms
"#,
)
Expand All @@ -491,7 +491,7 @@ mod test {
.oneshot(SubgraphRequest::fake_builder().build())
.await
.unwrap();
tokio::time::sleep(Duration::from_millis(1000)).await;
tokio::time::sleep(Duration::from_millis(300)).await;
let _response = plugin
.subgraph_service("test", test_service.boxed())
.oneshot(SubgraphRequest::fake_builder().build())
Expand All @@ -506,7 +506,7 @@ mod test {
router:
global_rate_limit:
capacity: 1
interval: 1sec
interval: 300ms
timeout: 500ms
"#,
)
Expand Down Expand Up @@ -536,7 +536,7 @@ mod test {
.oneshot(RouterRequest::fake_builder().build().unwrap())
.await
.is_err());
tokio::time::sleep(Duration::from_millis(1000)).await;
tokio::time::sleep(Duration::from_millis(300)).await;
let _response = plugin
.router_service(mock_service.clone().boxed())
.oneshot(RouterRequest::fake_builder().build().unwrap())
Expand Down
14 changes: 10 additions & 4 deletions apollo-router/src/plugins/traffic_shaping/rate/layer.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
use std::num::NonZeroU64;
use std::sync::atomic::AtomicU64;
use std::sync::atomic::AtomicUsize;
use std::sync::Arc;
use std::sync::RwLock;
use std::time::Duration;
use std::time::SystemTime;
use std::time::UNIX_EPOCH;

use tokio::time::Instant;
use tower::Layer;

use super::Rate;
Expand All @@ -14,7 +15,7 @@ use super::RateLimit;
#[derive(Debug, Clone)]
pub(crate) struct RateLimitLayer {
rate: Rate,
window_start: Arc<RwLock<Instant>>,
window_start: Arc<AtomicU64>,
previous_nb_requests: Arc<AtomicUsize>,
current_nb_requests: Arc<AtomicUsize>,
}
Expand All @@ -25,7 +26,12 @@ impl RateLimitLayer {
let rate = Rate::new(num, per);
RateLimitLayer {
rate,
window_start: Arc::new(RwLock::new(Instant::now())),
window_start: Arc::new(AtomicU64::new(
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("system time must be after EPOCH")
.as_millis() as u64,
)),
previous_nb_requests: Arc::default(),
current_nb_requests: Arc::new(AtomicUsize::new(1)),
}
Expand Down
2 changes: 1 addition & 1 deletion apollo-router/src/plugins/traffic_shaping/rate/rate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ impl Rate {
///
/// This function panics if `num` or `per` is 0.
pub(crate) fn new(num: NonZeroU64, per: Duration) -> Self {
assert!(per > Duration::from_millis(0));
assert!(per > Duration::default());

Rate {
num: num.into(),
Expand Down
47 changes: 35 additions & 12 deletions apollo-router/src/plugins/traffic_shaping/rate/service.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
use std::sync::atomic::AtomicU64;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering;
use std::sync::Arc;
use std::sync::RwLock;
use std::task::Context;
use std::task::Poll;
use std::time::Duration;
use std::time::SystemTime;
use std::time::UNIX_EPOCH;

use futures::ready;
use tokio::time::Instant;
use tower::Service;

use super::future::ResponseFuture;
Expand All @@ -17,7 +19,10 @@ use crate::plugins::traffic_shaping::rate::error::RateLimited;
pub(crate) struct RateLimit<T> {
pub(crate) inner: T,
pub(crate) rate: Rate,
pub(crate) window_start: Arc<RwLock<Instant>>,
/// We're using an atomic u64 because it's basically a timestamp in milliseconds for the start of the window
/// Instead of using an Instant which is not thread safe we're using an atomic u64
/// It's ok to have an u64 because we just care about milliseconds for this use case
pub(crate) window_start: Arc<AtomicU64>,
pub(crate) previous_nb_requests: Arc<AtomicUsize>,
pub(crate) current_nb_requests: Arc<AtomicUsize>,
}
Expand All @@ -32,25 +37,43 @@ where
type Future = ResponseFuture<S::Future>;

fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
let mut window_start = self.window_start.read().unwrap().elapsed();
let time_unit = self.rate.per();
let time_unit = self.rate.per().as_millis() as u64;

if window_start > time_unit {
let new_window_start = Instant::now();
*self.window_start.write().unwrap() = new_window_start;
window_start = new_window_start.elapsed();
let updated =
self.window_start
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |window_start| {
let duration_start = Duration::from_millis(window_start);
let duration_now = Duration::from_millis(
bnjjj marked this conversation as resolved.
Show resolved Hide resolved
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("system time must be after EPOCH")
.as_millis() as u64,
);
if duration_now.saturating_sub(duration_start) > self.rate.per() {
Some(
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("system time must be after EPOCH")
.as_millis() as u64,
)
} else {
None
}
});
// If it has been updated
if let Ok(_updated_window_start) = updated {
Geal marked this conversation as resolved.
Show resolved Hide resolved
self.previous_nb_requests.swap(
self.current_nb_requests.load(Ordering::SeqCst),
Ordering::SeqCst,
);
self.current_nb_requests.swap(1, Ordering::SeqCst);
}

let estimated_cap = (self.previous_nb_requests.load(Ordering::SeqCst)
* (time_unit
.checked_sub(window_start)
.checked_sub(self.window_start.load(Ordering::SeqCst))
.unwrap_or_default()
.as_millis()
/ time_unit.as_millis()) as usize)
/ time_unit) as usize)
+ self.current_nb_requests.load(Ordering::SeqCst);

if estimated_cap as u64 > self.rate.num() {
Expand Down