Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rate-limiting misbehavior after setting system clock back from erroneous future time to actual time #35

Open
cwpearson opened this issue Jun 28, 2019 · 3 comments
Assignees

Comments

@cwpearson
Copy link
Contributor

cwpearson commented Jun 28, 2019

If the system clock is set ahead of the actual time, and a submission is made, it will fail with an error that the clock skew is too large. For example

#31

Then, if the time is corrected, and another submission made, rai may fail with the following message:

$ ./rai-linux-amd64 -p gpu-algorithms-labs-master/labs/device_query/ 
Error: Last submission was made 1 hour from now. Due to the rate limit, submissions are not allows within a 0s  time window. 
Last submission was made 1 hour from now. Due to the rate limit, submissions are not allows within a 0s  time window. 
2019/06/28 11:43:06 error: Last submission was made 1 hour from now. Due to the rate limit, submissions are not allows within a 0s  time window.

The hidden rate-limiting flag has no effect on this behavior, so there is no work-around. If the system clock was ahead by several hours, the user is blocked for several hours.

Manjaro 18.0.4

./rai-linux-amd64 buildtime -p test/
BuildDate: 
@cwpearson
Copy link
Contributor Author

cwpearson commented Jun 28, 2019

The relevant lines are here:

https://github.com/rai-project/ratelimit/blob/7dcbd07007f98b8524d1f5d007e76b9ca76214ec/ratelimit.go#L47-L54

One option is to not block if the last request was from the future.

It's not entirely clear to me how the hidden rate-limit flag affects the value of options.limit, but if it sets it to 0, then we probably should change the predicate so that if the limit is 0 we don't check the rate. Right now, a previous submission from the future has a negative time, so it always gets blocked.

@cwpearson
Copy link
Contributor Author

A workaround for this is to delete the temporary files rai uses to track what the time of the last submission is. For example:

rm -r /tmp/rai*

@aschuh703
Copy link
Contributor

Now that every transaction is recorded by the server, the solution is to check last execution based on the last execution from the database. There will be no dependence on the client clock.

I will implement when I implement history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants