-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Listener catch-up fix #450
Conversation
f84439a
to
8a7c124
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks good to me. Just left a minor comment about timeouts.
vms/evm/subscriber.go
Outdated
) | ||
// Sleep if this wasn't the last retry | ||
if i < rpcMaxRetries-1 { | ||
time.Sleep(time.Duration(i+1) * time.Second) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: it might be more clear to specify the total timeout/sleep duration (10 seconds across 5 retries) as a constant. Any reason why the subscriber timeout is a constant on each retry, but the rpc timeout grows linearly with each retry?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, we should be consistent here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree but in which direction? constant? Ideally with network resources I like exponential backoffs with jitter but seemed like an overkill here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exponential backoff has come up in previous discussions, but we've come to similar conclusions that it's overkill. Since the overall number of RPC calls is low, optimizing how they're spaced hasn't been a top priority.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought we were tracking that already, but apparently not. I created #453 to do so.
var err error | ||
var header *types.Header | ||
attempt := 1 | ||
for { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why don't we do a regular for attempt=1; attempt < retryAttempts; attempt ++ here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
matching the way that subscriber reattempts right below this
Co-authored-by: cam-schultz <78878559+cam-schultz@users.noreply.github.com> Signed-off-by: Ian Suvak <ian.suvak@avalabs.org>
Why this should be merged
Addresses #443
How this works
Moves catchup process from subscriber to listener along with the headers channelAfter doing this I realized that this change can be made lighter by passing in an RPC client to the Subscriber as well. That would mean that we can keep the catchup and the headers on the subscriber level.I pivoted to keeping things on the subscriber level and just changing the client and adding retries there
How this was tested
CI
How is this documented
N/A