-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: send drop decisions in batch #1402
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm happy with this, but left one comment to think about.
collect/collect.go
Outdated
if len(traceIDs) >= i.Config.GetCollectionConfig().MaxDropDecisionBatchSize { | ||
send = true | ||
} | ||
case <-ticker.Chan(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This gives us a possible situation like this:
- Max size is reached at time T
- We send the batch
- We get one decision
- At T+1ms the ticker fires
- We send that batch
If the number of decisions per second is close to the max batch size, then the above is very likely to happen.
I've thought of two possible ways to deal with this:
- Keep a timestamp of the oldest item in the batch; if it's less than our ticker time, then don't send yet.
- Don't use a ticker, use a timer, and restart the timer after every send, no matter why we sent it.
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the timer idea. I will update the code to do that
Which problem is this PR solving?
To reduce network traffic from nodes communicating about trace decisions, drop decisions will be sent in batches. The batching logic is based on either a maximum batch size or a timer. Refinery will send a batch when the batch size limit is reached or when the timer expires, whichever comes first.
Short description of the changes
sendDropDecisions
goroutine