Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track Max & mean seconds to receive state messages #15586

Merged
merged 5 commits into from
Aug 12, 2022

Conversation

alovew
Copy link
Contributor

@alovew alovew commented Aug 12, 2022

Track max and mean seconds before receiving state message in Segment.

@github-actions github-actions bot added area/platform issues related to the platform area/worker Related to worker labels Aug 12, 2022
@alovew alovew temporarily deployed to more-secrets August 12, 2022 02:09 Inactive
@alovew alovew temporarily deployed to more-secrets August 12, 2022 02:11 Inactive
@alovew alovew temporarily deployed to more-secrets August 12, 2022 17:13 Inactive
Copy link
Contributor

@evantahler evantahler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment on lines 24 to 27
meanSecondsBeforeStateMessageEmitted:
type: integer
maxSecondsBeforeStateMessageEmitted:
type: integer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment is probably valuable for future us - that this message is from the Source.

Comment on lines +124 to +126
if (firstRecordReceivedAt == null) {
firstRecordReceivedAt = DateTime.now();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 this is a global time, and not per-stream

Comment on lines 557 to 558
when(messageTracker.getMaxSecondsToReceiveStateMessage()).thenReturn(10L);
when(messageTracker.getMeanSecondsToReceiveStateMessage()).thenReturn(8L);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 testing the math

Comment on lines 83 to 84
this.maxSecondsToReceiveStateMessage = new AtomicLong(0L);
this.meanSecondsToReceiveStateMessage = new AtomicLong(0L);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 These values will always be numeric (never null)

@alovew alovew force-pushed the anne/max-and-mean-time-to-receive-state-message branch from 3635a31 to 8f19838 Compare August 12, 2022 18:48
@alovew alovew temporarily deployed to more-secrets August 12, 2022 18:50 Inactive
@alovew alovew temporarily deployed to more-secrets August 12, 2022 20:36 Inactive
Copy link
Contributor

@gosusnp gosusnp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about the types we are using. From this PR, it looks like we are using AtomicLong but don't really have any concurrency issues.

maxSecondsToReceiveSourceStateMessage.set(secondsSinceLastStateMessage);
}

if (meanSecondsToReceiveSourceStateMessage.get() == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering why are we using AtomicLong? Overall, it feels like we should be either using a Long or trying to leverage the atomic updates such as compareAndExchange.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mostly just following the convention in the messagetracker for how we're tracking the other data

@alovew alovew temporarily deployed to more-secrets August 12, 2022 21:27 Inactive
@alovew alovew merged commit 1b1448d into master Aug 12, 2022
@alovew alovew deleted the anne/max-and-mean-time-to-receive-state-message branch August 12, 2022 22:07
rodireich pushed a commit that referenced this pull request Aug 20, 2022
* Segment tracking for max and mean seconds to receive state message from source
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/platform issues related to the platform area/scheduler area/worker Related to worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants