Refactor operation tracker ctor and introduce custom percentiles #1159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. refactor ctor of both simple and adaptive operation tracker by passing the router config and operation class. This allows operation tracker itself to populate its parameters based on type of operation class. 2. introduce custom percentiles of latency Histogram in adaptive operation tracker. The percentiles are configurable and on-demand. User can define arbitrary quantile like 91th percentile to better eveluate the latency distribution.
Codecov Report
@@ Coverage Diff @@
## master #1159 +/- ##
===========================================
+ Coverage 69.84% 69.94% +0.1%
- Complexity 5316 5344 +28
===========================================
Files 420 426 +6
Lines 32547 32626 +79
Branches 4139 4140 +1
===========================================
+ Hits 22732 22821 +89
+ Misses 8695 8667 -28
- Partials 1120 1138 +18
Continue to review full report at Codecov.
|
zzmao
reviewed
Apr 25, 2019
| List<String> customPercentiles = | ||
| Utils.splitString(verifiableProperties.getString("router.operation.tracker.custom.percentiles", ""), ","); | ||
| routerOperationTrackerCustomPercentiles = | ||
| Collections.unmodifiableList(customPercentiles.stream().map(Double::valueOf).collect(Collectors.toList())); |
There was a problem hiding this comment.
Can we convert string to list when it's needed? Previous example: storeCompactionTriggers
| @@ -237,23 +237,19 @@ protected GetRequest createGetRequest(BlobId blobId, MessageFormatFlags flag, Ge | |||
| /** | |||
| * Gets an {@link OperationTracker} based on the config and {@code partitionId}. | |||
| * @param partitionId the {@link PartitionId} for which a tracker is required. | |||
| * @param operationClass | |||
|
./gradlew build && ./gradlew test succeeded. Reminder to review, thanks. |
cgtz
reviewed
Apr 30, 2019
| return disk.getState() == HardwareState.AVAILABLE ? 1L : 0L; | ||
| } | ||
| }; | ||
| Gauge<Long> diskState = () -> disk.getState() == HardwareState.AVAILABLE ? 1L : 0L; |
There was a problem hiding this comment.
Thanks for cleaning up this code!
| // populate tracker parameters based on operation type | ||
| boolean crossColoEnabled = false; | ||
| boolean includeNonOriginatingDcReplicas = true; | ||
| int replicasRequired = Integer.MAX_VALUE; |
There was a problem hiding this comment.
how is this local variable used?
| crossColoEnabled = routerConfig.routerGetCrossDcEnabled; | ||
| includeNonOriginatingDcReplicas = routerConfig.routerGetIncludeNonOriginatingDcReplicas; | ||
| replicasRequired = routerConfig.routerGetReplicasRequired; | ||
| } else if (operationClass == PutOperation.class) { |
There was a problem hiding this comment.
One other option is to make an enum to represent the operation tracker types (i.e. GET, PUT, DELETE, TTL_UPDATE). This is a little more explicit and resilient to new classes/name changes.
| */ | ||
| @Config("router.operation.tracker.custom.percentiles") | ||
| @Default("") | ||
| public final List<Double> routerOperationTrackerCustomPercentiles; |
There was a problem hiding this comment.
how will these custom percentiles be used by operation tracker in the future? Right now, they are only registered in the MetricRegistry for viewing.
|
Good to merge? @cgtz |
cgtz
approved these changes
May 3, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
passing the router config and operation class. This allows operation
tracker itself to populate its parameters based on type of operation
class.
operation tracker. The percentiles are configurable and on-demand. User
can define arbitrary quantile like 91th percentile to better eveluate
the latency distribution.