Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use lock for Histogram #3547

Closed

Conversation

utpilla
Copy link
Contributor

@utpilla utpilla commented Aug 4, 2022

Use lock for Histogram updates

Stress Test Results (high contention) with Prometheus Exporter scraping interval = 10s

Multiple threads updating the same Histogram MetricPoint in parallel

C:\opentelemetry-dotnet\test\OpenTelemetry.Tests.Stress.Metrics>dotnet run --framework net6.0 --configuration Release
Running (concurrency = 8, prometheusEndpoint = http://localhost:9184/metrics/), press <Esc> to stop...
2022-08-05T00:18:05.8285320Z Loops: 664,533,915, Loops/Second: 5,193,733, CPU Cycles/Loop: 3,124, RunwayTime (Seconds): 121
2022-08-05T00:18:06.0325689Z Loops: 665,604,237, Loops/Second: 5,170,911, CPU Cycles/Loop: 3,104, RunwayTime (Seconds): 121
2022-08-05T00:18:06.2391271Z Loops: 666,664,950, Loops/Second: 5,129,816, CPU Cycles/Loop: 3,126, RunwayTime (Seconds): 121
2022-08-05T00:18:06.4450077Z Loops: 667,725,432, Loops/Second: 5,185,127, CPU Cycles/Loop: 3,106, RunwayTime (Seconds): 121
2022-08-05T00:18:06.6519226Z Loops: 668,790,473, Loops/Second: 5,139,500, CPU Cycles/Loop: 3,074, RunwayTime (Seconds): 122
2022-08-05T00:18:06.8585657Z Loops: 669,851,220, Loops/Second: 5,144,074, CPU Cycles/Loop: 3,169, RunwayTime (Seconds): 122

Stress Test Results (low contention) with Prometheus Exporter scraping interval = 10s

Multiple threads updating random Histogram MetricPoints in parallel

C:\opentelemetry-dotnet\test\OpenTelemetry.Tests.Stress.Metrics>dotnet run --framework net6.0 --configuration Release
Running (concurrency = 8, prometheusEndpoint = http://localhost:9184/metrics/), press <Esc> to stop...
2022-08-04T23:02:23.9567130Z Loops: 2,429,503,031, Loops/Second: 20,119,653, CPU Cycles/Loop: 1,128, RunwayTime (Seconds): 120
2022-08-04T23:02:24.2277354Z Loops: 2,434,415,612, Loops/Second: 19,988,132, CPU Cycles/Loop: 1,130, RunwayTime (Seconds): 121
2022-08-04T23:02:24.4725452Z Loops: 2,439,784,407, Loops/Second: 19,949,797, CPU Cycles/Loop: 1,128, RunwayTime (Seconds): 121
2022-08-04T23:02:24.7065189Z Loops: 2,444,642,611, Loops/Second: 20,134,782, CPU Cycles/Loop: 1,129, RunwayTime (Seconds): 121
2022-08-04T23:02:24.9475497Z Loops: 2,449,492,448, Loops/Second: 19,901,301, CPU Cycles/Loop: 1,130, RunwayTime (Seconds): 121
2022-08-04T23:02:25.1765231Z Loops: 2,453,822,909, Loops/Second: 19,435,860, CPU Cycles/Loop: 1,119, RunwayTime (Seconds): 122

Benchmark Results

// * Summary *

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
Intel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores
.NET SDK=6.0.302
[Host] : .NET 6.0.7 (6.0.722.32202), X64 RyuJIT
DefaultJob : .NET 6.0.7 (6.0.722.32202), X64 RyuJIT

Method BoundCount Mean Error StdDev
HistogramHotPath 10 46.44 ns 0.842 ns 0.746 ns
HistogramWith1LabelHotPath 10 97.67 ns 1.968 ns 3.447 ns
HistogramWith3LabelsHotPath 10 202.86 ns 4.035 ns 4.956 ns
HistogramWith5LabelsHotPath 10 292.78 ns 5.758 ns 7.282 ns
HistogramWith7LabelsHotPath 10 355.86 ns 7.033 ns 14.366 ns
HistogramHotPath 20 49.53 ns 1.006 ns 0.941 ns
HistogramWith1LabelHotPath 20 106.52 ns 2.099 ns 2.943 ns
HistogramWith3LabelsHotPath 20 207.65 ns 4.064 ns 5.139 ns
HistogramWith5LabelsHotPath 20 299.79 ns 5.967 ns 9.465 ns
HistogramWith7LabelsHotPath 20 358.21 ns 7.190 ns 14.023 ns
HistogramHotPath 50 59.17 ns 1.210 ns 2.055 ns
HistogramWith1LabelHotPath 50 115.68 ns 2.265 ns 3.023 ns
HistogramWith3LabelsHotPath 50 221.92 ns 4.318 ns 4.973 ns
HistogramWith5LabelsHotPath 50 310.69 ns 6.006 ns 11.571 ns
HistogramWith7LabelsHotPath 50 378.06 ns 7.383 ns 8.789 ns
HistogramHotPath 100 75.17 ns 1.533 ns 3.701 ns
HistogramWith1LabelHotPath 100 127.82 ns 2.528 ns 3.105 ns
HistogramWith3LabelsHotPath 100 235.38 ns 4.545 ns 10.258 ns
HistogramWith5LabelsHotPath 100 328.82 ns 6.550 ns 13.817 ns
HistogramWith7LabelsHotPath 100 393.21 ns 7.738 ns 10.592 ns

using System.Threading;
using BenchmarkDotNet.Attributes;

namespace Benchmarks.Metrics
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore this file.

@codecov
Copy link

codecov bot commented Aug 4, 2022

Codecov Report

Merging #3547 (f44920c) into main (666a0c1) will increase coverage by 0.13%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3547      +/-   ##
==========================================
+ Coverage   87.04%   87.17%   +0.13%     
==========================================
  Files         275      275              
  Lines        9959     9951       -8     
==========================================
+ Hits         8669     8675       +6     
+ Misses       1290     1276      -14     
Impacted Files Coverage Δ
src/OpenTelemetry/Metrics/HistogramBuckets.cs 100.00% <ø> (ø)
src/OpenTelemetry/Metrics/MetricPoint.cs 85.60% <100.00%> (+0.60%) ⬆️
src/OpenTelemetry/ProviderExtensions.cs 81.81% <0.00%> (-9.10%) ⬇️
...Telemetry/Metrics/PeriodicExportingMetricReader.cs 72.72% <0.00%> (-5.46%) ⬇️
...tpListener/Internal/PrometheusCollectionManager.cs 80.48% <0.00%> (+2.43%) ⬆️
...metryProtocol/Implementation/ActivityExtensions.cs 94.50% <0.00%> (+3.29%) ⬆️
...xporter.OpenTelemetryProtocol/OtlpTraceExporter.cs 77.27% <0.00%> (+18.18%) ⬆️
...entation/ExportClient/OtlpGrpcTraceExportClient.cs 78.57% <0.00%> (+28.57%) ⬆️

@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

@github-actions github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Aug 13, 2022
@github-actions
Copy link
Contributor

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Aug 20, 2022
@utpilla utpilla deleted the utpilla/Use-lock-for-Histogram branch November 23, 2023 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Stale Issues and pull requests which have been flagged for closing due to inactivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant