Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set debug mode to see the CI logs with debug info #5877

Closed
wants to merge 6 commits into from
Closed

Set debug mode to see the CI logs with debug info #5877

wants to merge 6 commits into from

Conversation

NachoEchevarria
Copy link
Contributor

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@github-actions github-actions bot added the area:tests unit tests, integration tests label Aug 12, 2024
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Aug 12, 2024 •
edited
Loading

Datadog Report

Branch report: nacho/FixEmailInstrumentedTests
Commit report: 05c4bc9
Test service: dd-trace-dotnet

✅ 0 Failed, 424797 Passed, 2662 Skipped, 26h 23m 11.95s Total Time

@andrewlock
Copy link
Member

andrewlock commented Aug 12, 2024 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (73ms)  : 63, 83
     .   : milestone, 73,
    master - mean (74ms)  : 64, 84
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (1,106ms)  : 1081, 1131
     .   : milestone, 1106,
    master - mean (1,110ms)  : 1090, 1130
     .   : milestone, 1110,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (109ms)  : 104, 113
     .   : milestone, 109,
    master - mean (111ms)  : 106, 117
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (801ms)  : 779, 823
     .   : milestone, 801,
    master - mean (803ms)  : 788, 817
     .   : milestone, 803,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (92ms)  : 89, 95
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (749ms)  : 734, 765
     .   : milestone, 749,
    master - mean (754ms)  : 732, 776
     .   : milestone, 754,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (192ms)  : 187, 196
     .   : milestone, 192,
    master - mean (191ms)  : 187, 194
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (1,200ms)  : 1177, 1223
     .   : milestone, 1200,
    master - mean (1,195ms)  : 1171, 1219
     .   : milestone, 1195,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (275ms)  : 271, 280
     .   : milestone, 275,
    master - mean (275ms)  : 270, 281
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (974ms)  : 956, 993
     .   : milestone, 974,
    master - mean (976ms)  : 958, 994
     .   : milestone, 976,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5877) - mean (264ms)  : 260, 268
     .   : milestone, 264,
    master - mean (265ms)  : 261, 269
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (5877) - mean (961ms)  : 938, 984
     .   : milestone, 961,
    master - mean (958ms)  : 932, 985
     .   : milestone, 958,
Loading

@andrewlock
Copy link
Member

andrewlock commented Aug 12, 2024 •
edited
Loading

Benchmarks Report for tracer 🐌

Benchmarks for #5877 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.188
  • 1 benchmarks are slower, with geometric mean 1.139
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.65μs 42.9ns 291ns 0.0155 0.00777 0 5.43 KB
master StartStopWithChild netcoreapp3.1 9.94μs 56.1ns 384ns 0.0248 0.00991 0 5.61 KB
master StartStopWithChild net472 16.1μs 62.7ns 243ns 1.02 0.307 0.0945 6.06 KB
#5877 StartStopWithChild net6.0 7.82μs 44.9ns 333ns 0.0154 0.0077 0 5.42 KB
#5877 StartStopWithChild netcoreapp3.1 9.99μs 56.9ns 402ns 0.024 0.0096 0 5.62 KB
#5877 StartStopWithChild net472 16μs 52.9ns 205ns 1.02 0.299 0.0866 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 484μs 235ns 908ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 634μs 551ns 2.13μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 854μs 795ns 2.97μs 0.431 0 0 3.3 KB
#5877 WriteAndFlushEnrichedTraces net6.0 467μs 193ns 746ns 0 0 0 2.7 KB
#5877 WriteAndFlushEnrichedTraces netcoreapp3.1 659μs 392ns 1.47μs 0 0 0 2.7 KB
#5877 WriteAndFlushEnrichedTraces net472 847μs 671ns 2.6μs 0.419 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 198μs 1.15μs 10.8μs 0.2 0 0 18.45 KB
master SendRequest netcoreapp3.1 224μs 1.31μs 11.7μs 0.216 0 0 20.61 KB
master SendRequest net472 0.000252ns 0.000252ns 0.000907ns 0 0 0 0 b
#5877 SendRequest net6.0 204μs 1.18μs 10.9μs 0.193 0 0 18.45 KB
#5877 SendRequest netcoreapp3.1 220μs 1.26μs 10.2μs 0.217 0 0 20.61 KB
#5877 SendRequest net472 0.00166ns 0.000567ns 0.0022ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5877

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.52 KB 41.75 KB 234 B 0.56%

Fewer allocations 🎉 in #5877

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.74 KB 41.39 KB -350 B -0.84%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 567μs 2.89μs 15μs 0.551 0 0 41.74 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 753μs 4.22μs 26.7μs 0.374 0 0 41.52 KB
master WriteAndFlushEnrichedTraces net472 859μs 4.41μs 19.7μs 8.08 2.55 0.425 53.29 KB
#5877 WriteAndFlushEnrichedTraces net6.0 548μs 1.54μs 5.55μs 0.584 0 0 41.39 KB
#5877 WriteAndFlushEnrichedTraces netcoreapp3.1 679μs 3.36μs 13.8μs 0.342 0 0 41.75 KB
#5877 WriteAndFlushEnrichedTraces net472 865μs 3.95μs 14.8μs 8.09 2.25 0.45 53.31 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.3μs 1.66ns 6.42ns 0.0142 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.69μs 1.14ns 4.41ns 0.0134 0 0 1.02 KB
master ExecuteNonQuery net472 1.99μs 2.31ns 8.65ns 0.156 0 0 987 B
#5877 ExecuteNonQuery net6.0 1.25μs 1.43ns 5.33ns 0.0145 0 0 1.02 KB
#5877 ExecuteNonQuery netcoreapp3.1 1.75μs 1.36ns 5.1ns 0.014 0 0 1.02 KB
#5877 ExecuteNonQuery net472 1.94μs 2.45ns 9.5ns 0.156 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5877

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.139 1,201.60 1,368.19

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.2μs 0.946ns 3.66ns 0.0135 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.53μs 0.6ns 2.33ns 0.0131 0 0 976 B
master CallElasticsearch net472 2.52μs 1.82ns 7.03ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.43μs 1.56ns 5.85ns 0.0134 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.66μs 0.849ns 3.18ns 0.0133 0 0 1.02 KB
master CallElasticsearchAsync net472 2.8μs 1.91ns 7.16ns 0.166 0 0 1.05 KB
#5877 CallElasticsearch net6.0 1.37μs 2.02ns 7.82ns 0.0138 0 0 976 B
#5877 CallElasticsearch netcoreapp3.1 1.55μs 1.24ns 4.3ns 0.0127 0 0 976 B
#5877 CallElasticsearch net472 2.5μs 1.39ns 5.38ns 0.158 0 0 995 B
#5877 CallElasticsearchAsync net6.0 1.37μs 1.39ns 5.37ns 0.013 0 0 952 B
#5877 CallElasticsearchAsync netcoreapp3.1 1.6μs 0.538ns 2.01ns 0.0136 0 0 1.02 KB
#5877 CallElasticsearchAsync net472 2.6μs 1.05ns 3.93ns 0.167 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.19μs 2.04ns 7.9ns 0.013 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.6μs 0.514ns 1.99ns 0.0128 0 0 952 B
master ExecuteAsync net472 1.69μs 1.21ns 4.54ns 0.145 0 0 915 B
#5877 ExecuteAsync net6.0 1.2μs 2.45ns 8.85ns 0.0132 0 0 952 B
#5877 ExecuteAsync netcoreapp3.1 1.57μs 0.747ns 2.79ns 0.0127 0 0 952 B
#5877 ExecuteAsync net472 1.68μs 0.376ns 1.41ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.25μs 1.38ns 5.36ns 0.03 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.05μs 1.23ns 4.76ns 0.0354 0 0 2.76 KB
master SendAsync net472 7.74μs 2.1ns 8.15ns 0.498 0 0 3.15 KB
#5877 SendAsync net6.0 4.15μs 2.35ns 9.1ns 0.0311 0 0 2.22 KB
#5877 SendAsync netcoreapp3.1 5.19μs 6.64ns 25.7ns 0.0359 0 0 2.76 KB
#5877 SendAsync net472 7.61μs 2.18ns 8.44ns 0.498 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.47μs 1.11ns 4.16ns 0.0233 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.13μs 8.79ns 34.1ns 0.0221 0 0 1.64 KB
master EnrichedLog net472 2.64μs 4.31ns 16.7ns 0.25 0 0 1.57 KB
#5877 EnrichedLog net6.0 1.49μs 0.891ns 3.33ns 0.0234 0 0 1.64 KB
#5877 EnrichedLog netcoreapp3.1 2.22μs 1.86ns 7.19ns 0.0222 0 0 1.64 KB
#5877 EnrichedLog net472 2.67μs 1.23ns 4.75ns 0.25 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 117μs 210ns 812ns 0 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 123μs 605ns 2.64μs 0 0 0 4.28 KB
master EnrichedLog net472 153μs 161ns 603ns 0.678 0.226 0 4.46 KB
#5877 EnrichedLog net6.0 117μs 270ns 1.04μs 0.0585 0 0 4.28 KB
#5877 EnrichedLog netcoreapp3.1 120μs 237ns 920ns 0.0592 0 0 4.28 KB
#5877 EnrichedLog net472 148μs 167ns 646ns 0.662 0.221 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.93μs 0.982ns 3.8ns 0.0312 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.1μs 1.8ns 6.99ns 0.029 0 0 2.2 KB
master EnrichedLog net472 4.82μs 0.791ns 3.06ns 0.321 0 0 2.02 KB
#5877 EnrichedLog net6.0 2.97μs 0.785ns 3.04ns 0.0297 0 0 2.2 KB
#5877 EnrichedLog netcoreapp3.1 4.36μs 1.6ns 5.98ns 0.03 0 0 2.2 KB
#5877 EnrichedLog net472 4.89μs 1.51ns 5.86ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.42μs 1.23ns 4.78ns 0.0162 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.75μs 1.32ns 5.11ns 0.0154 0 0 1.14 KB
master SendReceive net472 2.14μs 2.49ns 9.64ns 0.183 0.0011 0 1.16 KB
#5877 SendReceive net6.0 1.38μs 0.595ns 2.23ns 0.0159 0 0 1.14 KB
#5877 SendReceive netcoreapp3.1 1.74μs 1.07ns 4.14ns 0.0148 0 0 1.14 KB
#5877 SendReceive net472 2.03μs 0.532ns 1.92ns 0.183 0.00102 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.72μs 0.756ns 2.83ns 0.0218 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.85μs 4.56ns 17.7ns 0.021 0 0 1.65 KB
master EnrichedLog net472 4.37μs 2.79ns 10.8ns 0.324 0 0 2.04 KB
#5877 EnrichedLog net6.0 2.78μs 0.938ns 3.38ns 0.023 0 0 1.6 KB
#5877 EnrichedLog netcoreapp3.1 4.09μs 1.7ns 6.37ns 0.0225 0 0 1.65 KB
#5877 EnrichedLog net472 4.44μs 2.16ns 8.07ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5877

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.188 559.08 470.45

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 397ns 0.302ns 1.13ns 0.00803 0 0 576 B
master StartFinishSpan netcoreapp3.1 559ns 0.165ns 0.637ns 0.0077 0 0 576 B
master StartFinishSpan net472 646ns 0.326ns 1.26ns 0.0918 0 0 578 B
master StartFinishScope net6.0 559ns 0.528ns 2.04ns 0.00981 0 0 696 B
master StartFinishScope netcoreapp3.1 761ns 0.27ns 1.05ns 0.00938 0 0 696 B
master StartFinishScope net472 922ns 1.27ns 4.76ns 0.105 0 0 658 B
#5877 StartFinishSpan net6.0 389ns 0.22ns 0.822ns 0.00815 0 0 576 B
#5877 StartFinishSpan netcoreapp3.1 607ns 0.648ns 2.51ns 0.00778 0 0 576 B
#5877 StartFinishSpan net472 605ns 1.28ns 4.97ns 0.0915 0 0 578 B
#5877 StartFinishScope net6.0 470ns 0.616ns 2.3ns 0.00972 0 0 696 B
#5877 StartFinishScope netcoreapp3.1 732ns 2.08ns 8.04ns 0.00955 0 0 696 B
#5877 StartFinishScope net472 906ns 1.36ns 5.25ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 605ns 0.258ns 1ns 0.00986 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 873ns 1.23ns 4.76ns 0.00915 0 0 696 B
master RunOnMethodBegin net472 1.1μs 0.27ns 1.04ns 0.104 0 0 658 B
#5877 RunOnMethodBegin net6.0 581ns 0.472ns 1.83ns 0.00967 0 0 696 B
#5877 RunOnMethodBegin netcoreapp3.1 972ns 2.7ns 10.5ns 0.00963 0 0 696 B
#5877 RunOnMethodBegin net472 1.06μs 2.75ns 19.2ns 0.104 0 0 658 B

@andrewlock
Copy link
Member

andrewlock commented Aug 13, 2024 •
edited
Loading

Benchmarks Report for appsec 🐌

Benchmarks for #5877 compared to master:

  • All benchmarks have the same speed
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 73μs 108ns 418ns 0.0728 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 63.5μs 87.1ns 337ns 0.0952 0 0 6.95 KB
master AllCycleSimpleBody net472 48.7μs 80ns 310ns 1.31 0 0 8.34 KB
master AllCycleMoreComplexBody net6.0 79.6μs 141ns 547ns 0.12 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 70.7μs 108ns 419ns 0.141 0 0 10.37 KB
master AllCycleMoreComplexBody net472 55.4μs 77.2ns 299ns 1.87 0.0275 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 142ns 0.185ns 0.715ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 200ns 0.187ns 0.699ns 0.00372 0 0 272 B
master ObjectExtractorSimpleBody net472 165ns 0.141ns 0.527ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 2.11ns 8.18ns 0.0534 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.13μs 1.88ns 7.28ns 0.0496 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.8μs 2.4ns 9.29ns 0.602 0.00569 0 3.8 KB
#5877 AllCycleSimpleBody net6.0 73.7μs 97.7ns 379ns 0.0735 0 0 6.01 KB
#5877 AllCycleSimpleBody netcoreapp3.1 63μs 66.6ns 258ns 0.0945 0 0 6.95 KB
#5877 AllCycleSimpleBody net472 48.9μs 40ns 144ns 1.31 0 0 8.34 KB
#5877 AllCycleMoreComplexBody net6.0 79.1μs 131ns 507ns 0.118 0 0 9.51 KB
#5877 AllCycleMoreComplexBody netcoreapp3.1 70μs 105ns 405ns 0.139 0 0 10.36 KB
#5877 AllCycleMoreComplexBody net472 55.6μs 65ns 252ns 1.88 0.0276 0 11.85 KB
#5877 ObjectExtractorSimpleBody net6.0 146ns 0.145ns 0.56ns 0.00399 0 0 280 B
#5877 ObjectExtractorSimpleBody netcoreapp3.1 203ns 0.154ns 0.575ns 0.00369 0 0 272 B
#5877 ObjectExtractorSimpleBody net472 167ns 0.268ns 1.04ns 0.0446 0 0 281 B
#5877 ObjectExtractorMoreComplexBody net6.0 3.13μs 2.26ns 8.46ns 0.0531 0 0 3.78 KB
#5877 ObjectExtractorMoreComplexBody netcoreapp3.1 3.95μs 2.88ns 11.1ns 0.0499 0 0 3.69 KB
#5877 ObjectExtractorMoreComplexBody net472 3.75μs 2.01ns 7.53ns 0.601 0.00562 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 36.9μs 7.12ns 25.7ns 0.444 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.9μs 25.3ns 94.6ns 0.436 0 0 32.4 KB
master EncodeArgs net472 66.2μs 41.1ns 154ns 5.16 0.0661 0 32.5 KB
master EncodeLegacyArgs net6.0 72.8μs 30.6ns 110ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 105μs 145ns 560ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 152μs 135ns 522ns 0.306 0 0 2.15 KB
#5877 EncodeArgs net6.0 36.5μs 21ns 78.4ns 0.446 0 0 32.4 KB
#5877 EncodeArgs netcoreapp3.1 56μs 44.5ns 167ns 0.443 0 0 32.4 KB
#5877 EncodeArgs net472 64.6μs 32.4ns 125ns 5.17 0.0646 0 32.5 KB
#5877 EncodeLegacyArgs net6.0 72.4μs 26.7ns 104ns 0 0 0 2.14 KB
#5877 EncodeLegacyArgs netcoreapp3.1 111μs 274ns 1.06μs 0 0 0 2.14 KB
#5877 EncodeLegacyArgs net472 152μs 85.8ns 332ns 0.306 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 184μs 430ns 1.61μs 0 0 0 2.42 KB
master RunWafRealisticBenchmark netcoreapp3.1 195μs 99.7ns 360ns 0 0 0 2.37 KB
master RunWafRealisticBenchmark net472 209μs 38.9ns 151ns 0.313 0 0 2.43 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 63.9ns 230ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 129μs 71.2ns 266ns 0 0 0 1.45 KB
master RunWafRealisticBenchmarkWithAttack net472 139μs 46.2ns 173ns 0.208 0 0 1.48 KB
#5877 RunWafRealisticBenchmark net6.0 184μs 238ns 922ns 0 0 0 2.42 KB
#5877 RunWafRealisticBenchmark netcoreapp3.1 197μs 375ns 1.45μs 0 0 0 2.37 KB
#5877 RunWafRealisticBenchmark net472 210μs 62.4ns 242ns 0.314 0 0 2.43 KB
#5877 RunWafRealisticBenchmarkWithAttack net6.0 124μs 46ns 172ns 0 0 0 1.46 KB
#5877 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 131ns 508ns 0 0 0 1.45 KB
#5877 RunWafRealisticBenchmarkWithAttack net472 139μs 59.1ns 229ns 0.207 0 0 1.48 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5877

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 263.16 KB 267.16 KB 4 KB 1.52%

Fewer allocations 🎉 in #5877

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 255.76 KB 253.49 KB -2.27 KB -0.89%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 60.6μs 753ns 7.49μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 52.6μs 186ns 697ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 38.1μs 127ns 476ns 0 0 0 59.12 KB
master StringConcatAspectBenchmark net6.0 317μs 1.7μs 9.46μs 0 0 0 255.76 KB
master StringConcatAspectBenchmark netcoreapp3.1 339μs 1.89μs 12.4μs 0 0 0 263.16 KB
master StringConcatAspectBenchmark net472 284μs 7.33μs 71.1μs 0 0 0 278.53 KB
#5877 StringConcatBenchmark net6.0 58μs 795ns 7.79μs 0 0 0 43.34 KB
#5877 StringConcatBenchmark netcoreapp3.1 52.9μs 101ns 350ns 0 0 0 42.64 KB
#5877 StringConcatBenchmark net472 37.8μs 118ns 443ns 0 0 0 59.04 KB
#5877 StringConcatAspectBenchmark net6.0 306μs 1.59μs 7.48μs 0 0 0 253.49 KB
#5877 StringConcatAspectBenchmark netcoreapp3.1 341μs 2.83μs 26.8μs 0 0 0 267.16 KB
#5877 StringConcatAspectBenchmark net472 292μs 7.28μs 70.5μs 0 0 0 278.53 KB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area:tests unit tests, integration tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@NachoEchevarria @andrewlock