Regressions in System.Formats.Tar.Tests.Perf_TarFile

[performanceautofiler]

Run Information

Architecture	x64
OS	alpine 3.15
Baseline	9d84900aedd34a7d15e2fb8b92614f0429919061
Compare	2f82cb8c2109fe5c9119ed41d7d2c5228cf253ec
Diff	Diff

Regressions in System.Formats.Tar.Tests.Perf_TarFile

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateFromDirectory_Stream - Duration of single invocation	28.27 μs	43.89 μs	1.55	0.28	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Tar.Tests.Perf_TarFile*'

Payloads

Baseline
Compare

Histogram

System.Formats.Tar.Tests.Perf_TarFile.CreateFromDirectory_Stream

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.894984671787704 > 30.216427792291704.
IsChangePoint: Marked as a change because one of 6/2/2022 12:41:31 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -67.55426562077355 (T) = (0 -44452.910097712476) / Math.Sqrt((1284770.026738758 / (37)) + (539136.6577802796 / (34))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (34) - 2, .025) and -0.519244133769709 = (29259.88595882297 - 44452.910097712476) / 29259.88595882297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

Caused by #69469 cc @carlossanlop

[AndyAyersMS]

Also see dotnet/perf-autofiling-issues#5834

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	alpine 3.15
Baseline	9d84900aedd34a7d15e2fb8b92614f0429919061
Compare	2f82cb8c2109fe5c9119ed41d7d2c5228cf253ec
Diff	Diff

Regressions in System.Formats.Tar.Tests.Perf_TarFile

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateFromDirectory_Stream - Duration of single invocation	28.27 μs	43.89 μs	1.55	0.28	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Tar.Tests.Perf_TarFile*'

Payloads

Baseline
Compare

Histogram

System.Formats.Tar.Tests.Perf_TarFile.CreateFromDirectory_Stream

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.894984671787704 > 30.216427792291704.
IsChangePoint: Marked as a change because one of 6/2/2022 12:41:31 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -67.55426562077355 (T) = (0 -44452.910097712476) / Math.Sqrt((1284770.026738758 / (37)) + (539136.6577802796 / (34))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (34) - 2, .025) and -0.519244133769709 = (29259.88595882297 - 44452.910097712476) / 29259.88595882297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	AndyAyersMS
Labels:	area-System.IO, tenet-performance, tenet-performance-benchmarks, untriaged, refs/heads/main, RunKind=micro, Regression, CoreClr, x64, alpine 3.12
Milestone:	-

[DrewScoggins]

This also regressed on

• Ubuntu-Arm64 [Perf] Changes at 6/2/2022 5:13:37 AM perf-autofiling-issues#5960
• Ubuntu-Arm64 (Ampere) [Perf] Changes at 6/2/2022 5:13:37 AM perf-autofiling-issues#5930

[adamsitnik]

#69469 added some new logic that we just need to perform to be correct, so I am closing this as by design.