[Perf] Linux/x64: 4 Regressions on 2/25/2024 4:37:10 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 22.04
Queue	TigerUbuntu
Baseline	f32c428c86b4cc41e88e2e5a750c37dfb354e33a
Compare	5ef47c852ffd51aaeb52e34391fe4fed261c9f26
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ToString_MultipleSegments - Duration of single invocation 📝 - Benchmark Source 📈 - ADX Test Multi Config Graph	87.46 μs	101.66 μs	1.16	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

From #98623

Improvements:
dotnet/perf-autofiling-issues#29825
dotnet/perf-autofiling-issues#29808

[mangod9]

@EgorBo is this by design, so it could be closed?

[EgorBo]

@EgorBot -amd64 -intel -perf -commit 79dd9ba vs fab69ef --disasm

using BenchmarkDotNet.Attributes;
using System.Text;

public class MyProgram
{
    const int LOHAllocatedStringSize = 100_000;
    private string _stringLOH;
    private string _string100;
    private StringBuilder _builderSingleSegment100;
    private StringBuilder _builderSingleSegmentLOH;
    private StringBuilder _builderMultipleSegments100;
    private StringBuilder _builderMultipleSegmentsLOH;

    [GlobalSetup(Target = nameof(ToString_MultipleSegments))]
    public void Setup_ToString_MultipleSegments()
    {
        _builderMultipleSegments100 = Append_Char(100); // 16 + 32 + 48 + 96 char segments
        _builderMultipleSegmentsLOH = Append_Char(LOHAllocatedStringSize);
    }

    public StringBuilder Append_Char(int length)
    {
        StringBuilder builder = new StringBuilder();
        for (int i = 0; i < length; i++)
            builder.Append('a');
        return builder;
    }

    [Benchmark]
    [Arguments(LOHAllocatedStringSize)]
    public string ToString_MultipleSegments(int length) => (length == 100 ? _builderMultipleSegments100 : _builderMultipleSegmentsLOH).ToString();
}

[EgorBot]

Benchmark results on Intel

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
  Job-JXBPJF : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  Job-BMGTJF : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Method	Toolchain	length	Mean	Error	Ratio	Code Size
ToString_MultipleSegments	Main	100000	72.87 μs	1.077 μs	1.00	262 B
ToString_MultipleSegments	PR	100000	110.64 μs	1.693 μs	1.52	262 B

BDN_Artifacts.zip

Flame graphs: Main vs PR 🔥
Hot asm: Main vs PR
Hot functions: Main vs PR

For clean perf results, make sure you have just one [Benchmark] in your app.

[EgorBot]

Benchmark results on Amd

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
AMD EPYC 7763, 1 CPU, 16 logical and 8 physical cores
  Job-CNQFTW : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-FMFWZL : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2

Method	Toolchain	length	Mean	Error	Ratio	Code Size
ToString_MultipleSegments	Main	100000	110.0 μs	0.37 μs	1.00	262 B
ToString_MultipleSegments	PR	100000	145.6 μs	0.31 μs	1.32	262 B

BDN_Artifacts.zip

Flame graphs: Main vs PR 🔥
Hot asm: Main vs PR
Hot functions: Main vs PR

For clean perf results, make sure you have just one [Benchmark] in your app.

[mangod9]

@EgorBo so the regression still exists?

[EgorBo]

@EgorBo so the regression still exists?

It does, but there is not much we can do here.
Previously we used to call native memmove directly (in coop mode), now we call managed SpanHelpers.Memmove instead that forwards back to native memmove (with gc transition) for large sizes, so we effectively pay for that wrapper. It only happens for large sizes.

Since there are more improvements (11) than regressions (4) and the fact that that PR:

1. Fixed a bug on NAOT: NativeAOT: Memset/Memcpy helpers don't throw NRE #95517
2. Made Memmove more suspension friendly: memcpy/memset helpers on CoreCLR may not be suspension-friendly #98620

we can close it.