Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't run any micro benchmarks #3096

Closed
IDisposable opened this issue Jun 23, 2023 · 10 comments
Closed

Can't run any micro benchmarks #3096

IDisposable opened this issue Jun 23, 2023 · 10 comments

Comments

@IDisposable
Copy link
Contributor

Trying to do a simple run
dotnet run -c Release -f net8.0 --filter System.Collections* --logBuildOutput but it fails with some odd error I can't find referenced in the Troubleshooting docs. I get the same error ALSO if I give it a CoreRun argument pointing at a private build.

// Build Error: Standard output:

 Standard error:
 C:\Users\idisp\.nuget\packages\system.threading.channels\6.0.0\buildTransitive\netcoreapp2.0\System.Threading.Channels.targets(4,5): error : System.Threading.Channels doesn't support net8.0. Consider updating your TargetFramework to netcoreapp3.1 or later. [C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\d6511fa9-52e2-4936-a869-9a57d797c813\BenchmarkDotNet.Autogenerated.csproj]
@IDisposable
Copy link
Contributor Author

Full logs

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 628 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// Environment Variables:
        [DOTNET_ROOT_X64] = "C:\Program Files\dotnet"
        [LOGONSERVER] = "\\ZIPPY"
        [PATHEXT] = ".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY;.PYW"
        [PUBLIC] = "C:\Users\Public"
        [MIC_LD_LIBRARY_PATH] = "C:\Program Files (x86)\Common Files\Intel\Shared Libraries\compiler\lib\mic"
        [Path] = "C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Microsoft\jdk-11.0.19.7-hotspot\bin;C:\Program Files\AdoptOpenJDK\jdk-11.0.11.9-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Microsoft VS Code\bin;C:\ProgramData\chocolatey\bin;C:\ProgramData\chocolatey\lib\gsudo\bin\;C:\Program Files\dotnet\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Plantronics\Spokes3G\;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Program Files (x86)\dotnet-core-uninstall\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\Docker\Docker\resources\bin;C:\Program Files\Git\cmd;C:\Program Files\GitHub CLI\;C:\Program Files\nodejs\;C:\Program Files\WindowsApps\Microsoft.PowerShell_7.1.4.0_x64__8wekyb3d8bbwe;C:\Program Files\ffmpeg\bin;C:\Program Files\NVEncC;C:\Program Files\Utils;C:\Users\idisp\AppData\Local\Microsoft\WindowsApps;C:\Users\idisp\AppData\Local\gitkraken\bin;C:\Users\idisp\.dotnet\tools;C:\Users\idisp\AppData\Local\Microsoft\WindowsApps;C:\Program Files\AdoptOpenJDK\jdk-11.0.10.9-hotspot\bin;C:\Users\idisp\AppData\Roaming\npm"
        [WSLENV] = "WT_SESSION::WT_PROFILE_ID"
        [WT_SESSION] = "f162ed4c-bafd-4697-8e41-9d389fc7c174"
        [USERNAME] = "idisp"
        [APPDATA] = "C:\Users\idisp\AppData\Roaming"
        [OS] = "Windows_NT"
        [WT_PROFILE_ID] = "{0caa0dad-35be-5f56-a8ff-afceeeaa6101}"
        [OneDriveConsumer] = "C:\Users\idisp\OneDrive"
        [ProgramFiles] = "C:\Program Files"
        [ProgramW6432] = "C:\Program Files"
        [TMP] = "C:\Users\idisp\AppData\Local\Temp"
        [CommonProgramFiles(x86)] = "C:\Program Files (x86)\Common Files"
        [PROMPT] = "$P$G"
        [JAVA_HOME] = "C:\Program Files\Microsoft\jdk-11.0.19.7-hotspot\"
        [COMPUTERNAME] = "ZIPPY"
        [USERDOMAIN_ROAMINGPROFILE] = "ZIPPY"
        [ChocolateyInstall] = "C:\ProgramData\chocolatey"
        [ChocolateyLastPathUpdate] = "133202795112444183"
        [PROCESSOR_ARCHITECTURE] = "AMD64"
        [HOMEPATH] = "\Users\idisp"
        [LOCALAPPDATA] = "C:\Users\idisp\AppData\Local"
        [PROCESSOR_IDENTIFIER] = "Intel64 Family 6 Model 165 Stepping 2, GenuineIntel"
        [SystemRoot] = "C:\WINDOWS"
        [CMAKE_CXX_COMPILER] = "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\bin\Hostx86\x64\cl.exe"
        [ComSpec] = "C:\WINDOWS\system32\cmd.exe"
        [windir] = "C:\WINDOWS"
        [WSA_PACMAN_HOME] = "C:\Program Files\WSA PacMan"
        [OneDrive] = "C:\Users\idisp\OneDrive - 1904labs"
        [ProgramFiles(x86)] = "C:\Program Files (x86)"
        [ALLUSERSPROFILE] = "C:\ProgramData"
        [configsetroot] = "C:\WINDOWS\ConfigSetRoot"
        [MSBuildLoadMicrosoftTargetsReadOnly] = "true"
        [PSModulePath] = "%ProgramFiles%\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules;"
        [OneDriveCommercial] = "C:\Users\idisp\OneDrive - 1904labs"
        [CommonProgramFiles] = "C:\Program Files\Common Files"
        [HOMEDRIVE] = "C:"
        [USERDOMAIN] = "ZIPPY"
        [SystemDrive] = "C:"
        [FPS_BROWSER_APP_PROFILE_STRING] = "Internet Explorer"
        [INTEL_DEV_REDIST] = "C:\Program Files (x86)\Common Files\Intel\Shared Libraries\"
        [USERPROFILE] = "C:\Users\idisp"
        [IntelliJ IDEA Community Edition] = "C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2020.3.3\bin;"
        [FPS_BROWSER_USER_PROFILE_STRING] = "Default"
        [DriverData] = "C:\Windows\System32\Drivers\DriverData"
        [PROCESSOR_LEVEL] = "6"
        [PROCESSOR_REVISION] = "a502"
        [TEMP] = "C:\Users\idisp\AppData\Local\Temp"
        [CommonProgramW6432] = "C:\Program Files\Common Files"
        [ProgramData] = "C:\ProgramData"
        [NUMBER_OF_PROCESSORS] = "16"
        [DOTNET_MULTILEVEL_LOOKUP] = "0"
// start dotnet  restore /p:EnableUnsafeBinaryFormatterSerialization=true /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\9026f46f-4ef7-4530-9a91-d10187c37b33
C:\Users\idisp\.nuget\packages\system.threading.channels\6.0.0\buildTransitive\netcoreapp2.0\System.Threading.Channels.targets(4,5): error : System.Threading.Channels doesn't support net8.0. Consider updating your TargetFramework to netcoreapp3.1 or later. [C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\9026f46f-4ef7-4530-9a91-d10187c37b33\BenchmarkDotNet.Autogenerated.csproj]
// command took 1.31s and exited with 1
// ***** Done, took 00:00:01 (1.96 sec)   *****
// Found 10 benchmarks:
//   AddGivenSize<Int32>.List: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.HashSet: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Dictionary: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.SortedList: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Queue: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Stack: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ConcurrentDictionary: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ObservableCollection: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ICollection: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.IDictionary: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]

// Build Error: Standard output:

 Standard error:
 C:\Users\idisp\.nuget\packages\system.threading.channels\6.0.0\buildTransitive\netcoreapp2.0\System.Threading.Channels.targets(4,5): error : System.Threading.Channels doesn't support net8.0. Consider updating your TargetFramework to netcoreapp3.1 or later. [C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\9026f46f-4ef7-4530-9a91-d10187c37b33\BenchmarkDotNet.Autogenerated.csproj]

// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\9026f46f-4ef7-4530-9a91-d10187c37b33
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html

// ** Remained 618 (98.4%) benchmark(s) to run. Estimated finish 2023-06-22 22:49 (0h 0m from now) **
// ***** BenchmarkRunner: Finish  *****

// * Export *
  C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\BenchmarkDotNet.Artifacts\results\System.Collections.AddGivenSize_Int32_-report-github.md
  C:\Dev\GitHub\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\BenchmarkDotNet.Artifacts\results\System.Collections.AddGivenSize_Int32_-report-full.json

// * Detailed results *
AddGivenSize<Int32>.List: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
Runtime = ; GC =
There are not any results runs

// * Summary *

BenchmarkDotNet=v0.13.2.2052-nightly, OS=Windows 11 (10.0.22631.1900)
Intel Core i7-10875H CPU 2.30GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=8.0.100-preview.6.23279.4
  [Host] : .NET 8.0.0 (8.0.23.27214), X64 RyuJIT AVX2

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true  IterationTime=250.0000 ms
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1

| Method | Size | Mean | Error | Median | Min | Max |
|------- |----- |-----:|------:|-------:|----:|----:|
|   List |  512 |   NA |    NA |     NA |  NA |  NA |

Benchmarks with issues:
  AddGivenSize<Int32>.List: Job-HGYVGC(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]

// * Legends *
  Size   : Value of the 'Size' parameter
  Mean   : Arithmetic mean of all measurements
  Error  : Half of 99.9% confidence interval
  Median : Value separating the higher half of all measurements (50th percentile)
  Min    : Minimum
  Max    : Maximum
  1 ns   : 1 Nanosecond (0.000000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
Run time: 00:00:00 (0.03 sec), executed benchmarks: 0

Global total time: 00:00:02 (2.59 sec), executed benchmarks: 0
// * Artifacts cleanup *

@IDisposable IDisposable mentioned this issue Jun 23, 2023
Closed
@LoopedBard3
Copy link
Member

Hello! I am unable to repro this locally. Can you try a run after clearing the top-level artifacts directory, otherwise do you still hit this error with a clean repository clone?

@IDisposable
Copy link
Contributor Author

Yes, did a complete clean and build on both my clone of dotnet/runtime and dotnet/performance and it reproduces every way. :(

@LoopedBard3
Copy link
Member

If you run dotnet --info what version(s) are printed? Depending on the versions already installed, you may need to install an updated dotnet version. The net8 versions are available: https://github.com/dotnet/installer#installers-and-binaries if the SDK version from dotnet --info is not at least 8.0.

@IDisposable
Copy link
Contributor Author

IDisposable commented Jun 27, 2023 •
edited
Loading 


C:\Dev\dotnet\runtime>git clean -xdf

C:\Dev\dotnet\runtime>dotnet --info
C:\Dev\dotnet\runtime\.dotnet
.NET SDK:
 Version:   8.0.100-preview.5.23303.2
 Commit:    3fe444af72

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Dev\dotnet\runtime\.dotnet\sdk\8.0.100-preview.5.23303.2\

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.0-preview.5.23280.8
  Architecture: x64
  Commit:       bc78804f5d

.NET SDKs installed:
  8.0.100-preview.5.23303.2 [C:\Dev\dotnet\runtime\.dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.0-preview.5.23302.2 [C:\Dev\dotnet\runtime\.dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.0-preview.5.23280.8 [C:\Dev\dotnet\runtime\.dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 8.0.0-preview.5.23302.2 [C:\Dev\dotnet\runtime\.dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  C:\Dev\dotnet\runtime\global.json

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

C:\Dev\dotnet\runtime>

@LoopedBard3
Copy link
Member

Any ideas @DrewScoggins @caaavik-msft @cincuranet?

The last thing that is coming to mind for me would be to clear the NuGet caches by running dotnet nuget locals -c all.

@IDisposable
Copy link
Contributor Author

The clean of the NuGet cache did it!

C:\Users\idisp>dotnet nuget locals -c all
Clearing NuGet HTTP cache: C:\Users\idisp\AppData\Local\NuGet\v3-cache
Clearing NuGet global packages folder: C:\Users\idisp\.nuget\packages\
Clearing NuGet Temp cache: C:\Users\idisp\AppData\Local\Temp\NuGetScratch
Clearing NuGet plugins cache: C:\Users\idisp\AppData\Local\NuGet\plugins-cache
Local resources cleared.

C:\Users\idisp>cd \dev\dotnet\performance

C:\Dev\dotnet\performance>git clean -xdf

C:\Dev\dotnet\performance>git pull
Already up to date.

C:\Dev\dotnet\performance>dotnet --info
.NET SDK:
 Version:   8.0.100-preview.7.23322.33
 Commit:    29bfd81725

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\8.0.100-preview.7.23322.33\

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.0-preview.6.23321.6
  Architecture: x64
  Commit:       5fd32d9989

.NET SDKs installed:
  5.0.406 [C:\Program Files\dotnet\sdk]
  7.0.304 [C:\Program Files\dotnet\sdk]
  8.0.100-preview.7.23322.33 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-preview.6.23321.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 8.0.0-preview.6.23321.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.19 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 8.0.0-preview.7.23321.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

C:\Dev\dotnet\performance>cd src\benchmarks\micro

C:\Dev\dotnet\performance\src\benchmarks\micro>dotnet run -c Release -f net8.0 --filter System.Collections* --logBuildOutput

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 792 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// Environment Variables:
        [WSLENV] = "WT_SESSION::WT_PROFILE_ID"
        [MIC_LD_LIBRARY_PATH] = "C:\Program Files (x86)\Common Files\Intel\Shared Libraries\compiler\lib\mic"
        [ProgramFiles] = "C:\Program Files"
        [DriverData] = "C:\Windows\System32\Drivers\DriverData"
        [ChocolateyLastPathUpdate] = "133202795112444183"
        [NUMBER_OF_PROCESSORS] = "16"
        [FPS_BROWSER_APP_PROFILE_STRING] = "Internet Explorer"
        [SystemDrive] = "C:"
        [ProgramData] = "C:\ProgramData"
        [WSA_PACMAN_HOME] = "C:\Program Files\WSA PacMan"
        [ProgramW6432] = "C:\Program Files"
        [PROCESSOR_REVISION] = "a502"
        [PROCESSOR_LEVEL] = "6"
        [FPS_BROWSER_USER_PROFILE_STRING] = "Default"
        [PROMPT] = "$P$G"
        [USERPROFILE] = "C:\Users\idisp"
        [ProgramFiles(x86)] = "C:\Program Files (x86)"
        [USERDOMAIN] = "ZIPPY"
        [COMPUTERNAME] = "ZIPPY"
        [PROCESSOR_ARCHITECTURE] = "AMD64"
        [HOMEDRIVE] = "C:"
        [Path] = "C:\Program Files\Python311\Scripts\;C:\Program Files\Python311\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\ProgramData\chocolatey\bin;C:\ProgramData\chocolatey\lib\gsudo\bin\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files (x86)\dotnet-core-uninstall\;C:\Program Files\Docker\Docker\resources\bin;C:\Program Files (x86)\Plantronics\Spokes3G\;C:\Program Files\nodejs\;C:\Program Files\GitHub CLI\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\WindowsApps\Microsoft.PowerShell_7.1.4.0_x64__8wekyb3d8bbwe;C:\Program Files\Utils;C:\Program Files\ffmpeg\bin;C:\Program Files\NVEncC;C:\Users\idisp\AppData\Local\Microsoft\WindowsApps;C:\Users\idisp\AppData\Local\gitkraken\bin;C:\Users\idisp\.dotnet\tools;C:\Users\idisp\AppData\Local\Microsoft\WindowsApps;C:\Users\idisp\AppData\Roaming\npm"
        [OneDrive] = "C:\Users\idisp\OneDrive - 1904labs"
        [HOMEPATH] = "\Users\idisp"
        [MSBuildLoadMicrosoftTargetsReadOnly] = "true"
        [JAVA_HOME] = "C:\Program Files\Microsoft\jdk-11.0.19.7-hotspot\"
        [WT_SESSION] = "1902f877-58da-4854-98b4-e0627ee72282"
        [OS] = "Windows_NT"
        [ComSpec] = "C:\WINDOWS\system32\cmd.exe"
        [PATHEXT] = ".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY;.PYW"
        [TEMP] = "C:\Users\idisp\AppData\Local\Temp"
        [APPDATA] = "C:\Users\idisp\AppData\Roaming"
        [USERNAME] = "idisp"
        [WT_PROFILE_ID] = "{0caa0dad-35be-5f56-a8ff-afceeeaa6101}"
        [INTEL_DEV_REDIST] = "C:\Program Files (x86)\Common Files\Intel\Shared Libraries\"
        [OneDriveCommercial] = "C:\Users\idisp\OneDrive - 1904labs"
        [USERDOMAIN_ROAMINGPROFILE] = "ZIPPY"
        [windir] = "C:\WINDOWS"
        [CMAKE_CXX_COMPILER] = "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\bin\Hostx86\x64\cl.exe"
        [DOTNET_ROOT_X64] = "C:\Program Files\dotnet"
        [PROCESSOR_IDENTIFIER] = "Intel64 Family 6 Model 165 Stepping 2, GenuineIntel"
        [PUBLIC] = "C:\Users\Public"
        [CommonProgramFiles] = "C:\Program Files\Common Files"
        [configsetroot] = "C:\WINDOWS\ConfigSetRoot"
        [TMP] = "C:\Users\idisp\AppData\Local\Temp"
        [ALLUSERSPROFILE] = "C:\ProgramData"
        [SystemRoot] = "C:\WINDOWS"
        [LOCALAPPDATA] = "C:\Users\idisp\AppData\Local"
        [LOGONSERVER] = "\\ZIPPY"
        [ChocolateyInstall] = "C:\ProgramData\chocolatey"
        [CommonProgramFiles(x86)] = "C:\Program Files (x86)\Common Files"
        [CommonProgramW6432] = "C:\Program Files\Common Files"
        [PSModulePath] = "%ProgramFiles%\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules;"
        [OneDriveConsumer] = "C:\Users\idisp\OneDrive"
        [DOTNET_MULTILEVEL_LOOKUP] = "0"
// start dotnet  restore /p:EnableUnsafeBinaryFormatterSerialization=true /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771
  Determining projects to restore...
  Restored C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771\BenchmarkDotNet.Autogenerated.csproj (in 870 ms).
  3 of 4 projects are up-to-date for restore.
// command took 3.78s and exited with 0
// start dotnet  build -c Release /p:EnableUnsafeBinaryFormatterSerialization=true --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771
MSBuild version 17.7.0-preview-23321-03+b58ecd263 for .NET
  Reporting -> C:\Dev\dotnet\performance\artifacts\bin\Reporting\Release\netstandard2.0\Reporting.dll
  BenchmarkDotNet.Extensions -> C:\Dev\dotnet\performance\artifacts\bin\BenchmarkDotNet.Extensions\Release\netstandard2.0\BenchmarkDotNet.Extensions.dll
  MicroBenchmarks -> C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\MicroBenchmarks.dll
  BenchmarkDotNet.Autogenerated -> C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771\bin\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771.dll
Build succeeded.
    0 Warning(s)
    0 Error(s)
Time Elapsed 00:00:19.58
// command took 20s and exited with 0
// ***** Done, took 00:00:24 (24.54 sec)   *****
// Found 10 benchmarks:
//   AddGivenSize<Int32>.List: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.HashSet: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Dictionary: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.SortedList: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Queue: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.Stack: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ConcurrentDictionary: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ObservableCollection: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.ICollection: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
//   AddGivenSize<Int32>.IDictionary: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]

// **************************
// Benchmark: AddGivenSize<Int32>.List: Job-TAMELJ(PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [Size=512]
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet 3c9cb503-defe-4ac6-a76a-e7b28ba2c771.dll --anonymousPipes 2684 2680 --benchmarkName "System.Collections.AddGivenSize<Int32>.List(Size: 512)" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 0 in C:\Dev\dotnet\performance\artifacts\bin\MicroBenchmarks\Release\net8.0\3c9cb503-defe-4ac6-a76a-e7b28ba2c771\bin\Release\net8.0
// BeforeAnythingElse

// Benchmark Process Environment Information:
// Runtime=.NET 8.0.0 (8.0.23.32106), X64 RyuJIT AVX2
// GC=Concurrent Workstation
// HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256
// Job: Job-KPXYGD(PowerPlanMode=00000000-0000-0000-0000-000000

,,,

@IDisposable
Copy link
Contributor Author

Thanks you!

@adamsitnik
Copy link
Member

@IDisposable an alternative is to use our python script that is used by the CI. It downloads the latest SDK, adds it to PATH and forwards your args to the micro benchmarks exe:

py .\scripts\benchmarks_ci.py -f net8.0 --filter $hereGoesYourFilter --coreRun $alsoSupported

@IDisposable
Copy link
Contributor Author

Cool, turns out I had to turn off the py App Execution Alias, and that --corerun is case-sensitive (ugh) then this works

py .\scripts\benchmarks_ci.py -f net8.0 --filter *Frozen* --corerun "C:\Dev\dotnet\runtime_baseline\artifacts\bin\testhost\net8.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\CoreRun.exe" "C:\Dev\dotnet\runtime\artifacts\bin\testhost\net8.0-windows-Release-x64\shared\Microsoft.NETCore.App\8.0.0\CoreRun.exe"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@IDisposable @adamsitnik @LoopedBard3