You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rx.NET has a benchmark suite, but it currently works only for .NET Framework, and not for .NET 6.0 and later.
If we change the <TargetFramework> to net6.0, the benchmarks fail to run. This seems to be because BenchmarkDotNet expects benchmark projects to use the Release build configuration, but the Rx.NET benchmark project has multiple configurations with the goal of being able to run the benchmarks against older Rx.NET libraries.
This seems like it should offer the following advantages:
we can introduce new benchmarks to address performance scenarios not previously covered, and run them against older libraries to see whether the scenario regressed
we can rerun benchmarks against older libraries on different hardware
That second point is useful if we want to start tracking performance on processor architectures for which we did not have historical data. It's also useful if we want to change the machine type on which we regularly run benchmarks—in addition to being able to re-establish baselines for the current version against new hardware, we can see if the change in performance over time looked different on the new hardware.
Currently it can run benchmarks back as far as v3.1.1.
The problem is that that libraries are selected by choosing different build configurations, and this conflicts with how BenchmarkDotNet wants to do things. (The symptom is a build failure, which I think is caused by inconsistent setting of the CURRENT compilation symbol, meaning that BenchmarkDotNet generates some source code trying to use a benchmark that isn't actually present.)
BenchmarkDotNet now has ways of configuring which external library versions to use. E.g., I found this in a Microsoft blog:
So I think it should be possible for us to use this mechanism to change the library references. (We'd also need to find some way to control the preprocessors defines so that we can omit benchmarks that just can't run on older versions.)
The text was updated successfully, but these errors were encountered:
Rx.NET has a benchmark suite, but it currently works only for .NET Framework, and not for .NET 6.0 and later.
If we change the
<TargetFramework>
tonet6.0
, the benchmarks fail to run. This seems to be because BenchmarkDotNet expects benchmark projects to use the Release build configuration, but the Rx.NET benchmark project has multiple configurations with the goal of being able to run the benchmarks against older Rx.NET libraries.This seems like it should offer the following advantages:
That second point is useful if we want to start tracking performance on processor architectures for which we did not have historical data. It's also useful if we want to change the machine type on which we regularly run benchmarks—in addition to being able to re-establish baselines for the current version against new hardware, we can see if the change in performance over time looked different on the new hardware.
Currently it can run benchmarks back as far as v3.1.1.
The problem is that that libraries are selected by choosing different build configurations, and this conflicts with how BenchmarkDotNet wants to do things. (The symptom is a build failure, which I think is caused by inconsistent setting of the
CURRENT
compilation symbol, meaning that BenchmarkDotNet generates some source code trying to use a benchmark that isn't actually present.)BenchmarkDotNet now has ways of configuring which external library versions to use. E.g., I found this in a Microsoft blog:
So I think it should be possible for us to use this mechanism to change the library references. (We'd also need to find some way to control the preprocessors defines so that we can omit benchmarks that just can't run on older versions.)
The text was updated successfully, but these errors were encountered: