C# multithreaded results and high resolution timer issue

[enki42]

I notice in the results page (https://plummerssoftwarellc.github.io/PrimeView) there is no multithreaded c# results. Is this cause it just runs output exe as default (i.e. no arguments passed in)? Solution 1 does have multithreading in it so looks like it can give multithreaded results. (sorry if this is a stupid question)

And while @Kinematics code looks great for the optimization of single threaded perf I feel like the multithreaded side can be improved on. For example:

• Parallel.For makes use of the thread pool which may waste time creating threads in the thread pool. Seems like manually creating all the threads you want before starting the benchmark and then just start the processing during the benchmark would give better results. (not sure if that counts as cheating but doesn't seem to break the rules in CONTRIBUTING.md)
• Constantly polling the high resolution clock to see if 5 seconds is up seems like a slow way to do it. Use a timer so you can set a flag to be checked when the thread should be done

Do benchmarks in different languages handle the high resolution timer vs normal timer issue? If one lanaguage isn't using the high resolution timer not only is the results off but the perf hit using it isn't something they are dealing with.

[rbergen]

PrimeView shows all "Primes standard output format" results that are generated by the solutions that provide a Dockerfile and don't block its usage for automated benchmarks. If a solution does include provisions for multithreading but they don't show up in the results that PrimeView presents, then I can only assume the Dockerfile for that solution does not (successfully) run the multithreaded variation(s) that the solution includes.

From day 1, one of the ground rules of this project have been that improvements to existing solutions are very welcome. If you think improvements can be made to a solution then code them, and then please check if your changes indeed improve performance before submitting your changes.

Different languages provide different means for keeping track of time, with varying inherent granularity. For (modern) languages with a fair shot at one of the top spots, we do check if a timing approach is used that is sufficiently accurate/granular. If multiple accurate means to measure time exist within a language then it's up to the solution developer to pick the one that's most clever in their language of choice.

[Kinematics]

I tested using a timer instead of a stopwatch on the single threaded version, and it had no measurable effect. Measurements of each were identical within the limit of noise, covering the same range of about 40 iterations out of 10,000. Considering how math heavy each prime run is, the addition of a single math check per run is rather unlikely to move the needle, even for a high resolution timer.

Since the code for running the multithreaded versions is basically the same, aside from one extra time lookup every ~8-16 iterations, I don't have reason to think that it would be more relevant there. I might come back to it after checking the threading concerns.

On the standard output results, yeah, the default run does not include multi-threading. It either runs the models single-threaded or multi-threaded, but not both. I vaguely remember an intent that the official scores should be single-threaded only, and I also think I remember that the containerized versions performed really poorly with multi-threading. I uninstalled Docker when they changed their policies, so haven't messed with containers in years.

It would not be too difficult to add a flag that will cause the solution to run everything both single-threaded and multi-threaded.

As for the threading code itself, experimenting with that would be doable, but I'm not sure I'd have time to get to it in the immediate future. I'd be happy to see your own results, though.

A quick test on the threading: With an 8 core processor with hyperthreading, I get a 5.7x speedup with 8 threads and 7.0x speedup with 16 threads, using Parallel.For(). Manual threading would want to beat that.

[rbergen]

I vaguely remember an intent that the official scores should be single-threaded only,

In the context of this project that is still the case. Dave did make an episode where he compared the top multithreaded solutions, but the careful observer may have noticed that that was the lead up to an episode where he compared CPU performance to GPU performance - and with the latter parallelization really is a thing.

and I also think I remember that the containerized versions performed really poorly with multi-threading.

Docker Desktop in Windows does, or at least used to - I also stopped using that for the same reason. Docker Engine does much better in my experience, probably because it runs directly in WSL and doesn't have to cross the Windows/WSL boundary for a whole heap of things.