Update loops in CpuMath to be more efficient

[jwood803]

Fixes issue #835

[Ivanidzo4ka]

Did you run any benchmarks to make sure it bring speed improvements?

[danmoseley]

@jwood803 the end to end benchmarks are documented here. They take a bit of time to run.

There are micro benchmarks for CPUMath specifically which might be best to start with. They are here

[danmoseley]

Oh, I just realized this is a duplicate of #994. That was not linked properly to #835.

As you see in #994, it is unfortunately not at all clear that this will actually help. See comment.

@tannergooding perhaps we should close #835 without action.

[tannergooding]

creating a temp for pDstEnd - 4 is probably better, as that would better ensure that each loop isn't doing pDstEnd - 4 before the comparison

[tannergooding]

@tannergooding perhaps we should close #835 without action.

I don't believe so, both PRs have the same problem, which is that each iteration of the loop may do a substraction/addition as part of the comparison.

As commented on #1177 (comment), creating a temp var pLoopEnd = pDstEnd - 4; and then using it in the comparison (while (pDstCurrent <= pLoopEnd)) will help ensure that the subtraction happens once, rather than once per loop iteration.

[jwood803]

@danmosemsft Thanks a lot for letting me know about the benchmarks! I ran it two different times with the PredictionEngine benchmark. Below are the results. So far it looks like the changes provide a bit better performance.

Without changes:

With changes:

[tannergooding]

Could you update the SSE case as well?

[tannergooding]

Where is pDstEnd defined, I wouldn't expect this could be a constant....

[danmoseley]

@tannergooding does this look OK now?

Nice perf improvement @jwood803 thank you!

[tannergooding]

nit: stray newline here

[jwood803]

This should be updated to remove the extra new line.

[tannergooding]

LGTM

[eerhardt]

This was a bad change/merge

Error	CS0266	Cannot implicitly convert type 'float*' to 'int'. An explicit conversion exists (are you missing a cast?)	Microsoft.ML.CpuMath(netcoreapp3.0)	F:\git\machinelearning2\src\Microsoft.ML.CpuMath\SseIntrinsics.cs	758	Active

                float* pDstEnd = pdst + dst.Length;
                float* pDstCurrent = pdst;
                int destinationEnd = pDstEnd - 4;

pDstEnd is a float*. Subtracting 4 still gives you a float*, which is not convertable to int.

The Intrinsics/netcoreapp3.0 build is broken because of this.

@shauheen @danmosemsft - maybe now would be a good time to get a netcoreapp3.0 CI leg running.

I've opened #1495 for the build break.