Performance regressions in BaseBenchmarks due to #54647 (Cleanup MemoryRef)

[Zentrik]

Both [["array", "index", ("sumeach_view", "BitMatrix")]] and [["array", "index", ("sumlinear_view", "BitMatrix")]] have a ~50% regression in min wall time.

[["union", "array", ("skipmissing", "filter", "Union{Missing, Int8}", 1)]], [["union", "array", ("skipmissing", "filter", "Union{Missing, Int64}", 1)]], [["union", "array", ("skipmissing", "filter", "Bool", 0)]] and [["union", "array", ("skipmissing", "filter", "Int64", 0)]] have a ~20% regression in min wall time.

All were bisected to fa038d9.
Here's the nanosoldier report, https://github.com/JuliaCI/NanosoldierReports/blob/master/benchmark/by_date/2024-06/10/report.md.
The BitMatrix regression doesn't show up in the report due to a high tolerance, the data is at https://tealquaternion.camdvr.org/compare.html?start=b8e714dfa3d81edd56ccda853322b6e42d94cf60&end=77c28ab286f48afe2512e2ae7f7310b87ca3345e&stat=min-wall-time&name=BitM instead.

[oscardssmith]

that's rather surprising... It was basically just a rename?

[Zentrik]

c8ae7a0 fixed the regression, so seems like it definitely was due to fa038d9.

[vtjnash]

I think #54681 was merged to revert it

[Zentrik]

No, I don't think so? If you look at the Nanosoldier report you'll see the commit range b8e714d...77c28ab includes #54681 and yet we saw a regression.

Though a lot of these benchmarks did improve with 17857ca and something in 58b239c...6e5e87b, looks like [["union", "array", ("skipmissing", "filter", "Union{Missing, Int64}", 1)]] still hasn't improved.

https://tealquaternion.camdvr.org/graphs.html?start=2024-06-04&benchmark=union.array.%28skipmissing%2C+filter%2C+Union%7BMissing%2C+Int64%7D%2C+1%29&stat=min-wall-time&kind=raw