Add benchmarks for Nullable

[nalimilan]

Includes in particular a reduced version of NullableArrays to measure loop
performance.

This is my first experience with this package, so please give my advice on what benchmarks should look like.

[KristofferC]

I would use the samerand function instead of just seeding it in case more benchmarks to this file are added in between the current ones.

[jrevels]

Like @KristofferC mentioned, you can do using .RandUtils and then replace uses of rand in this module with samerand.

[jrevels]

Thanks for the contribution, the more benchmarks the merrier!

Since the benchmark parameters here don't yet exist in etc/params.jld (that file is populated during the tuning process, which has to be executed on Nanosoldier), the method which loads the parameters into the module's BenchmarkGroup is failing.

I should make that method less fragile...let me fix that, then you can rebase this PR onto the fix so that tests here pass.

EDIT: Done with the fix. Feel free to rebase this against master and see if the tests pass.

[nalimilan]

Thanks! I've just rebased against master, used samerand as advised, and improved a few things in the benchmarks.

I'd still like to hear @johnmyleswhite and @DavidGold's comment about these. FWIW, I've just checked that perf_sum and perf_countnulls use SIMD (though perf_countequals doesn't, even after tweaking isequal).

[nalimilan]

Oh, and it is fine/recommended to test so many types in a loop like I do? If that takes too much time, I can keep only a subset of them.

[davidagold]

Thanks for doing this, Milan. Will this be coordinated with JuliaLang/julia#18510 ?

I can think of more benchmarks for NullableArrays. In particular, we should have benchmarks for iterating over zipped NullableVectors. I'll see about modernizing NullableArrays/perf.

[nalimilan]

Thanks for doing this, Milan. Will this be coordinated with JuliaLang/julia#18510 ?

The idea was that if we merge these benchmarks first, they will allow checking that the move to hasvalue doesn't affect performance. Then we can also apply the same change to the isnull field of NullableArray.

I can think of more benchmarks for NullableArrays. In particular, we should have benchmarks for iterating over zipped NullableVectors. I'll see about modernizing NullableArrays/perf.

Could you give an example of what you mean? Is that just something similar to perf_countequals, but using zip instead of eachindex? Anyway, once we have more operators on Nullable in base, we'll be able to add more benchmarks.

[jrevels]

Oh, and it is fine/recommended to test so many types in a loop like I do? If that takes too much time, I can keep only a subset of them.

Should be fine. The majority of turnaround time for Nanosoldier jobs is actually building Julia rather than running benchmarks.

The idea was that if we merge these benchmarks first, they will allow checking that the move to hasvalue doesn't affect performance.

Just making sure you're aware - benchmarks merged here won't immediately be available via Nanosoldier, as they have to undergo a tuning process (which I can trigger tomorrow, if this PR is merged by then).

Other than triggering the process and committing the results, the tuning process is pretty automated. When I get the chance, I should probably set up a cron job that does this nightly...

[jrevels]

Also, let me know once you're ready for me to merge this and I'll do so

[nalimilan]

Thanks. Should be good to merge now. It would be nice if you could run it on Nanosoldier, as it would make a reference point for current PRs against Base.