Add test for how to directly load new MKL ILP64

[staticfloat]

MKL added a new library in v2022 that allows us to directly link against
a library that provides ILP64-namespaced symbols.

[giordano]

This should have an entry in the changelog in the README? Does this make libblastrampoline require MKL 2022 or it still works with previous versions?

[giordano]

Oh wow, you need to find the right incantation to make this work: #52 (comment)

[staticfloat]

@ViralBShah okay, so there are a few things going on here:

1. To link against libmkl_intel_ilp64.so, we first have to manually link against libmkl_core.so and libmkl_intel_thread.so. That's not a big burden, and it's easily done.

2. We have to add some extra smarts here (that is sadly a breaking change) where when we call lbt_forward() we can hint to LBT which symbol suffix it should try first. This is to avoid false-positives related to symbol naming: it turns out that libmkl_intel_ilp64.so contains both dgemm_ AND dgemm_64_. "Alright", you say, "They're just providing ILP64 symbols under both names for compatibility for how it used to be". But it gets worse; they also have dgemm_64_ within libmkl_intel_lp64.so, and those symbols are... wait for it.... ILP64. This makes no sense to me, and IMO should be a bug that we raise with Intel.

The result of this is that we can't rely on a heuristic to tell the difference between libmkl_intel_ilp64.so and libmkl_intel_lp64.so; we must hint to the LBT loader that it should look for dgemm_64_ first when loading libmkl_intel_ilp64.so, and that it should look for dgemm_ first when loading libmkl_intel_lp64.so.

3. The windows dynamic libraries seem to be missing from the Anaconda releases. I'm not sure if this is Intel's fault or Anaconda's. On one hand, the intel docs state that the library to link against for dynamic linking is mkl_intel_{i,}lp64_dll.lib, which is.... an unusual file extension for a dynamic library, so I wouldn't be surprised if the Anaconda folks just missed copying it due to the unusual file extension. I think we need to bug intel for a proper .dll that we can link against via dlopen().

All that being said, I do have Linux and macOS working so far (and we can load LP64 and ILP64 versions of MKL simultaneously, which is nice).

[ViralBShah]

I have emailed Intel.

[imciner2]

On one hand, the intel docs state that the library to link against for dynamic linking is mkl_intel_{i,}lp64_dll.lib, which is.... an unusual file extension for a dynamic library,

A .lib file on Windows is an import library for the DLL it is associated with, so it contains the definition of all symbols that are publicly exported from the DLL that other programs can use. It should only be needed if you are letting the linker find all the symbols, and you shouldn't need it if you are going to do a dlopen() call and find the symbols manually.

[ViralBShah]

@staticfloat If what @imciner2 says is true, should we be able to go ahead on windows?

[staticfloat]

I followed the advice of the Intel people, and it looks like it works on all platforms! They suggested using libmkl_rt directly and completely ignoring the libmkl_intel_{i,}lp64 files, which seems to work!

So with this change, (which we should ship in Julia v1.8, and which is backwards-incompatible thanks to the suffix_hint parameter that is given to lbt_forward()) we will be able to do the following in MKL.jl to load dual-interface:

# Load ILP64 forwards
lbt_forward(libmkl_rt; clear=true, suffix_hint="64")
# Load LP64 forward
lbt_forward(libmkl_rt; suffix_hint="")