Making the C API more convenient to use

[kaivalnp]

1. Can we start publishing the shared library for the C API (libfaiss_c.so) to Conda, so that users don't have to build it locally?
2. libfaiss_c.so is currently linked to libfaiss.so, but we don't have optimized versions of the library (AVX2, AVX512, SVE, etc). This could be overcome offline with something like patchelf (replace libfaiss.so dependency with optimized version), but can we start creating (and publishing) equivalent versions of the C API like libfaiss_avx2.so -> libfaiss_c_avx2.so, etc

[mengdilin]

@kaivalnp How did you access the C API libfaiss_c.so from conda installed faiss? AFAICT, we don't upload it to conda from looking at the conda build script https://github.com/facebookresearch/faiss/blob/main/conda/faiss/build-lib.sh#L11

If you want faiss to link to the right build from cmake, I think you can try replacing the linking with link_to_faiss_lib a utility that will link to the optimized faiss version that is being built like

faiss/tests/CMakeLists.txt

Lines 43 to 45 in aff6bfc

	include(../cmake/link_to_faiss_lib.cmake)
	link_to_faiss_lib(faiss_test)

[kaivalnp]

@mengdilin I built it locally, since it is not available from conda -- I opened this issue to ask for the C API to be published to conda so that users don't have to build it themselves

link_to_faiss_lib seems appropriate, but if we were to publish the C API to conda, we'll need separate versions of libfaiss_c.so (like libfaiss_c_avx2.so, libfaiss_c_avx512.so, etc) which are linked to the correct library (in line with optimized versions of libfaiss.so)

[mengdilin]

Gotcha, will back it into a feature request and follow up with the team on the work

[kaivalnp]

Thanks @mengdilin! Do let me know if I can help with some PRs (in case you agree with the suggestions)..

[mengdilin]

@kaivalnp if you are happy to submit PRs to expose C-APIs to conda publication with validation steps, I am happy to review your PRs :D

Right now the team is a bit overwhelmed with other asks so we may not get to this ask as quickly

[kaivalnp]

@mengdilin I'm not super familiar with the publish process (or the repo in general), but I took an attempt in #4186 :)