-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce wheel package size for faiss-gpu CUDA 11.0 build #57
Comments
Relevant pytorch/pytorch#56055 |
Seems one approach is to drop architecture-specific binary in CUDA libraries via nvprune \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_70,code=sm_70 \
-gencode arch=compute_75,code=sm_75 \
-gencode arch=compute_80,code=sm_80 \
-gencode arch=compute_80,code=compute_80 \
-o /usr/local/cuda/lib64/libcublas_static_slim.a \
/usr/local/cuda/lib64/libcublas_static.a Currently there are four dependencies, and applying
In Python 3.9, the original file size of |
The major problem is that CUDA 11.0 splits cublasLt API into a different static lib, and that seems to significantly increase the final binary size. In CUDA 10.x, cublasLt API was within the single static lib.
|
Strangely, faiss does not use
|
Ok, changing the order of linker flag in |
With CUDA 11.6, the resulting wheel further goes up to 345MB in Linux. After |
Alternative is to give up static linking and relies on dynamic linking. This will significantly reduce the wheel size, while requires users to install CUDA runtime libraries elsewhere. |
With avx2 extension, the package is ~430MB. |
It seems there are CUDA runtime packages on PyPI. |
Hi! Did you consider to place package on GitLab PyPI index or place it to dockerhub as image? ping me if you need help |
@theLastOfCats You can manually download packages from the release page. |
Hi @kyamagu! For your reference, by changing from static linking to dynamic linking of CUDA, the wheel size has been reduced to 63MB. It seems possible to reduce the wheel size to less than 60MB by either narrowing down the target architecture or switching from static linking to dynamic linking of OpenBLAS. Fork Repository: https://github.com/Di-Is/faiss-wheels/tree/pypi-cuda Build Script# Test CMD
CPU_TEST_CMD="pytest {project}/faiss/tests && pytest -s {project}/faiss/tests/torch_test_contrib.py"
GPU_TEST_CMD="cp {project}/faiss/tests/common_faiss_tests.py {project}/faiss/faiss/gpu/test/ && pytest {project}/faiss/faiss/gpu/test/test_*.py && pytest {project}/faiss/faiss/gpu/test/torch_*.py"
# Common Setup
export CIBW_BEFORE_ALL="bash scripts/build_Linux.sh"
export CIBW_TEST_COMMAND="${CPU_TEST_CMD}"
export CIBW_BEFORE_TEST_LINUX="pip install torch --index-url https://download.pytorch.org/whl/cpu"
export CIBW_ENVIRONMENT_LINUX="FAISS_OPT_LEVEL=${FAISS_OPT_LEVEL:-generic} BUILD_PARALLELISM=${BUILD_PARALLELISM:-3} CUDA_VERSION=12.1"
export CIBW_DEBUG_KEEP_CONTAINER=TRUE
if [ "$FAISS_ENABLE_GPU" = "ON" ]; then
if [ "$CONTAINER_GPU_ACCESS" = "ON" ]; then
export CIBW_TEST_COMMAND="${CIBW_TEST_COMMAND} && ${GPU_TEST_CMD}"
export CIBW_CONTAINER_ENGINE="docker; create_args: --gpus all"
export -n CIBW_BEFORE_TEST_LINUX
fi
export CIBW_ENVIRONMENT_LINUX="${CIBW_ENVIRONMENT_LINUX} FAISS_ENABLE_GPU=ON"
export CIBW_REPAIR_WHEEL_COMMAND="auditwheel repair -w {dest_dir} {wheel} --exclude libcublas.so.12 --exclude libcublasLt.so.12 --exclude libcudart.so.12"
else
export CIBW_ENVIRONMENT_LINUX="${CIBW_ENVIRONMENT_LINUX} FAISS_ENABLE_GPU=OFF"
export CIBW_REPAIR_WHEEL_COMMAND="auditwheel repair -w {dest_dir} {wheel}"
fi
python3 -m cibuildwheel --output-dir wheelhouse --platform linux |
@Di-Is CUDA backward compatibility is complicated, and the PyPI release should not expect any external dependency other than a few linked to CPython binary. https://github.com/pypa/manylinux You can build a source package for your environment, but that wheel will not be compatible with other environments. |
I believe that installing the appropriate Nvidia drivers is not a matter of package management but rather a part of system setup, and the responsibility for execution lies with the user.
It is correct that wheel files should be self-contained. |
If the following conditions are met, Faiss installed from the created wheel should work properly.
1.As mentioned earlier, it is the user's responsibility. |
This is not a matter of My current approach is to at least leave the source distribution that works with any custom environment. Right now, I can't spend time on the GPU binary distribution, but you can try designing a build and test matrix to resolve the issues in the above configurations. |
Hi @kyamagu, Will all releases until pypi is resolved, have wheel packages available for download? I see currently only 1.7.3 has this. This is missing in 1.7.4 and 1.8.0 releases. Thanks! |
@CandiedCode Currently, there is no plan to support GPU binary wheels. You can build the source package on your environment. |
The CUDA 11.0 build in #56 bloats the wheel package size from 85.5 MB to 216.5 MB. Needs to investigate file size reduction.
The text was updated successfully, but these errors were encountered: