Replace libcudart.so with PyTorch's CUDA APIs #346
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…andbytes into cuda-bin-switch-and-cli
…bitsandbytes_cpu_so Fixed typo libsbitsandbytes_cpu.so
…no_absmax Gix a bug in (de)quantize_no_absmax with multiple GPUs
Fix bitsandbytes-foundation#157; Add XDG_GREETER_DATA_DIR to ignorelist
…lize_8bit Implement proper serialization of Linear8bitLt
|
The origin from my branch works like below: work@main1[bitsandbytes-torch]:~/.local/lib/python3.8/site-packages/bitsandbytes$ python
Python 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bitsandbytes
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/work/.local/lib/python3.8/site-packages/bitsandbytes/bitsandbytes/libbitsandbytes_cuda117.so
/home/work/.local/lib/python3.8/site-packages/bitsandbytes/bitsandbytes/cuda_setup/main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib'), PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/cuda-11.6/include'), PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/cuda/extras/CUPTI/lib64')}
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0
CUDA SETUP: Highest compute capability among GPUs detected: 11.7
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary /home/work/.local/lib/python3.8/site-packages/bitsandbytes/bitsandbytes/libbitsandbytes_cuda117.so...
>>> |
rapsealk
changed the title
lizelive
reviewed
Apr 26, 2023
bitsandbytes/cuda_setup/main.py
Outdated
| CUDASetup.get_instance().add_log_entry(f'CUDA SETUP: libcudart.so path is {cudart_path}') | ||
| CUDASetup.get_instance().add_log_entry(f'CUDA SETUP: Is seems that your cuda installation is not in your path. See https://github.com/TimDettmers/bitsandbytes/issues/85 for more information.') | ||
| version = int(version.value) | ||
| version = torch._C._cuda_getCompiledVersion() |
There was a problem hiding this comment.
why not use torch.version.cuda here?
There was a problem hiding this comment.
@lizelive Thanks for the attention! I was trying to keep the rest of the code as much as possible. Actually, the whole method get_cuda_version() can be rewritten using torch.version.cuda instead, like torch.version.cuda.replace(".", "") as you mentioned.
|
Thanks. Maybe this works for me. My system
Add this to my .bashrc. Interestingly, But when I'm in any directory, |
|
@TimDettmers Hi! It seems that the |
|
@TimDettmers @rapsealk This makes a lot of sense. I'm happy to help. |
|
Thank you for this and sorry for being slow on this. This has been merged with #375 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR resolves #264
I have been struggling with using
bitsandbyteson my environment: nvcr.io/nvidia/pytorch:22.05-py3 image running on docker. When runningnvidia-smi, it successfully detects a GPU device.However, it fails to detect a GPU device when trying to import
bitsandbytes.After an investigation, I found that it fails to execute
libcudart.cudaGetDeviceCount()afterlibcuda.cuInit()with error code 304 - which means "OS call failed or operation not supported on this OS".https://github.com/TimDettmers/bitsandbytes/blob/9e7cdc9ea95e9756d9f5621a0e2c7e2538363fae/bitsandbytes/cuda_setup/main.py#L345
While I was looking for a way to solve this problem, I found that
PyTorchprovides similar APIs which work like a charm on my machine. So I replaced severallibcudaorlibcudartrelated commands withtorchones - I think it would be okay sincebitsandbytesalready has a dependency ontorch.