ScaleLLM Now Supports both CUDA 11.8 and CUDA 12.1

[guocuimi]

It's exciting to announce that ScalleLLM supports both CUDA 11.8 and CUDA 12.1. We've just released a new version specifically for this purpose. You can check it out here: New Release for CUDA 11.8 Support.

In response to user feedback regarding issues linked to NVIDIA driver versions, we've integrated support for CUDA 11.8. This update enables compatibility with older NVIDIA drivers, ensuring a smoother and more accessible experience for our users. We are dedicated to continuous improvement and highly value your feedback and suggestions. Please don't hesitate to reach out with any issues or advice you may have.

Currently, the Docker image is being built, and you can track its progress here: Docker Image Build Progress. Once completed, the new image tailored for CUDA 11.8 will be available in this repository: Docker Repository for CUDA 11.8 Image.

To update your Docker image, follow this example command:

docker run -it --gpus=all --net=host \
  -v $HOME/.cache/huggingface/hub:/models \
  -e HF_MODEL_ID=TheBloke/Llama-2-7B-chat-AWQ \
  -e DEVICE=cuda:0 \
  docker.io/vectorchai/scalellm_cu118:latest --logtostderr