Deep learning models implemented as web microservices
Docker containers with CUDA support are recommeded
(from: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
$ sudo apt-get update
$ sudo apt-get install -y nvidia-container-toolkit
$ sudo docker run --rm --gpus all ubuntu:22.04 nvidia-smi
$ cd docker_img_files/image_classifier_server
$ docker build --tag image_classifier .
$ docker volume create \
--driver local \
-o o=bind \
-o type=none \
-o device="${HOME}/.docker_volues" \
image_classifier_volume
$ docker volume list
NOTE: for the first run, models are downloaded. This can take some time (models are in GBs)
Run bash
docker run -it --rm --gpus all -p 8000:8000 -v image_classifier_volume:/root image_classifier bash
Run as daemon
$ docker run -d --gpus all -p 8000:8000 -v image_classifier_volume:/root image_classifier
$ python image_classifier_test.py --img_path path/to/image