Commercial use - onnxruntime with CUDA

[mcicat]

Hello everyone,
if I would like to use onnxruntime for inferencing a model on a customer PC with cuda is it required to install Cuda Toolkit on that machine? (I initially thought that Cuda runtimes are included with the Onnxruntime).

If yes, that means that Visual Studio should also be installed on that machine. Is it legally okay to install the Visual Studio Community for this purpose? Community version is available for single developer projects in case of commercial use, but in this case it won't be used for development at all. It's just needed for the libraries...

[ytaous]

@faxu

[mcicat]

Visual Studio is a requirement for installing Cuda (as stated here https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html). Are you trying to say, that ORT will work if I install Cuda without VS (it is possible, but Cuda complains and notifies you, that it probably won't work), or that VS is not needed by the ORT itself?

[HectorSVC]

CUDA is a toolkit to help develop platform/application on Nvidia GPU, it relies on VS to build the code. If you just run Ort with CUDA , there's no need to have VS installed. Application developer's can have the required Ort lib and CUDA runtime libs packed into their package so that end user don't need to install it.

[mcicat]

Okay, I think I'm getting it. But to sum it up, just one last question. Will inference with Cuda work if I install Cuda without Visual Studio even tough VS is a requirement for Cuda?

[HectorSVC]

Yes. It will work.

[mcicat]

Thank You very much.