Which version of DIGITS are you using?

version 5.1-dev

I should precise that the gpu0 is busy by other frameworks such as tensorflow

can digits see that? I am not sure
if not it would be nice to get the GPU option in inference too

I should precise that the gpu0 is busy by other frameworks such as tensorflow

Ah, thanks for the clarification. No, DIGITS isn't aware of what other processes may be doing on the GPU. You should isolate GPUs for DIGITS with CUDA_VISIBLE_DEVICES=0,1 ./digits-devserver and for Tensorflow with CUDA_VISIBLE_DEVICES=2,3 python main.py, for example.

Thanks for the quick solution!

Still, the best would be to provide this option in the inference page to change the GPU on the fly.
Same feature exists in training :)

@lukeyeager I have an implementation of this that I can submit on a PR. but I don't know if the layout is okay for you. Could you please take a look at it?

If the user doesn't have more than one GPU, nothing will change:

But if the user has multiple GPUs, it looks like this:

where Next Available is the default choice and has the same behavior as the current implementation. If the user selects a GPU (only one can be selected), then this GPU will be used.

Do you think of any test I should do before submitting it? I did some tests and I've been using this for a while on our lab. Everything seems to work properly.

@rodrigoberriel Hello, will this make selection of GPU for inference from command line (curl.....) possible? As far as I know, you cannot decide for GPU at command line inference, no?

Thanks,

any news on this PR?

if in the long run the CPU option is also added it is even better :)

Wondering any updates on this? I still need inference applied by the first available gpu within digits environment but I think its locking inference to gpu0, in multigpu environments..?