Load a lot of models in Triton

[mnaranjorion]

Good morning,
I'm trying to do a load of many models (20 models, although the number can grow) with Triton, but after about 40 sec, still during the initial load process, the process dies with the "Killed" message.
Not much more information is provided in the logs.
Is there any way to perform an orderly loading of models so that the process does not die?
What is the limit that Triton accepts?
Can it be configured to load according to the demand of these models?
Thank you very much
Best regards

[dyastremsky]

Have you tried loading with verbose logging enabled (--log-verbose=1)? That may help figure out what's going on. Triton does not have a limit beyond the limits of your environment (e.g. if it is running out of memory).

It sounds like for your use case, EXPLICIT model loading mode works best. See here. You can load the models and see when memory runs out, and you can load/unload as needed.