mT5 directory structure

[hpk23]

I'm trying to use the mT5 model, for mT5, tensorrt-llm build creates an engine for encoder and decoder, how should I organize the directory structure in this case? (In all models, there seems to be only an example where there is only a decoder).

ensemble
 - 1
 - config.pbtxt

postprocessing
 - 1
  - model.py
 - config.pbtxt

preprocessing
 - 1
  - model.py
 - config.pbtxt

tensorrt_llm
 - 1
   - model.py
 - config.pbtxt

[symphonylyh]

Hi @hpk23 , we're working on a trition backend example for T5 structure. The progress is tracked under NVIDIA/TensorRT-LLM#800. We appreciate your patience for a few more weeks as we finalize the structure