an beginner for TGI, for to trigger the op which is written by triton, use which model-id, thanks #2759
Unanswered
alanguo1234
asked this question in
Q&A
an beginner for TGI, for to trigger the op which is written by triton, use which model-id, thanks
#2759
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
I have installed the TGI based on Nivida RTX4090, if hope to use text-generation-launcher to trigger the op which is written by triton(such as server/text_generation_server/layers/attention/flash_attn_triton.py), pls tell me the whole command, thanks .
Beta Was this translation helpful? Give feedback.
All reactions