Replies: 3 comments 7 replies
-
Still no one knows if Nvidia Triton supports LLaMA2. |
Beta Was this translation helpful? Give feedback.
0 replies
-
You are working on it? This should be like a 10 minute task? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Can I run llama-2-7b-chat on Triton?
Any link to example code will be very helpful.
https://ai.meta.com/llama/
It must be the original "ckpt" version of the Meta LLaMA2 model that you download from their website: https://ai.meta.com/llama/
I know that Triton supports the Hugging Face version, but unfortunately their model is defective, the results are completely broken.
Beta Was this translation helpful? Give feedback.
All reactions