Base models supported by llama.cpp

[ggerganov]

This will be a live list containing all major base models supported by llama.cpp. Having this list will help maintainers to test if changes break some functionality in certain architectures. Please feel free to add more items - just don't add duplicates or finetunes

• LLaMa
  • 7B
    • F16: https://huggingface.co/meta-llama/Llama-2-7b-hf
    • Q: https://huggingface.co/TheBloke/Llama-2-7B-GGUF
  • 13B
    • F16: https://huggingface.co/meta-llama/Llama-2-13b-hf
    • Q: https://huggingface.co/TheBloke/Llama-2-13B-GGUF
• Mistral
  • 7B
    • F16: https://huggingface.co/mistralai/Mistral-7B-v0.1
    • Q: https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF
• Mixtral
  • 8x7B
    • F16: https://huggingface.co/mistralai/Mixtral-8x7B-v0.1
    • Q: https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF
• Phi-2
  • 2.7B
    • F16: https://huggingface.co/microsoft/phi-2
    • Q: https://huggingface.co/TheBloke/phi-2-GGUF
• Stable LM 2
  • 1.6B
    • F16: https://huggingface.co/stabilityai/stablelm-2-1_6b
    • Q: https://huggingface.co/second-state/stablelm-2-1.6b-GGUF
• ..

[ddh0]

• Stable LM 2 1.6B
  • F16: https://huggingface.co/stabilityai/stablelm-2-1_6b
  • Q: https://huggingface.co/second-state/stablelm-2-1.6b-GGUF

[saaraahfar]

Does this also mean that llama3 and mistral-7b v0.2 or v0.3 supported?

[teleprint-me]

Mistral v0.1, v0.2, and v0.3 all use the MistralForCausalLM architecture which are all based off of the LlamaForCausalLM architecture. Llama 3 still uses the same architecture, but the vocab was modified for some reason. If you use the huggingface models and the convert-hf script, the models will convert successfully.