Switch to llama.cpp for gguf conversion

[prabhu]

Switched to llama.cpp master for gguf conversion and Q8_0 quantization. Works on my machine.

❯ ollama create cdx1-gguf -f Modelfile
gathering model components
copying file sha256:39ef2abdaad48bdbd1a8466c85b14a9646caf8049ec62e2b8d6383c56e035dc9 100%
parsing GGUF
using existing layer sha256:39ef2abdaad48bdbd1a8466c85b14a9646caf8049ec62e2b8d6383c56e035dc9
using autodetected template chatml
using existing layer sha256:a9f47bf267b28cae24d9d2b7e1d9204045da3b72568e4dcf5f66968e9d1fdf18
using existing layer sha256:de1eea7f14a061696a1490b000e7c36b45bb768c07a236acc828c69580770887
creating new layer sha256:6ee123ab45fc93d2d4223e606e0866d49b943b3826853a5db46fed97cdc50001
writing manifest
success

❯ ollama show cdx1-gguf
  Model
    architecture        llama
    parameters          14.7B
    context length      16384
    embedding length    5120
    quantization        Q8_0

  Parameters
    num_ctx        16384
    stop           "<|im_start|>"
    stop           "<|im_end|>"
    temperature    0.05
    top_k          10
    top_p          0.5

  System
    You are cdxgen, a CycloneDX and an xBOM expert.

  License
    apache-2.0

❯ ollama run cdx1-gguf
>>> tell me about cdxgen
cdxgen is a command-line tool that generates BOMs in various formats including JSON, XML, YAML, and CycloneDX. It supports multiple programming languages and frameworks.

>>>

[prabhu]

• Detailed docs on using the model with ollama