Sadly, I don't have time to support this package at this moment, I recommend using https://github.com/SciSharp/LLamaSharp that supports the latest version of llama.cpp
Connect llama.cpp to Unity3d in two clicks
The bindings were made for b1518 version of llama.cpp, so if you need libraries for your OS, build from sources from release b1518 You can find Windows CUDA llama.cpp dll here - https://github.com/SciSharp/LLamaSharp
- Add git repo as package Window -> Package Manager -> Add from Git URL https://github.com/mrtrizer/UnityLlamaCpp.git
- Download a GGUF model, for example this - https://huggingface.co/TheBloke/speechless-mistral-dolphin-orca-platypus-samantha-7B-GGUF/blob/main/speechless-mistral-dolphin-orca-platypus-samantha-7b.Q4_K_M.gguf
- Put model file in StreamingAssets/Models
- Find Test.prefab in package dir and use component context menu to Run it, it should generate some response to a prompt
- Use LlamaExample.cs as and example