Welcome to Lumina Discussions!

[jarroddavis68]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[erenious]

Hello, when I run the test I get the error "the specified module could not be found".Critical Initialization Error "Failed to load extracted Deps DLL: The specified module could not be found".

[jarroddavis68]

Hi, sorry to hear you are having issues. Let's see if we can get it sorted out.

First, I need some info about the device you are trying to run Lumina from:

• Windows version (run winver)
• Delphi version
• The timedate/size of Lumina.pas and Lumina.res
• Are you using the very latest repo version?

The error seems to indicate LoadLibrary is failing. Either the DLL is not physically there or it's the wrong bits. (64 bit app, trying to load a 32 bit dll for example).

Lumina is a 64bit application, can only run on 64bit Windows and must be Windows 10 or higher as the C libs it references are built against the Win10 SDK. Any older versions of Windows are not supported. If you are using an older version of Delphi, I know it can work on as low as Delphi 10.4 as reported by a user. It was built/tested on Windows 11, Delphi 12.2.

[erenious]

Hello again, thank you for your answer. I got the DLL error I mentioned with Windows 10 version 22H2, RAD Studio 12.2. I think it doesn't work with Windows 10. I'm using the latest repo version. 

When I tried it with Windows 11 version 24h2, it passed the DLL Load part successfully, but this time it gives the error "EExternalException: External exception E06D7363" on the line  FModel := llama_load_model_from_file( AsUtf8(FModelFilename), FModelParams). 

[erenious]

Processor: Intel(R) Xeon(R) CPU E5-2643 v4 @ 3.40GHz (24 CPUs), ~3.4GHz
Memory: 65536MB RAM
GPU : NVIDIA Quadro K620, ram 34733 mb, VRAM 2007 MB
I am trying to load the gemma-2-2b-it-abliterated-Q8_0.gguf model.

[jarroddavis68]

if this is ~ 2GB VRAM, then that model will not fit. The amount of VRAM must be the model size + context length size at minimum. Try lowering the context size, but I suspect it will make little difference. You will need to set the GPULayers to 0. Try setting the maxcontex to something like 512, gpulayers to 0.

[erenious]

Hello Jarrod, I was able to test it on another computer. Thank you for your help. I have a question I'd like to ask. My computer has 3 GPUs: one is onboard, one has 2 GB VRAM, and the other has 8 GB VRAM. When I set the GPU layer to -1, it selects the one with 2 GB VRAM. Can I select the GPU I want?

[jarroddavis68]

Hello Jarrod, I was able to test it on another computer. Thank you for your help.
You are most welcome.

I have a question I'd like to ask. My computer has 3 GPUs: one is onboard, one has 2 GB VRAM, and the other has 8 GB VRAM. When I set the GPU layer to -1, it selects the one with 2 GB VRAM. Can I select the GPU I want?

Ok, try this. I added a new AMainGPU param to TLumina.LoadModel. It defaults to 0, which is the first GPU device. However, setting it to -1 (which it defaults to in Lumina) is supposed to auto select the best GPU. So, I'm hoping it will select the one with the 8GB of VRAM. But if it does not, then just manually select it. 0 is the default GPU, will default to the onboard one, then 1 and 2 should select the others.

I have no way to officially test this as I only have a single GPU so I'm relying on your feedback to validate this. Once it has been vetted, I will push it to the repo. Thanks for your assistance.

Hovering the mouse over LoadModel should show updated docs.

Lumina.zip

In the future, create an issue for bugs or feature requests so it can be easily tracked and documented.

[erenious]

Thanks. When I set the AMainGPU parameter to -1, it automatically selected the best GPU.

[jarroddavis68]

Good to hear. Thanks. Working on a new inference library that should work better with all versions of Delphi. Stay tuned. Be sure to join my discord if you want to test early builds. Message me there and I will add you to the list.