TypeError: 'type' object is not subscriptable

[SvenPfiffner]

Hi there!
First of all many thanks to you for your great work on this lightweight and intuitive llama interface. I got it running and inferring within minutes and was more than impressed.

I ran into one small issue during setting it up though:

Bug:
Importing the package under python < 3.9 via

from llama_ai import LlamaAI

throws

(gguf_llama.py, line 55) - TypeError: 'type' object is not subscriptable

Suspected reason
Typing syntax in the function definition of create_embeddings is only supported from python 3.9 onwards

def create_embeddings(self, text:str) -> list[float]:

Fix
To get the package running on python < 3.9 (in my case 3.8.19), simply remove typing for the return type like this

def create_embeddings(self, text:str):

a cleaner fix could be to use the typing module there, which is compatible with previous python versions.

I am not sure whether compatibility with python < 3.9 is intended for this package at all, but I can verify that this small fix makes this amazing project work under python 3.8.

[laelhalawani]

Kind thanks for your input, I really appreciate it.
We could think up a version for 3.8, since it takes just that to make it compatible.
Did you have any other issues with 3.8?

[SvenPfiffner]

No, otherwise it works like a charm for me.

What I did to get it working on 3.8 was importing "List" from the already imported "typing" package and using that instead of pythons built-in "list", which got support for typing in 3.9. So the following two changes where all that was needed to have your package working under python 3.8

from typing import Any, Optional, Union, List
...
def create_embeddings(self, text:str) -> List[float]

instead of

from typing import Any, Optional, Union
...
def create_embeddings(self, text:str) -> list[float]

Thanks again for your amazing work!