Add openai embedding API

[Ying1123]

This is a follow-up PR for

• Add llama embedding modules [unreachable code] - step 1/3 #983
• Add io struct for embedding models [unreachable code] - step 2/3 #987
• Add e5-mistral embedding model - step 3/3 #988

For embedding API, input as a list will be covered in the next PR.

[yichuan520030910320]

LGTM

[yichuan520030910320]

btw, I have one small question, why switch the max_num_tokens from 1 to 0?

[Ying1123]

btw, I have one small question, why switch the max_num_tokens from 1 to 0?

Did you mean switch from 0 to 1? This is because our default logic for generation model suppose it will generate 1 token after prefill. Although embedding models do not generate tokens, I added one dummy token to cheat with the shared part of the code.

[yichuan520030910320]

btw, I have one small question, why switch the max_num_tokens from 1 to 0?

Did you mean switch from 0 to 1? This is because our default logic for generation model suppose it will generate 1 token after prefill. Although embedding models do not generate tokens, I added one dummy token to cheat with the shared part of the code.

I see, thanks for answering