-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[doc] Add LMI Text Embedding Inference user guide #2022
Conversation
|
||
You can leverage LMI Text Embedding inference using the following starter configurations: | ||
|
||
### serving.properties |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not promote this usage, we should guide user to:
- Use djl:// model if the model already exist
- use HF_MODEL_ID to convert the model at runtime
- Manually import the model into DJL model format with our djl_convert/djl_import tool
- and finally fully customize with
serving.properties
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the user give HF_MODEL_ID, then the code will automatically convert for them.
a457f74
to
73f18e7
Compare
|
||
``` | ||
OPTION_ENGINE=OnnxRuntime | ||
MODEL_URL=<your model url> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MODEL_URL=<your model url> | |
HF_MODEL_ID=djl://ai.djl.huggingface.onnxruntime/BAAI/bge-base-en-v1.5 |
You can specify the djl:// model url to load a model from the DJL model zoo. | ||
|
||
``` | ||
OPTION_ENGINE=OnnxRuntime |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need OPTION_ENGINE djl model zoo
1.1664978, | ||
0.79496926, | ||
0.28931668, | ||
1.2245488, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is too long, could we trim into ... in the middle? And also show the expected shape
Co-authored-by: Frank Liu <frankfliu2000@gmail.com>
Description
Brief description of what this PR is about