new grpc server and client for new knobs.#54

Merged

SimonCqk merged 1 commit intokubedl-io:mainfrom ZHANGWENTAI:multi_framework_server

Oct 31, 2024

+1,526-8

Commits on Oct 31, 2024

new grpc server and client for tuning inference framework, dtype, input length and batch size on LLMs from huggingface
ZHANGWENTAI
committed