VLLM Sampling params

[HRashidi]

Feature Summary

• Add support to set other values for vllm sampling params

Justification/Rationale

• The user should be able set all available params supported by vllm like repetition_penalty, ...

Proposed Implementation (if any)

• Using vllm.sampling_params model as input directly instead of declaring the SamplingParam in the sdk (It can be too much arguments for the user)
• Adding a kw_dict for setting other values inside the SamplingParam

[movchan74]

Did we get a request for extra sampling parameters from someone?

Just to clarify why we are not using sampling params model from vLLM (vllm.sampling_params.SamplingParams):

1. vLLM uses msgspec for SamplingParams not Pydantic so it will not work well as parameter to the endpoints.
2. We want to support the same SamplingParams for different text generation deployments. Right now we support vLLM and HuggingFace Transformers. Soon HQQ Text Generation Models. So we don't want to be dependent on vLLM.