Support top n sigma sampling

[Snowdar]

Motivation

Support a more complete samling options for decoding, especialy for openai compatible api.

Modifications

1. Add the rencent top-n-sigma samping as a decoding option and it will be applied to the logits after temperature applied when top_n_sigma > 0. The batch processing has been implemented.
2. Add top-k, min-p and top_n_sigma params for openai compatible api server (Note, please add these three params by extra_body when using openai SDK)

client example:

client = OpenAI(api_key='no key', base_url='http://localhost:30000/v1')
response = client.chat.completions.create(
      model='llama',
      messages=dict(role='user', content='hi'),
      temperature=0.3,
      max_tokens=128,
      top_p=0.95,
      frequency_penalty=0.,
      presence_penalty=0.,
      extra_body=dict(
          min_p=0.,
          top_k=-1,
          top_n_sigma=1, # enable it
      ),
)

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[merrymercy]

Thanks for the contribution. I left a few comments.

[Snowdar]

Thanks for the contribution. I left a few comments.

OK. All of these have completed. Could you please view it again?

[merrymercy]

top-k and min-p are fixed in #2499.
However, top n sigma is still not a common sampling strategy, we may not want to maintain it in the core right now.
For these things, it is better to support them via a custom logit processer interface. Can you help review this issue? #2291

I will close this for now.

[Snowdar]

top-k and min-p are fixed in #2499. However, top n sigma is still not a common sampling strategy, we may not want to maintain it in the core right now. For these things, it is better to support them via a custom logit processer interface. Can you help review this issue? #2291

I will close this for now.

So if the custom logit processer could be supported，then any other sampling codes will not be merged into sglang (considering sharing the custom samping to other people by the sglang itself)?