You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We would like to support custom logit processors registered by users, so users can easily implement their custom sampling methods without the need to change the sglang code.
Steps
After the following line, allowing running some user code to process the logits
BTW, constrained decoding and function calling can also be implemented this way, such as by crafting a mask kernel.
The schema part can continue to use outlines or xgrammar.
We would like to support custom logit processors registered by users, so users can easily implement their custom sampling methods without the need to change the sglang code.
Steps
sglang/python/sglang/srt/layers/sampler.py
Line 37 in 5f12f0e
custom_params
toSamplingParams
sglang/python/sglang/srt/sampling/sampling_params.py
Line 43 in 9a00e6f
See also huggingface/transformers#14779
The text was updated successfully, but these errors were encountered: