huggingface text-generation-inference Ideas · Discussions · GitHub

Is there a way to get GenerateResponse full json when calling sagemaker endpoints
Q&A nth-attempt

Sort by: Latest activity

Categories

Ideas Discussions

Share ideas for new features

You must be logged in to vote

Do we have any Grafana Dashboard example on top of TGI metrics ? Could metrics be explained in more detail ?

luk-cer started Oct 10, 2023 in Ideas · Closed

14
You must be logged in to vote

microsoft/Phi-3-vision-128k-instruct support

0Falli0 started May 24, 2024 in Ideas

0
You must be logged in to vote

Improve decode latency

scse-l started Jan 3, 2024 in Ideas

0